AWS - Integration Flashcards
What are the most popular AWS integration services?
AWS - SQS, SNS and Amazon MQ, Amazon EventBridge, the three primary services that are used in modern applications to integrate with other applications or microservices
What is Amazon MQ?
It is a managed message broker service that makes it easy to set up and operate message brokers in the cloud.
It allows asynchronous integration to applications.
AWS MQ manages the administration and maintenance of ActiveMQ, a popular open-source message broker.
What is SNS?
SNS - Simple Notification Service, Pub/Sub messaging solution
It allows businesses to fully manage SMS, email, pub/sub messaging, as well as mobile push notifications. This messaging service allows both application to application and person to application messaging.
What is SQS?
Amazon SQS is AWS’s Simple Queue Service, allowing users to manage message queues— whether they’re working with a distributed system, microservice, or serverless application. Application components can then send and receive messages among one another in staggering volumes.
Point-to-point integration
What is AWS SWF?
SWF - Simple WorkFlow Service
It is used to trigger a workflow. For example, imagine that you have an expense report that needs to be approved by three people. When you submit an experience report the workflow is triggered and the workflow services keep track of each individual who needs to approve your expenses report.
What are the 2 types of SQS queues?
What is the difference between them?
[1] TYPES of SQS queues:
a) standard (default queue)
b) FIFO
[2] DIFFERENCES:
a) standards queues provide best-effort ordering. It means it tries to deliver the message in the same order as they are sent. However, occasionally more than one copy of a message might be delivered out of order.
Fifo queues, on other hand, offer first-in-first-out delivery. The order is preserved.
b) Standard queues grantees at least one delivery but sometimes duplicate or more than one copy can be delivered.
FIFO queues ensure a message is delivered exactly once and remains available until the consumer processes and deletes it.
c) Standard queues offer unlimited throughput. It supports nearly-unlimited number of transactions per second.
FIFO queues on other hand are limited to 300 transactions per second (300 TPS)
What is SQS visibility timeout?
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html
https://medium.com/event-driven-utopia/aws-sqs-visibility-timeout-explained-c13d8a728ab5
When the producer sends a message to SQS, it is stored in the queue until consumed by a consumer. When the consumer is ready, it polls SQS for new messages and ultimately receives the message.
Once a message is received by a consumer, SQS doesn’t automatically delete the message. Because there’s no way for SQS to guarantee that the message has been received by the consumer. The message might get lost in transit or the consumer can fail while processing the message.
So the consumer must delete the message from the queue after receiving and processing it.
While a consumer is processing a message in the queue, SQS temporary hides the message from other consumers.
This is done by setting a visibility timeout on the message, a period of time during which SQS prevents other consumers from receiving and processing the message.
The default visibility timeout for a message is 30 seconds.
How it works?
The visibility timeout begins when SQS hands over a message to the consumer. During this time, the consumer has to do two things.
- It has to complete the processing of the message.
- Delete the message from the queue.
However, if the consumer fails before deleting the message, the visibility timeout will expire and the message becomes visible to other consumers for receiving.
If a message must be received only once, the consumer should delete it within the duration of the visibility timeout.
When to use Standard SQS and when FIFO queues?
[1] STANDARD QUEUES USE CASES
A) Standard queues can be used in any scenario, as long as the application can process messages that arrive more than once and are out of order.
B) Decouple live user requests from intensive background work: Let users upload media while resizing or encoding it.
C) Batch messages for future processing: Schedule multiple entries to be added to a database.
[2] FIFO QUEUE USE CASES:
A) FIFO queues are designed to enhance messaging between applications when the order of operations and events is critical, or where duplicates can’t be tolerated.
B) Ensure that user-entered commands are executed in the right order.
C)
When to use Standard SQS and when FIFO queues?
[1] STANDARD QUEUES USE CASES
A) Standard queues can be used in any scenario, as long as the application can process messages that arrive more than once and are out of order.
B) Decouple live user requests from intensive background work: Let users upload media while resizing or encoding it.
C) Batch messages for future processing: Schedule multiple entries to be added to a database.
[2] FIFO QUEUE USE CASES:
A) FIFO queues are designed to enhance messaging between applications when the order of operations and events is critical, or where duplicates can’t be tolerated.
B) Ensure that user-entered commands are executed in the right order.
C)Display the correct product price by sending price modifications in the right order.
What is the difference between SQS and SNS?
[1]
SQS - Simple Queue Service
SNS - Simple Notification Service
[2]
SNS is publish-subscribe service
SQS is distributed queueing service
[3]
SNS is a notification service that allows you to send individual messages or bulk messages to a large number of recipients.
Amazon SNS makes it simple and cost effective to send push notification to mobile devices, email recipients or even send messages to other distributed services
Amazon SQS is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
SQS is not push based system it is pull based services. Receivers have to poll SQS to receive messages.
!!! In SQS messages can’t be received by multiple receivers at the same time.
Polling inherently introduces some latency in message delivery in SQS unlike SNS where messages are immediately pushed to subscribers.
REASSUMING
a) Message consumption
SQS: Pull mechanism - consumer poll messages from queue
SNS: Push mechanism - SNS pushes messages to consumers.
b) Persistence
SQS: Messages are persisted for some duration if no consumer is available. The retention period value is from 1 minute to 14 days.
SNS: No persistence. Whichever consumer is present at the time of message arrival, gets the message and the message is deleted. If no consumer is available then the message is lost.
c) Consumer type
SQS: All the consumers are supposed to be identical and hence process the messages in exact same way.
SNS: All the consumers are (supposed to be) processing the messages in a different ways.
What is the fanout integration pattern?
How it can be implemented using AWS SQS and SNS?
Fan-out is a messaging pattern where messages are broadcast in a one-to-many destinations and not halting the process that executes the messaging to wait for any response to that message..
