Simple Queue Service Flashcards
Why to use SQS?
Traffic control(my understanding); Decoupling application processes. Application scalability; Guaranteed message delivery; Message order guarantee; Asynchronous data processing; Building resilience in the system; Bring elasticity to application; analyzing data flow and performance of the processes. Building redundancy with SQS;
Main Features of SQS
redundant infrastructure: at least once and FIFO message delivered at least once and exactly once.
Multiple producers and consumers.
Queue wise configurable settings.
Variable message size. maximum 256KB. if need larger size, store it in S3 or DynamoDB. SQS holds the pointer to the S3 object in the queue.
Queue access control.
Delay queue:
Payment Card industry compliance.
Health Insurance Portability and Accountability Act compliance.
Types of Queue
Standard Queues and FIFO queues.
Standard Queue: The Order may change; available in all QWS regions. supports almost all unlimited number of TPS with each API action. message may be delivered more than once. used when throughput is more importance than order.
FIFO: order is fixed. available only in US East, US West, EU(Ireland). supports limited throughput. up to 300 messages per second without any batching. with batch size of 10 messages, it can support up to 3000 messages per second. message delivered exactly once. when order is more important than the throughput. ending with .fifo
Dead Letter Queue
for storing failed messages.
The dead-letter queue of a FIFO queue must also be a FIFO queue. Similarly, the dead-letter queue of a standard queue must also be a standard queue.
Queue Attributes
Default visibility Timeout: 12 hours the length of time that a message is received from a queue will be invisible to other receiving components.
Message Retention Period: 1 minute – 14 days
Maximum Message Size: 256KB
Delivery Delay: 0-15 minutes
Receive Message wait time: 0-20 seconds – if set to a value > 0, long polling is enabled.
Content-Based Deduplication:
SQS Limits
In flight messages per queue – queue – standard: maximum 120,000 inflight messages; FIFO: 20,000. returns overLimit error if exceeded.
Queue Name – queue – max 80 characters(hyphen and underscore are OK).
Message attributes – 10 metadata attributes
Message Batch – 10 messages per batch
Message content – XML, JSON, and any unformatted text.
Message retention – min – 1 minute – 14 days, default is 4 days.
message size – min: 1 byte; max: 256KB
message visibility timeout – max 12 hours
policy information – max: 8192 bytes, 20 statements, 50 principals, or 10 conditions.
SQS vs SNS
pulling vs pushing
message Lifecycle
component 1 sends message A to a queue, and then the message is redundantly distributed across SQS servers;
- when component 2 is ready, it retrieves the message from SQS. while message A is being processed, it remains in the queue but is not returned to any subsequent request for the duration of the visibility timeout.
- component 2 deletes the message from the queue during that visibility timeout, or else it will get processed again.
visibility time out
used to block other components from processing a message.
can be controlled via the SQS API.
Long Polling vs Short Polling
Short polling returns a result immediately, even if the queue is empty. It also only checks a subset of servers, which could cause false empty responses.
Long polling does not return a response until there is a message in the queue. It also checks every server to avoid false empty responses.
Long polling should be used when possible, because it can reduce costs and false empty responses.
Using API
ChangeMessageVisibility – change a message’s visibility timeout.
SetQueueAttributes – change a queue visibility timeout.
ReceiveMessageWaitTimeSeconds – >0 enable long polling at the queue level.
SendMessage – send a message to a queue, can be used to set the message’s delay, as well as message attributes, and the message body.
delete a message;
GetQueueURL,
Advantage from using a queue
§ Components are loosely coupled for high availability, reliability, and
scalability
§ Protection against losing data on application failure
How is Amazon SQS different from Amazon MQ?
Amazon MQ – If you’re using messaging with existing applications, and want to move your messaging to the cloud quickly and easily; It supports industry-standard APIs and protocols so you can switch from any standards-based message broker to Amazon MQ without rewriting the messaging code in your applications.
SQS/SNS – If you are building brand new applications in the cloud, we recommend you consider Amazon SQS and Amazon SNS.
How is Amazon SQS different from Amazon Kinesis Streams?
SQS– Decoupling the components of an application:
Configuring individual message delay:
Dynamically increasing concurrency or throughput at read time:
Scaling transparently: – Routing related records to the same record processor:
Allowing multiple applications to consume the same stream concurrently:
SQS free Tier
The Amazon SQS Free Tier provides you with 1 million requests per month at no charge.
