Messaging and Kinesis Flashcards
What is SQS?
Simple Queue Service
Q: How is Amazon SQS different from Amazon Simple Notification Service (SNS)?
Amazon SNS allows applications to send time-critical messages to multiple subscribers through a “push” mechanism, eliminating the need to check periodically or “poll” for updates. Amazon SQS is a message queue service used by distributed applications to exchange messages through a polling model and can be used to decouple sending and receiving components.
Does Amazon SQS provide message ordering?
Yes. FIFO (first-in-first-out) queues preserve the exact order in which messages are sent and received. If you use a FIFO queue, you don’t have to place sequencing information in your messages. For more information, see FIFO Queue Logic in the Amazon SQS Developer Guide.
Standard queues provide a loose-FIFO capability that attempts to preserve the order of messages. However, because standard queues are designed to be massively scalable using a highly distributed architecture, receiving messages in the exact order they are sent is not guaranteed.
Does Amazon SQS guarantee delivery of messages?
Standard queues provide at-least-once delivery, which means that each message is delivered at least once.
FIFO queues provide exactly-once processing, which means that each message is delivered once and remains available until a consumer processes it and deletes it. Duplicates are not introduced into the queue.
What is a visibility timeout?
The visibility timeout is a period of time during which Amazon SQS prevents other consuming components from receiving and processing a message.
when a message is polled by a consumer, it becomes invisible to other customers. E.g., if the message visibility timeout is 30 seconds, it means that message will not be visible to other consumers for 30 seconds after it is being polled. After 30 seconds, a message will be visible for other consumers to process.
If a consumer needs more time to process the message, it can call the change message visibility API to get more time. It will inform SQS to extend the visibility timeout and now and not make this visible to other consumers to process
How SQS can scale?
Consumer polling messages from the queue can be deployed on EC2 instances as part of ASG. Cloud watch can be configured to trigger an alarm if length of SQS reaches a certain threshold. The alarm ApproximateNumberOfMessages can be used for this purpose. The alarm can trigger an autoscale group to add more EC2 instances
how SQS can help in decoupling different application tiers?
let’s take an example of a video processing application. when A user submits the request for processing a video, the request can be taken in the form of SQS message. The SQS message it can be processed by the back-end processing application when it has the resources available. This is how application tiers can be decoupled.
what is a long poll?
the consumer can be configured a message to arrive if there is none in the queue. This helps in decreasing the number of API calls made to SQS while increasing the efficiency and latency of an application.
The wait time can be between one second to 20 seconds
what is SQS FIFO queue
FIFO queue ensures the ordering of messages in the queue. The messages are delivered to the consumer in the order they were received. It limits throughput to 300 messages/s without batching and 3000 messages with batching. Exactly once sound capability for the messages is implemented in FIFO.
How to use SQS as a buffer to database rights
If the application is writing a transaction in the database then the database has to be available for transactions. If something goes wrong in the database, the transaction may fail. SQS can be used in the middleware. An application can send a request in the form of messages to SQS (with infinitely scalable) and then the consumer poll the messages and tries to insert them into the database. This will ensure that all the requests or transactions get written to the database.
How to use SQS as a buffer to database rights
If the application is writing a transaction in the database then the database has to be available for transactions. If something goes wrong in the database, the transaction may fail. SQS can be used in the middleware. An application can send a request in the form of messages to SQS (with infinitely scalable), and then the consumer poll the messages and tries to insert them into the database. This will ensure that all the requests or transactions get written to the database.
What is Amazon SNS service?
Sns service is a pub sub-service in which even producer only send messages to 1SNS topic. Many subscribers may listen to those messages via subscribing to SNS topic. Each subscriber receives all the messages. They can be 1.2 million subscriber per topic
What kind of subscribers can be there for SNS?
Subscribers can be an email address, as email, mobile notification, HTTP endpoints.
What is SNS Fanout pattern?
The Fanout scenario is where a message published to an SNS topic is replicated and pushed to multiple endpoints, such as Kinesis Data Firehose delivery streams, Amazon SQS queues, HTTP(S) endpoints, and Lambda functions. This allows for parallel asynchronous processing.
How SNS message filtering work?
JSON policy can be used to filter the messages sent to SNS topic subscriptions. If a subscription does not have a filter policy it receives every message. For example: if an order has a state as new, it can be sent to the SQS queue for new orders. If the order is canceled, it can be sent to the SQS queue for canceled orders.
