Application Integration | Amazon Simple Queue Service (SQS) Flashcards
What is Amazon SQS?
Overview
Amazon Simple Queue Service (SQS) | Application Integration
Amazon Simple Queue Service (Amazon SQS) is a web service that gives you access to message queues that store messages waiting to be processed. With Amazon SQS, you can quickly build message queuing applications that can run on any computer.
Amazon SQS offers a reliable, highly-scalable, hosted queue for storing messages in transit between computers. With Amazon SQS, you can move data between diverse, distributed application components without losing messages and without requiring each component to be always available.
Amazon SQS can help you build a distributed application with decoupled components, working closely with the Amazon Elastic Compute Cloud (Amazon EC2) and other AWS infrastructure web services.
What can I do with Amazon SQS?
Overview
Amazon Simple Queue Service (SQS) | Application Integration
Because Amazon SQS is highly-scalable, you pay only for what you use. You can start small and grow your application alongside your business needs, with no performance or reliability compromises. Amazon SQS lets you stop worrying about how your messages are stored and managed and helps you focus on building robust, sophisticated message-based applications.
Here are just a few ideas:
Integrate Amazon SQS with other AWS services to make applications more flexible and reliable.
Use Amazon SQS to create work queues with each message as a task to be completed by a process. Let one (or many) computers read tasks from the message queue and then process them.
Build a microservice architecture and use message queues to connect your microservices.
Keep notifications of significant business events in an Amazon SQS message queue. Each event can have a corresponding message in a message queue, and applications that need to be aware of the event can read and process the messages.
How can I get started using Amazon SQS?
Overview
Amazon Simple Queue Service (SQS) | Application Integration
You can create an Amazon SQS queue and send a message in a few steps by completing our 10-minute tutorial, Send Messages Between Distributed Applications.
For additional detail, see the Amazon SQS Developer Guide, and sample code in the Resource Center.
What are the benefits of Amazon SQS over homegrown or packaged message queuing systems?
Overview
Amazon Simple Queue Service (SQS) | Application Integration
Amazon SQS provides several advantages over building your own software for managing message queues or using commercial or open-source message queuing systems that require significant up-front time for development and configuration.
These alternatives require ongoing hardware maintenance and system administration resources. The complexity of configuring and managing these systems is compounded by the need for redundant storage of messages that ensures messages are not lost if hardware fails.
In contrast, Amazon SQS requires no administrative overhead and little configuration. Amazon SQS works on a massive scale, processing billions of messages per day. You can scale the amount of traffic you send to Amazon SQS up or down without any configuration. Amazon SQS also provides extremely high message durability, giving you and your stakeholders added confidence.
How is Amazon SQS different from Amazon SNS?
Overview
Amazon Simple Queue Service (SQS) | Application Integration
Amazon Simple Queue Service (SQS) and Amazon SNS are both messaging services within AWS, which provide different benefits for developers. Amazon SNS allows applications to send time-critical messages to multiple subscribers through a “push” mechanism, eliminating the need to periodically check or “poll” for updates. Amazon SQS is a message queue service used by distributed applications to exchange messages through a polling model, and can be used to decouple sending and receiving components. Amazon SQS provides flexibility for distributed components of applications to send and receive messages without requiring each component to be concurrently available.
A common pattern is to use SNS to publish messages to Amazon SQS queues to reliably send messages to one or many system components asynchronously. For more details, see What is Pub/Sub Messaging?
How is Amazon SQS different from Amazon MQ?
Overview
Amazon Simple Queue Service (SQS) | Application Integration
Amazon MQ, Amazon SQS, and Amazon SNS are messaging services that are suitable for anyone from startups to enterprises. If you’re using messaging with existing applications, and want to move your messaging to the cloud quickly and easily, we recommend you consider Amazon MIt supports industry-standard APIs and protocols so you can switch from any standards-based message broker to Amazon MQ without rewriting the messaging code in your applications. If you are building brand new applications in the cloud, we recommend you consider Amazon SQS and Amazon SNS. Amazon SQS and SNS are lightweight, fully managed message queue and topic services that scale almost infinitely and provide simple, easy-to-use APIs. You can use Amazon SQS and SNS to decouple and scale microservices, distributed systems, and serverless applications, and improve reliability.
Overview
Amazon Simple Queue Service (SQS) | Application Integration
Does Amazon SQS provide message ordering?
Overview
Amazon Simple Queue Service (SQS) | Application Integration
Yes. FIFO (first-in-first-out) queues preserve the exact order in which messages are sent and received. If you use a FIFO queue, you don’t have to place sequencing information in your messages. For more information, see FIFO Queue Logic in the Amazon SQS Developer Guide.
Standard queues provide a loose-FIFO capability that attempts to preserve the order of messages. However, because standard queues are designed to be massively scalable using a highly distributed architecture, receiving messages in the exact order they are sent is not guaranteed.
Does Amazon SQS guarantee delivery of messages?
Overview
Amazon Simple Queue Service (SQS) | Application Integration
Standard queues provide at-least-once delivery, which means that each message is delivered at least once.
FIFO queues provide exactly-once processing, which means that each message is delivered once and remains available until a consumer processes it and deletes it. Duplicates are not introduced into the queue.
How is Amazon SQS different from Amazon Kinesis Streams?
Overview
Amazon Simple Queue Service (SQS) | Application Integration
Amazon SQS offers a reliable, highly-scalable hosted queue for storing messages as they travel between applications or microservices. It moves data between distributed application components and helps you decouple these components. Amazon SQS provides common middleware constructs such as dead-letter queues and poison-pill management. It also provides a generic web services API and can be accessed by any programming language that the AWS SDK supports. Amazon SQS supports both standard and FIFO queues.
Use Amazon SQS when you need each unique message to be consumed only once and for cases such as the following:
Decoupling the components of an application: You have a queue of work items and want to track the successful completion of each item independently. Amazon SQS tracks the ACK/FAIL results, so the application does not have to maintain a persistent checkpoint or cursor. After a configured visibility timeout, Amazon SQS deletes acknowledged messages and redelivers failed messages.
Configuring individual message delay: You have a job queue and you need to schedule individual jobs with a delay. With Amazon SQS, you can configure individual messages to have a delay of up to 15 minutes.
Dynamically increasing concurrency or throughput at read time: You have a work queue and want to add more readers until the backlog is cleared. With Amazon Kinesis Streams, you can scale up to a sufficient number of shards (however, you must provision enough shards ahead of time). Amazon SQS requires no pre-provisioning.
Scaling transparently: You buffer requests and the load changes as a result of occasional load spikes or the natural growth of your business. Because Amazon SQS can process each buffered request independently, Amazon SQS can scale transparently to handle the load without any provisioning instructions from you.
Amazon Kinesis Streams allows real-time processing of streaming big data and the ability to read and replay records to multiple Amazon Kinesis Applications. The Amazon Kinesis Client Library (KCL) delivers all records for a given partition key to the same record processor, making it easier to build multiple applications that read from the same Amazon Kinesis stream (for example, to perform counting, aggregation, and filtering).
Use Amazon Kinesis Streams when you need multiple consumers to be able to process each record and for use cases such as the following:
Routing related records to the same record processor: You stream MapReduce. Actions such as counting and aggregation are simpler when all records for a given key are routed to the same record processor.
Allowing multiple applications to consume the same stream concurrently: You have one application that updates a real-time dashboard and another that archives data to Amazon Redshift. You want both applications to consume data from the same stream concurrently and independently.
Does Amazon use Amazon SQS for its own applications?
Billing
Amazon Simple Queue Service (SQS) | Application Integration
Yes. Developers at Amazon use Amazon SQS for a variety of applications that process large numbers of messages every day. Key business processes in both Amazon.com and Amazon Web Services use Amazon SQS.
How much does Amazon SQS cost?
Billing
Amazon Simple Queue Service (SQS) | Application Integration
You pay only for what you use, and there is no minimum fee.
The cost of Amazon SQS is calculated per request, plus data transfer charges for data transferred out of Amazon SQS (unless data is transferred to Amazon EC2 instances or to AWS Lambda functions within the same region). For detailed pricing breakdowns per queue type and region, see Amazon SQS Pricing.
What can I do with the Amazon SQS Free Tier?
Billing
Amazon Simple Queue Service (SQS) | Application Integration
The Amazon SQS Free Tier provides you with 1 million requests per month at no charge.
Many small-scale applications are able to operate entirely within the limits of the Free Tier. However, data transfer charges might still apply. For more information, see Amazon SQS Pricing.
The Free Tier is a monthly offer. Free usage does not accumulate across months.
Will I be charged for all Amazon SQS requests?
Billing
Amazon Simple Queue Service (SQS) | Application Integration
Yes, for any requests beyond the free tier. All Amazon SQS requests are chargeable, and they are billed at the same rate.
Do Amazon SQS batch operations cost more than other requests?
Billing
Amazon Simple Queue Service (SQS) | Application Integration
No. Batch operations (SendMessageBatch, DeleteMessageBatch, and ChangeMessageVisibilityBatch) all cost the same as other Amazon SQS requests. By grouping messages into batches, you can reduce your Amazon SQS costs.
How will I be charged and billed for my use of Amazon SQS?
Billing
Amazon Simple Queue Service (SQS) | Application Integration
There are no initial fees to begin using Amazon SQS. At the end of the month, your credit card will be automatically charged for the month’s usage.
You can view your charges for the current billing period at any time on the AWS website:
Log into your AWS account.
Under Your Web Services Account, select Account Activity.
How can I track and manage the costs associated with my Amazon SQS queues?
Billing
Amazon Simple Queue Service (SQS) | Application Integration
You can tag and track your queues for resource and cost management using cost allocation tags. A tag is a metadata label comprised of a key-value pair. For example, you can tag your queues by cost center and then categorize and track your costs based on these cost centers.
For more information, see Tagging Your Amazon SQS Queues in the Amazon SQS Developer Guide. For more information on cost allocation tagging of AWS resources, see Using Cost Allocation Tags in the AWS Billing and Cost Management User Guide.
Do your prices include taxes?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
Except as noted otherwise, our prices don’t include any applicable taxes and duties such as VAT or applicable sales tax.
For customers with a Japanese billing address, the use of AWS in any region is subject to Japanese Consumption Tax. For more information, see the Amazon Web Services Consumption Tax FAQ.
Can I use Amazon SQS with other AWS services?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
Yes. You can make your applications more flexible and scalable by using Amazon SQS with compute services such as Amazon EC2, Amazon EC2 Container Service (Amazon ECS), and AWS Lambda, as well as with storage and database services such as Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB.
One common use case is a distributed, decoupled application whose multiple components and modules need to communicate with each other, but can’t do the same amount of work simultaneously. In this case, Amazon SQS message queues carry messages to be processed by the application running on Amazon EC2 instances.
The Amazon EC2 instances can read the message queue, process the job, and then post the results as messages to another Amazon SQS message queue (for example, for further processing by another application). Because Amazon EC2 allows applications to scale up and down dynamically, application developers can vary the number of compute instances based on the amount of messages in the Amazon SQS queues using Auto Scaling, to ensure that jobs are executed in a timely manner.
Can you give me an example use case for Amazon SQS?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
Here is how a video transcoding website uses Amazon EC2, Amazon SQS, Amazon S3, and Amazon DynamoDB together:
End users submit videos to be transcoded to the website.
The videos are stored in Amazon S3, and a request message is placed in an incoming Amazon SQS queue with a pointer to the video and to the target video format within the message.
The transcoding engine that runs on a set of Amazon EC2 instances reads the request message from the incoming queue, retrieves the video from Amazon S3 using the pointer, and transcodes the video into the target format.
The converted video is put back into Amazon S3 and another response message is placed in another outgoing Amazon SQS queue with a pointer to the converted video.
At the same time, metadata about the video (format, date created, length, and so on) is indexed into Amazon DynamoDB for querying.
During this workflow, a dedicated Auto Scaling instance can constantly monitor the incoming queue. Based on the number of messages in the incoming queue, the Auto Scaling instance dynamically adjusts the number of transcoding Amazon EC2 instances to meet the response time requirements of the website’s customers.
How can I interact with Amazon SQS?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
You can access Amazon SQS using the AWS Management Console, which helps you create Amazon SQS queues and send messages easily.
Amazon SQS also provides a web services API. It is also integrated with the AWS SDKs, allowing you to work in the programming language of your choice.
What are the available operations for message queues?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
For information on message queue operations, see Amazon SQS Product Details.
Who can perform operations on a message queue?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
Only an AWS account owner (or an AWS account that the account owner has delegated rights to) can perform operations on an Amazon SQS message queue.
Can I use Java Message Service (JMS) with Amazon SQS?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
Yes. You can take advantage of the scale, low cost, and high availability of Amazon SQS without the worry and high overhead of running your own JMS cluster.
Amazon provides the Amazon SQS Java Messaging Library that implements the JMS 1.1 specification and uses Amazon SQS as the JMS provider. For more information, see Using JMS with Amazon SQS in the Amazon SQS Developer Guide.
How does Amazon SQS identify messages?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
All messages have a global unique ID that Amazon SQS returns when the message is delivered to the message queue. The ID isn’t required to perform any further actions on the message, but it is useful for tracking the receipt of a particular message in the message queue.
When you receive a message from the message queue, the response includes a receipt handle that you must provide when deleting the message.
How does Amazon SQS handle unsuccessfully-processed messages?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
In Amazon SQS, you can use the API or the console to configure dead letter queues, which are queues that receive messages from other source queues.
If you make a queue into a dead letter queue, it receives messages after a maximum number of processing attempts cannot be completed. You can use dead letter queues to isolate messages that can’t be processed for later analysis.
For more information, see “Can I use a dead letter queue with FIFO queues?” on this page and Using Amazon SQS Dead Letter Queues in the Amazon SQS Developer Guide.
What is a visibility timeout?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
The visibility timeout is a period of time during which Amazon SQS prevents other consuming components from receiving and processing a message. For more information, see Visibility Timeout in the Amazon SQS Developer Guide.
How does Amazon SQS allow multiple readers to access the same message queue without losing messages or processing them multiple times?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
Every Amazon SQS queue has a configurable visibility timeout. A message is not visible to any other reader for a designated amount of time when it is read from a message queue. As long as the amount of time it takes to process the message is less than the visibility timeout, every message is processed and deleted.
If the component processing of the message fails or becomes unavailable, the message again becomes visible to any component reading the message queue once the visibility timeout ends. This allows multiple components to read messages from the same message queue, each one working to process different messages.
What is the maximum limit for message visibility?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
The maximum visibility timeout for an Amazon SQS message is 12 hours.
Does Amazon SQS support message metadata?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
Yes. An Amazon SQS message can contain up to 10 metadata attributes. You can use message attributes to separate the body of a message from the metadata that describes it. This helps process and store information with greater speed and efficiency because your applications don’t have to inspect an entire message before understanding how to process it.
Amazon SQS message attributes take the form of name-type-value triples. The supported types include string, binary, and number (including integer, floating-point, and double). For more information, see Using Amazon SQS Message Attributes in the Amazon SQS Developer Guide.
How can I determine the time-in-queue value?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
To determine the time-in-queue value, you can request the SentTimestamp attribute when receiving a message. Subtracting that value from the current time results in the time-in-queue value.
What is the typical latency for Amazon SQS?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
Typical latencies for SendMessage, ReceiveMessage, and DeleteMessage API requests are in the tens or low hundreds of milliseconds.
For anonymous access, what is the value of the SenderId attribute for a message?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
When the AWS account ID is not available (for example, when an anonymous user sends a message), Amazon SQS provides the IP address.
What is Amazon SQS long polling?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
Amazon SQS long polling is a way to retrieve messages from your Amazon SQS queues. While the regular short polling returns immediately, even if the message queue being polled is empty, long polling doesn’t return a response until a message arrives in the message queue, or the long poll times out.
Long polling makes it inexpensive to retrieve messages from your Amazon SQS queue as soon as the messages are available. Using long polling might reduce the cost of using SQS, because you can reduce the number of empty receives. For more information, see Amazon SQS Long Polling in the Amazon SQS Developer Guide.
Is there an additional charge for using Amazon SQS long polling?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
No. Long-polling ReceiveMessage calls are billed exactly the same as short-polling ReceiveMessage calls.
When should I use Amazon SQS long polling, and when should I use Amazon SQS short polling?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
In almost all cases, Amazon SQS long polling is preferable to short polling. Long-polling requests let your queue consumers receive messages as soon as they arrive in your queue while reducing the number of empty ReceiveMessageResponse instances returned.
Amazon SQS long polling results in higher performance at reduced cost in the majority of use cases. However, if your application expects an immediate response from a ReceiveMessage call, you might not be able to take advantage of long polling without some modifications to your application.
For example, if your application uses a single thread to poll multiple queues, switching from short polling to long polling will probably not work, because the single thread will wait for the long-poll timeout on any empty queues, delaying the processing of any queues that might contain messages.
In such an application, it is a good practice to use a single thread to process only one queue, allowing the application to take advantage of the benefits that Amazon SQS long polling provides.
What value should I use for my long-poll timeout?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
In general, you should use maximum 20 seconds for a long-poll timeout. Because higher long-poll timeout values reduce the number of empty ReceiveMessageResponse instances returned, try to set your long-poll timeout as high as possible.
If the 20-second maximum doesn’t work for your application (see the example in the previous question), set a shorter long-poll timeout, as low as 1 second.
All AWS SDKs work with 20-second long polls by default. If you don’t use an AWS SDK to access Amazon SQS, or if you configured your AWS SDK to specifically have a shorter timeout, you might need to modify your Amazon SQS client to allow longer requests or to use a shorter long-poll timeout.
What is the AmazonSQSBufferedAsyncClient for Java?
Features, Functionality, and Interfaces
Amazon Simple Queue Service (SQS) | Application Integration
The AmazonSQSBufferedAsyncClient for Java provides an implementation of the AmazonSQSAsyncClient interface and adds several important features:
Automatic batching of multiple SendMessage, DeleteMessage, or ChangeMessageVisibility requests without any required changes to the application
Prefetching of messages into a local buffer that allows your application to immediately process messages from Amazon SQS without waiting for the messages to be retrieved
Working together, automatic batching and prefetching increase the throughput and reduce the latency of your application while reducing your costs by making fewer Amazon SQS requests. For more information, see Client-Side Buffering and Request Batching in the Amazon SQS Developer Guide.
The Amazon SQS Buffered Asynchronous Client does not currently support FIFO queues.
