SQS, SNS, SWF Flashcards
SQS - Exam Tips
- Amazon SQS is a web service that gives you access to a message queue that can be used to store messages while waiting for a computer to process them (Read FAQ!)
- Does not maintain or guarantee message order for Standard queues. FIFO queues preserve the exact order in which messages are sent and received
- Message groups: Messages are grouped into distinct, ordered “bundles” within a FIFO queue. For each message group ID, all messages are sent and received in strict order. However messages with different message group ID values might be sent and received out of order.
SQS - Exam Tips
- Amazon SQS is a web service that gives you access to a message queue that can be used to store messages while waiting for a computer to process them (Read FAQ!)
- Does not maintain or guarantee message order for Standard queues. FIFO queues preserve the exact order in which messages are sent and received
- Message groups: Messages are grouped into distinct, ordered “bundles” within a FIFO queue. For each message group ID, all messages are sent and received in strict order. However messages with different message group ID values might be sent and received out of order.
SQS - Exam Tips
- Guarantees At-Least-Once Delivery for Standard queues, which might result in duplicate messages. FIFO queues provide Exact One Time Delivery
- Use Amazon Simple Queue Service (SQS) to Decouple Components
- Loose Coupling: Queuing Chain Pattern enables asynchronous processing
SQS - Exam Tips
- Guarantees At-Least-Once Delivery, but does not guarantee Exact One Time Delivery which might result in duplicate messages (Not true anymore with the introduction of FIFO queues)
- Use Amazon Simple Queue Service (SQS) to Decouple Components
- Loose Coupling: Queuing Chain Pattern enables asynchronous processing
SQS - Exam Tips
- Retention Period: Holds message for 4 days by default, and can be changed from 1 min to 14 days after which the message is deleted
- Visibility Timeout: Period of time during which the message once read by a consumer is not visible to the other consumers till the timeout occurs (prevents multiple components from processing the same message). 30 seconds visibility timeout by default up to 12 hours maximum (use ChangeMessageVisibility)
- Long Polling: Doesn’t return a response until a message arrives in the message queue, or the long poll times out. Default long polling time is 20 seconds (and maximum) and can be as low as 1 second
SQS - Exam Tips
- Retention Period: Holds message for 4 days by default, and can be changed from 1 min to 14 days after which the message is deleted
- Visibility Timeout: Period of time during which the message once read by a consumer is not visible to the other consumers till the timeout occurs (prevents multiple components from processing the same message). 30 seconds visibility timeout by default up to 12 hours maximum
- Long Polling: Doesn’t return a response until a message arrives in the message queue, or the long poll times out. Default long polling time is 20 seconds (and maximum) and can be as low as 1 second
SQS -Exam Tips
- 256kb message size now available
- Billed at 64kb “Chunks”. A 256kb message will be 4x64kb “chunks”
- You can’t prioritize messages within one Standard queue. Build multiple queues for prioritization.
- Dead Letter Queues: A queue of messages that were not able to be processed (Can sideline and isolate the unsuccessfully processed messages)
- Amazon SQS Free Tier provides with 1 million requests per month at no charge
- FIFO Queues are limited to 300 transactions per second (TPS)
SQS -Exam Tips
- 256kb message size now available
- Billed at 64kb “Chunks”. A 256kb message will be 4x64kb “chunks”
- You can’t prioritize messages within one queue. Build multiple queues for prioritization.
- Dead Letter Queues: A queue of messages that were not able to be processed (Can sideline and isolate the unsuccessfully processed messages)
- Amazon SQS Free Tier provides with 1 million requests per month at no charge
SQS vs SWF
- SWF is an orchestration service to coordinate work across distributed components
- SQS has a retention period of 14 days, SWF up to 1 year for worklflow executions
- Amazon SWF presents a task-oriented API, whereas Amazon SQS offers a message-oriented API
- Amazon SWF ensures that a task is assigned only once and is never duplicated. With Amazon SQS, you need to handle duplicated messages and may also need to ensure that a message is processed only once
- Amazon SWF keeps track of all the tasks and events in an application. With Amazon SQS, you need to implement your own application-level tracking, especially if your application uses multiple queues
- Coordinate Tasks with Amazon Simple Workflow (SWF)
SQS vs SWF
- SQS has a retention period of 14 days, SWF up to 1 year for worklflow executions
- Amazon SWF presents a task-oriented API, whereas Amazon SQS offers a message-oriented API
- Amazon SWF ensures that a task is assigned only once and is never duplicated. With Amazon SQS, you need to handle duplicated messages and may also need to ensure that a message is processed only once
- Amazon SWF keeps track of all the tasks and events in an application. With Amazon SQS, you need to implement your own application-level tracking, especially if your application uses multiple queues
- Coordinate Tasks with Amazon Simple Workflow (SWF)
SWF Actors
- Workflow Starters - An application that can initiate (start) a workflow. Could be your e-commerce website when placing an order or a mobile app searching for bus times
- Deciders - Control the flow of activity tasks in a workflow execution. If something has finished in a workflow (or fails) a Decider decides what to do next
- Activity Workers - Programs that interact with Amazon SWF to get tasks, process received tasks, and return results
- SWF Domains - Workflow and activity types and the workfklow execution itself are all scoped to a domain. Domains isolate a set of types, executions, and tasks lists from others within the same account.
SWF Actors
- Workflow Starters - An application that can initiate (start) a workflow. Could be your e-commerce website when placing an order or a mobile app searching for bus times
- Deciders - Control the flow of activity tasks in a workflow execution. If something has finished in a workflow (or fails) a Decider decides what to do next
- Activity Workers - Carry out the activity tasks
SQS - Fanning Out
Create an SNS topic first using SNS. Then create and subscribe multiple SQS queues to the SNS topic.
Now whenever a message is sent to the SNS topic, the message will be fanned out to the SQS queues, i.e. SNS will deliver the message to all the SQS queues that are subscribed to the topic.
SQS - Fanning Out
Create an SNS topic first using SNS. Then create and subscribe multiple SQS queues to the SNS topic.
Now whenever a message is sent to the SNS topic, the message will be fanned out to the SQS queues, i.e. SNS will deliver the message to all the SQS queues that are subscribed to the topic.
SNS vs SQS
- Both Messaging Services in AWS
- Use Amazon Simple Notification Service (SNS) to Deliver Notifications (publisher-subscriber model)
- SNS - Push
- SQS - Polls (Pulls)
- SNS Subscribers/protocols:
- HTTP/HTTPS
- Email (Plain or JSON)
- SQS
- Application
- Lambda
- Mobile Push Messaging
- SNS messages can be customized for each protocol
SNS vs SQS
- Both Messaging Services in AWS
- Use Amazon Simple Notification Service (SNS) to Deliver Notifications (publisher-subscriber model)
- SNS - Push
- SQS - Polls (Pulls)
- SNS Subscribers:
- HTTP/HTTPS
- Email (Plain or JSON)
- SQS
- Application
- Lambda
- Mobile Push Messaging
API Gateway
- Makes it easy for developers to publish, maintain, monitor, and secure APIs at any scale. You can create an API that acts as a “front door” for applications to access data or functionality for your back-end services (EC2, AWS Lambda, etc.)
- API Gateway has caching capabilities to increase performance
- API Gateway is low cost and scales automatically
- You can throttle API Gateway to prevent attacks
- You can log results to CloudWatch
- If you are using Javascript/AJAX that uses multiple domains with API Gateway, ensure that you have enabled CORS (Cross-Origin Resource Sharing) on API Gateway (Error - “Origin policy cannot be read at the remote resource”)
API Gateway
- Makes it easy for developers to publish, maintain, monitor, and secure APIs at any scale. You can create an API that acts as a “front door” for applications to access data or functionality for your back-end services (EC2, AWS Lambda, etc.)
- API Gateway has caching capabilities to increase performance
- API Gateway is low cost and scales automatically
- You can throttle API Gateway to prevent attacks
- You can log results to CloudWatch
- If you are using Javascript/AJAX that uses multiple domains with API Gateway, ensure that you have enabled CORS (Cross-Origin Resource Sharing) on API Gateway (Error - “Origin policy cannot be read at the remote resource”)