AWS Integration & Messaging Flashcards
There are two patterns of application communication
1) Synchronous communications (application to application)
2) Asynchronous / Event based (application to queue to application)
name all three different models for decoupling your application
- using SQS: queue model
- using SNS: pub/sub model
- using Kinesis: real-time streaming model
SQS - How many messages can be in the queue?
Unlimited
SQS - What’s the size lmitation per message sent?
256kb
What’s SQS – Delay Queue
• Delay a message (consumers don’t see it immediately) up to 15 minutes • Default is 0 seconds (message is available right away)
What parameter overrides the default SQS – Delay Queue?
DelaySeconds parameter
What can you expect to get back from SQS
- Message identifier
* MD5 hash of the body
What’s the lifecycle of SQS - Consuming Messages?
- Poll SQS for messages (receive up to 10 messages at a time) • Process the message within the visibility timeout
- Delete the message using the message ID & receipt handle
What is SQS - Visibility timeout?
When a consumer polls a message from a queue, the message is “invisible” to other
consumers for a defined period… the Visibility Timeout:
What is the limits and default for SQS - Visibility timeout?
• Set between 0 seconds and 12 hours (default 30 seconds)
SQS - What’s the API to change the visibility while processing a message?
ChangeMessageVisibility
What’s the API to tell SQS the message was successfully processed?
DeleteMessage
What is a Dead Letter Queue?
-If a consumer fails to process a message within the Visibility Timeout…
the message goes back to the queue!
• We can set a threshold of how many times a message can go back to the queue – it’s called a “redrive policy”
• After the threshold is exceeded, the message goes into a dead letter queue (DLQ)
• We have to create a DLQ first and then designate it dead letter queue
-Make sure to process the messages in the Failure DLQ before they expire!
Define AWS SQS - Long Polling.
-When a consumer requests message from the queue, it can optionally “wait” for messages to arrive if there are none in the queue
• This is called Long Polling
• LongPolling decreases the number of API calls made to SQS while increasing the efficiency and latency of your application.
-The wait time can be between 1 sec to 20 sec (20 sec preferable)
SQS Long polling can be enabled at the queue level or at the API level using what param?
WaitTimeSeconds
SQS - FIFO has higher or lower throughput?
Lower throughput (up to 3,000 per second with batching, 300/s without)
SQS - FIFO Message groups allow for what?
Possibility to group messages for FIFO ordering using “Message GroupID”
SNS - Up to ___ subs per topic
Up to 10,000,000 subscriptions per topic
SNS - What’s the topics limit?
100,000 topics limit
SNS - Who can subscribe?
- SQS
- HTTP / HTTPS (with delivery retries – how many times)
- Lambda
- Emails
- SMS messages
- Mobile Notifications
SNS - How do you publish a topic (within your AWS Server – using the SDK)?
- Create a topic
- Create a subscription (or many)
- Publish to the topic
Explain AWS Kinesis
- Kinesis is a managed alternative to Apache Kafka
- Great for application logs, metrics, IoT, clickstreams
- Great for “real-time” big data
- Great for streaming processing frameworks (Spark, NiFi, etc…)
Kineses Data is automatically replicated to how many AZ’s?
Data is automatically replicated to 3 AZ
What are the three types of Kinesis?
- Kinesis Streams: low latency streaming ingest at scale
- Kinesis Analytics: perform real-time analytics on streams using SQL
- Kinesis Firehose: load streams into S3, Redshift, ElasticSearch…
