MQ Flashcards
SQS and SNS are called
“cloud-native” services. That means that they’re using proprietary protocols that are just available on AWS.
they scale really well, we don’t manage them, they’re kind of serverless.
But, traditional applications, when they run on-premise,
they are also been using queues and topics and notification systems.
they’ve been using their own little protocol, so you can be MQTT, AMQP, STOMP, Openwire, WSS. And so when you migrating one of these application to the cloud, you don’t want to re-engineer all the application messaging layer to use SQS and SNS.
Amazon MQ is an in-between way
Amazon MQ
is a managed Apache ActiveMQ.
we can use an hosted MQTT queue in the cloud, and that hosted MQTT queue in the cloud is Amazon MQ.
doesn’t “scale” as SQS/SNS,
it’s because we have to provision it,
runs on a dedicated machine,
we can still have high availability with failover doing multi-AZ.
Apache ActiveMQ
is a great way of doing, you know, messaging, such as
only using the protocol MQTT, etc.
Amazon MQ features
has both a queue feature and a topic feature.
for exam
if you see we are migrating an app from on-premise with cloud, and the app is using the MQTT or the AMQP protocol or whatever, the answer is Amazon MQ.
You are preparing for the biggest day of sale of the year, where your traffic will increase by 100x. You have already setup SQS standard queue. What should you do?
do nothing, SQS scales automatically
You would like messages to be processed by SQS consumers only after 5 minutes of being published to SQS. What should you do?
increase DelaySeconds parameter
Delay queues let you postpone the delivery of new messages to a queue for a number of seconds. If you create a delay queue, any messages that you send to the queue remain invisible to consumers for the duration of the delay period. The default (minimum) delay for a queue is 0 seconds. The maximum is 15 minutes
Your consumers poll 10 messages at a time and finish processing them in 1 minute. You notice that your messages are processed twice, as other consumers also receive the messages. What should you do?
increase Visibility Timeout
Immediately after a message is received, it remains in the queue. To prevent other consumers from processing the message again, Amazon SQS sets a visibility timeout, a period of time during which Amazon SQS prevents other consumers from receiving and processing the message. Increasing the timeout gives more time to the consumer to process that message and will prevent duplicate readings of the message
You’d like your messages to be processed exactly once and in order. Which do you need?
SQS FIFO Queue
FIFO (First-In-First-Out) queues are designed to enhance messaging between applications when the order of operations and events is critical, or where duplicates can’t be tolerated. FIFO queues also provide exactly-once processing but have a limited number of transactions per second (TPS).
You’d like to send a message to 3 different applications all using SQS. You should
SNS + SQS Fan out pattern
This is a common pattern as only one message is sent to SNS and then “fan out” to multiple SQS queues
You have a Kinesis stream usually receiving 5MB/s of data and sending out 8 MB/s of data. You have provisioned 6 shards. Some days, your traffic spikes up to 2 times and you get a throughput exception. You should
add more shards
Each shard allows for 1MB/s incoming and 2MB/s outgoing of data
You are sending a clickstream for your users navigating your website, all the way to Kinesis. It seems that the users data is not ordered in Kinesis, and the data for one individual user is spread across many shards. How to fix that problem?
you should use a partition key that represents the identity of the user
By providing a partition key we ensure the data is ordered for our users
We’d like to perform real time analytics on streams of data. The most appropriate product will be
Kinesis Analytics is the product to use, with Kinesis Streams as the underlying source of data
We’d like for our big data to be loaded near real time to S3 or Redshift. We’d like to convert the data along the way. What should we use?
Kinesis Streams + Kinesis Firehose
