Messaging

Question 1

SQS

Answer 1

Simple Queuing Service

• used to decouple applications
• unlimited throughput and number of messages in the queue
• messages are persisted until consumers (applications) remove the message

Question 2

SQS - Security

Answer 2

Encryption - can use in-flight HTTPS API, at-rest KMS keys or client-side encryption

Access Controls - IAM policies to regulate access to the SQS API

SQS Access Policies - S3 bucket policies

Question 3

SQS – Message Visibility Timeout

Answer 3

After a message is polled by a consumer, it becomes invisible to other consumers, by default 30s

If a message is not processed within the visibility timeout, it will be processed twice unless the consumer asks for more time

Question 4

SQS – Dead Letter Queue

Answer 4

Set a threshold of how many times a message can go back to the queue and if exceeded, send to dead letter queue

Retention of messages in DLQ is 14 days

Question 5

SQS – Delay Queue

Answer 5

Delay visibility of a message to a consumer

Question 6

SQS – FIFO Queue

Answer 6

First In First Out (ordering of messages in the queue)

Question 7

SNS

Answer 7

A queue that follows a subscription model to send messages which allows for multiple consumers to consume the same message based on a topic

Question 8

SNS - Security

Answer 8

Similar to SQS:

Encryption - can use in-flight HTTPS API, at-rest KMS keys or client-side encryption

Access Controls - IAM policies to regulate access to the SNS API

SNS Access Policies - S3 bucket policies

Question 9

SNS + SQS: Fan Out

Answer 9

Pattern that push messages once into SNS, then is received in all SQS queues that are subscribers

Question 10

AWS Kinesis

Answer 10

Kinesis is a managed alternative to Apache Kafka which is a application for big data i.e. logs, IoT, clickstreams

Question 11

Kinesis Streams

Answer 11

Low latency streaming ingestion at scale to perform analytics

Question 12

Kinesis Streams Shards

Answer 12

Streams are divided in ordered Shards / Partitions
- can scale over time

For a message, the same “partition” key goes to the same partition

Question 13

Kinesis Data Firehose

Answer 13

Fully Managed Service, no administration, automatic scaling, serverless to load (big) data into Redshift / Amazon S3 / ElasticSearch / Splunk

Question 14

Kinesis Data Analytics

Answer 14

Perform real-time analytics on Kinesis Streams using SQL

Question 15

Ordering data into SQS

Answer 15

Use Group ID to scale consumers of SQS

Question 16

Amazon MQ

Answer 16

Used for migrating non-proprietary messaging applications into the cloud that use other protocols such as MQTT, AMQP, STOMP, Openwire, WSS
- doesn’t scale as much as SQS/SNS