Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

.๐’œ๐’ฒ๐’ฎ ๐’ž๐‘’๐“‡๐“‰๐’พ๐’ป๐’พ๐‘’๐’น ๐’Ÿ๐‘’๐“‹๐‘’๐“๐‘œ๐“…๐‘’๐“‡ > Section 12: AWS Integration & Messaging: SQS, SNS & Kinesis > Flashcards

Section 12: AWS Integration & Messaging: SQS, SNS & Kinesis Flashcards

1
Q

What are the two patterns of application communication?

A 
Synchronous communications (application to application)
Asynchronous / Event based communications (application to queue to application)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Why can synchronous communications between applications be problematic?

A

If there are sudden spikes of traffic, the destination probably wonโ€™t be able to handle everyhting at once (Ex: An app usually encode 10 videos/hour but now suddenly receives 1000 videos in a short amount time)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What are the three integration and messaging fully managed services by AWS?

A

SQS
SNS
Kinesis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is the model of SQS?

A

Queue model

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is the model of SNS?

A

Pup/Sub model

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is the model of Kinesis

A

Real-time streaming model

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What are the two types of queue available in SQS?

A

Standard Queue

FIFO Queue

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What name defines the entities sending messages to an SQS queue?

A

Producers

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What name defines the entities consuming messages of an SQS queue?

A

Consumers

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

How are messages consumed from an SQS queue?

A

They are polled by the consumers

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

How old is the AWS SQS standard queue?

A

Over 10 years old

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What do you have to do in order to scale your standard AWS queue?

A

Nothing, it scales automatically to 10,000s messages per second

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is the default retention rate of messages in a standard SQS queue?

A

4 days

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is the maximum retention configurable for messages in an SQS queue?

A

14 days

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is the limit of how many messages can be in a standard SQS queue?

A

No limit

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is the latency for SQS queues

A

< 10ms

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Can the number of consumers of an SQS queue scale? If so, on which axis?

A

Yes, it can scale horizontally

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Can a standard SQS queue have duplicate messages?

A

It can occasionally

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Are messages in order in a standard SQS queue?

A

Not necessarily (best effort ordering is built into the service)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

How can you get messages 100% in order in SQS?

A

Use a FIFO Queue rather than a standard SQS queue

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What is the maximum size of messages in an SQS queue?

A

256KB

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

How can you add a delay between the moment a message is sent and the moment consumers see the message in an SQS queue?

A

By adding a delivery delay at the queue level

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

How can you override the default delivery delay for a certain message sent in an SQS standard queue?

A

By overriding the default DelaySeconds parameter when sending a message

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

How can you override the default delivery delay for a certain message sent in an SQS FIFO queue?

A

You canโ€™t, DelaySeconds is only available in standard queues (It make sense, other FIFO would not be respected)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

What consists of a message (sent to an SQS queue) ?

A

Body (String, up to 256KB)

Attributes (Metadata)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

What does the producer gets back when sending a message to an SQS queue?

A

Message ID

MD5 hash of the body

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

How many messages at a time can a consumer receive when polling an SQS queue?

A

Up to 10

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

What is called the time period within which a message is hidden from the consumers because it has been consumed?

A

The visibility timeout

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

What happens during the visibility timeout?

A

The message received by the consumer is still in the SQS queue but is considered โ€œin flightโ€ therefore it canโ€™t be received by other consumers

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

What is the responsibility of the consumer when he successfully finishes to process a message?

A

He needs to delete the message from the queue using the message ID and the receipt handle

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
31
Q

What is the default time period of the visibility timeout in an SQS queue?

A

30 seconds

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
32
Q

What is the maximum time period of the visibility timeout in an SQS queue?

A

12 hours

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
33
Q

What can happen if the time period of the visibility timout of an SQS queue is too high?

A

If the consumer fails to process the message, there will be a long delay before trying to process the message again

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
34
Q

What can happen if the time period of the visibility timout of an SQS queue is too low?

A

If the consumer needs time to process the message, another consumer will receive the message and the message will be processed more than once

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
35
Q

What can you do if you set the visibility timeout of your SQS queue too low and a consumer needs more time to process a message?

A

Use the ChangeMessageVisibility API to increase the length of the visibility timeout of the message being processed at the moment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
36
Q

What is the API that a consumer of an SQS queue needs to call when successfully finishing to process a message?

A

The DeleteMessage API

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
37
Q

(SQS queue) Where should messages that fail to get processed multiple times in a row be transferred to?

A

To a DLQ

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
38
Q

What does DLQ stand for?

A

Dead Letter Queue

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
39
Q

What is the redrive policy?

A

The redrive policy specifies the source queue, the dead-letter queue, and the conditions under which Amazon SQS moves messages from the former to the latter if the consumer of the source queue fails to process a message a specified number of times.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
40
Q

What strategy for our consumers allows us to save costs when using SQS queues

A

Long polling

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
41
Q

What is the long polling?

A

A consumer requests message from queue and โ€œwaitโ€ for message if there are none at the moment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
42
Q

What is the maximum wait time when doing long polling to as SQS queue?

A

20 seconds

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
43
Q

What is the preferred wait time when doing long polling to as SQS queue?

A

20 seconds

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
44
Q

At what level can long polling be enable?

A

At the queue level or at the API level

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
45
Q

What is the name of the parameter which allows us to set the time for long polling?

A

WaitTimeSeconds

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
46
Q

What happens to messages which donโ€™t get deleted within the visibility timeout period?

A

They become visible again in the SQS queue, up to the defined treshold (redrive policy), when they will therefore be transferred to the DLR

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
47
Q

What is the particular naming rule for FIFO queues in SQS

A

They must end with .fifo

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
48
Q

What is the maximum number of messages per second (with / without batching) for FIFO queues?

A

3000 messages/sec with batching

300 messages/sec without batching

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
49
Q

Can there be duplicates in a FIFO queue?

A

No

50
Q

Can there be โ€œper message delayโ€ in SQS FIFO queues?

A

No, only per queue delay

51
Q

What are the two features exclusive to FIFO queues?

A

Deduplication

Sequencing

52
Q

How can you get deduplication in FIFO queues?

A

By providing a MessageDeduplicationId with your message

53
Q

Do you have to provide your own MessageDeduplicationId when using deduplication in a FIFO queue or is there a better way to do it?

A

You can use content based deduplication (the MessageDeduplicationId is generated as the SHA-256 of the message body (not the attributes))

54
Q

What is the deduplication interval in FIFO queues?

A

A 5-minute period where the queue will track for duplicate messages

55
Q

How to get sequencing in FIFO queue?

A

By specifying the same MessageGroupId to messages which you absolutely want to get processed in order

56
Q

Are messages with different MessageGroupId sure to get processed in order in a FIFO queue?

A

No, different consumers can poll and receive messages with different MessageGroupId. Only messages with the same MessageGroupId are sure to get processed one after the other

57
Q

Letโ€™s say you have user which executes some actions like โ€œAdds X to cartโ€, โ€œPurchase Xโ€, โ€œCancels Xโ€, what could be the MessageGroupId for the messages (that will get added to a FIFO queue) related to that user?

A

user_id

58
Q

What can you use if you need to send messages that are larger than 256KB in a SQS queue?

A

Use the SQS Extended Client for Java or a custom solution for other environments

59
Q

What does the SQS Extended Client do?

A

Producer sends large message to S3
Producer sends small metadata message to SQS queue
Consumer polls/receive the small message
Consumer retrieves the large message from S3

60
Q

What encryption do you get with SQS?

A

In flight using the HTTPS endpoint

SSE can be enabled using KMS

61
Q

What does SSE stand for?

A

Server Side Encryption

62
Q

What is encrypted when using SSE in SQS?

A

Only the messages body, not the metadata

63
Q

What might be wrong if we canโ€™t make our applications work with SQS

A

We probably have a problem with our IAM policies attached to the Roles of our applications

64
Q

How to get finer grained control over IP when working with SQS?

A

Using SQS queue access policy

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-overview-of-managing-access.html

65
Q

Which of these SQS API donโ€™t have a batch API

SendMessage
ReceiveMessage
DeleteMessage
ChangeMessageVisibility

A

ReceiveMessage

Because you call already receive up to 10 messages at a time

66
Q

What does PurgeQueue does (SQS)?

A

Delete all the messages in a queue

67
Q

What are the three most common use cases for SQS?

A

Decouple applications
(for example to handle payments asynchronously)

Buffer writes to a database
(for example a voting application)

Handle large loads of messages coming in
(for example an email sender)

68
Q

Can SQS be integrated with Auto Scaling? If so, how?

A

Yes, through CloudWatch

69
Q

What can you use if you want to send a message to multiple receivers in a single call?

A

SNS

70
Q

Where does an event producer sends messages to with SNS?

A

To one SNS topic

71
Q

What is the maximum amount of receivers per topic in SNS?

A

10,000,000

72
Q

What is the maximum number of topics in an AWS account?

A

100,000

73
Q

What can SNS subscribers be?

A 
SQS
HTTP/HTTPS
Lambda
Emails
SMS messages
Mobile Notifications
74
Q 
How does these services use SNS?
CloudWatch
ASG
Amazon S3
CloudFormation
A

CloudWatch: Alarms
ASG: Notifications from alarms to trigger auto scalling
Amazon S3: On bucket events
CloudFormation: State changes, failed to build, etc.

75
Q

What are the steps to publish in a topic in SNS?

A

Create a topic
Create a subscription
Publish to the topic

76
Q

What are the steps to direct publish in SNS (mobile apps sdk)?

A

Create a platform application
Create a platform endpoint
Publish to the platform endpoint

77
Q

What is the SNS + SQS: Fan out strategy?

A

Push once in SNS, receive in many SQS

78
Q

What is AWS Kinesis?

A

Kinesis is a managed alternative to Apache Kafka

79
Q

What is Kinesis great for?

A

Application logs, metrics, IoT, clickstreams
โ€œReal-timeโ€ big data
Streaming processing frameworks (Spark, NiFi, etc.)

80
Q

Is there data replication with Kinesis?

A

Yes, data is automatically replicated to 3 AZ.

81
Q

What are the three compoenents of Kinesis?

A

Kinesis Streams
Kinesis Analytics
Kinesis Firehose

82
Q

What is Kinesis stream?

A

Low latency streaming ingest at scale

83
Q

What does Kinesis Analytics do?

A

Performs real-time analytics

84
Q

What language does Kinesis Analytics leverages?

A

SQL

85
Q

What does Kinesis Firehose do?

A

Loads streams into S3, Redshift, ElasticSearch, etc.

86
Q

Of what are composed Kinesis Streams?

A

Shards/Partitions

87
Q

What is the default data retention period in Kinesis Streams?

A

1 day

88
Q

What is the max data retention period in Kinesis Streams?

A

7 days

89
Q

Do you have the ability to reprocess / replay data in Streams?

A

Yes

90
Q

Can many applications consume the same stream?

A

Yes

91
Q

Once data is inserted in Kinesis, can it be deletted?

A

No

92
Q

What is the write capacity of a shard?

A

1 MB/s or 1000 messages/sec

93
Q

What is the read capacity of a shard?

A

2 MB/s

94
Q

How are Kinesis Streams billed?

A

They are billed per hour per shard provisioned

95
Q

How many shards can a stream have?

A

As many as you want

96
Q

Can the number of shards of a stream evolve over time?

A

Yes (reshard/merge)

97
Q

When records enters a stream, do they stay in order?

A

Records in the same shard are in order

98
Q

How can you make sure that records that need to stay in order get in the same shard/partition?

A

By providing the same PartitionKey

99
Q

How can you avoid the โ€œhot partitionโ€ ?

A

By providing highly distributed Partition Keys

100
Q

What do messages received by Kinesis stream get?

A

A sequence number

101
Q

What API can you use to send messages to Kinesis

A

PutRecord (without batching)

PutRecords (with batching)

102
Q

Can you send messages to a Kinesis Stream from the Console?

A

No, you have to use the CLI, SKSs, or producer libraries from various frameworks

103
Q

What error will we get if we go over the limit from our Kinesis Stream? (exceeding MB/s or TPS for any shard)

A

ProvisionedThroughputExceeded

104
Q

What might be the cause of a ProvisionedThroughputExceeded error?

A

A โ€œhot partition/shardโ€

105
Q

How can you solve ProvisionedThroughputExceeded ?

A

Retries with backoff
Increase shards (scaling)
Ensure your partion keys are distributed enough not to get a โ€œhot partitionโ€

106
Q

What can you use to consume a Kinesis Stream?

A

CLI
SDK
or the Kinesis Client Library

107
Q

What does KCL stand for?

A

Kinesis Client Library

108
Q

What does KCL uses to checkpoint offsets?

A

DynamoDB

109
Q

What does KCL uses to track other workers and share the work amongst shards?

A

DynamoDB

110
Q

How many shards can a KCL read?

A

Many

111
Q

You canโ€™t have more ___ than ___
Words to place:
KCL
shards

A

You canโ€™t have more KCL than shards

112
Q

What can KCL run on?

A

EC2, EB, even on premise applications

113
Q

How is Kinesis secure?

A

IAM policies (control access / authorization)
Encryption in flight (HTTPS)
Encryption at rest (KMS)

114
Q

Can you encrypt/decrypt client side when using Kinesis?

A

Yes but itโ€™s harder

115
Q

Are VPC endpoints available for Kinesis?

A

Yes!

116
Q

What does AWS Kinesis Data Analytics do?

A

Perform real-time analytics on Kinesis Streams using SQL

117
Q

What do you have to do to make Kinesis Data Analytics scale?

A

Nothing, itโ€™s a fully managed, you pay for what you use

118
Q

What can you load data into when using AWS Kinesis Firehose?

A

Redshift, S3, ElasticSearch, Splunk

119
Q

What do you have to do to make AWS Kinesis Firehose scale?

A

Nothing, itโ€™s a fully managed service, you pay for the amount of data going through Firehose

120
Q

What is AWS Kinesis meant for?

A

For real-time big data, analytics and ETL

121
Q

What does ETL stand for?

A

โ€œExtract, Transform, Loadโ€

.๐’œ๐’ฒ๐’ฎ ๐’ž๐‘’๐“‡๐“‰๐’พ๐’ป๐’พ๐‘’๐’น ๐’Ÿ๐‘’๐“‹๐‘’๐“๐‘œ๐“…๐‘’๐“‡ (26 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions