Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

AWS Cert > SQS, SNS, Kinesis > Flashcards

SQS, SNS, Kinesis Flashcards

1
Q
  • Oldest offering (over 10 years old)
  • Fully managed service, used to decouple applications
  • Attributes:
    • Unlimited throughput, unlimited number of messages in queue
    • Default retention of messages: 4 days, maximum of 14 days
    • Low latency (<10 ms on publish and receive)
    • Limitation of 256KB per message sent
  • Can have duplicate messages (at least once delivery, occasionally)
  • Can have out of order messages (best effort ordering)
A

Amazon SQS – Standard Queue

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Produced to SQS using the ______?

A

SDK (SendMessage API)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

The message is ___________ in SQS until a consumer deletes it

A

persisted

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

SQS – Producing Messages - Message retention

A

default 4 days, up to 14 days

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is the SQS standard

A

unlimited throughput

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

SQS Consumers run on ____?

A

EC2 instances
On-premise servers
AWS Lambda

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

How many messages can a SQS Consumer receive at a time?

A

10

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

SQS Consumer - Delete the messages using the __________ API

A

DeleteMessage

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q
  • Consumers receive and process messages in parallel
  • At least once delivery
  • Best-effort message ordering
  • Consumers delete messages after processing them
  • We can scale consumers horizontally to improve throughput of processing
A

SQS – Multiple EC2 Instances Consumers

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Amazon SQS - Encryption

A
  • In-flight encryption using HTTPS API
  • At-rest encryption using KMS keys
  • Client-side encryption if the client wants to perform encryption/decryption i
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Amazon SQS - Access Controls

A

IAM policies to regulate access to the SQS API

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

2 uses for SQS Access Policies?

A
  • Useful for cross-account access to SQS queues
  • Useful for allowing other services (SNS, S3…) to write to an SQS queue
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

After a message is polled by a consumer, it becomes _______ to other consumers

A

invisible

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

By default, the “message visibility timeout” is __________?

A

30 seconds

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

If a message is not processed within the visibility timeout, it will be processed ___________?

A

TWICE

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

A consumer could call the ____________ API to get more time

A

ChangeMessageVisibility

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What happens if a visibility timeout is high (hours)?

A

consumer crashes, re-processing will take time

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What happens if visibility timeout is too low (seconds)?

A

we may get duplicates

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

If someone wants to decrease the latency and increase efficiency and decrease there API calls to a SQS Queue .. what should they do?

A

Long Polling

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q
  • Limited throughput: 300 msg/s without batching, 3000 msg/s with
  • Exactly-once send capability (by removing duplicates)
  • Messages are processed in order by the consumer
A

Amazon SQS – FIFO Queue

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What if you want to send one message to many receivers?

A

Amazon SNS

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q
  • The “event producer” only sends message to one SNS topic
  • As many “event receivers” (subscriptions) as we want to listen to the SNS topic notifications
  • Each subscriber to the topic will get all the messages (note: new feature to filter messages)
  • Up to 12,500,000 subscriptions per topic
  • 100,000 topics limit
A

Amazon SNS

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What are the 2 types of publishing for AWS SNS?

A

Topic Publish (using the SDK)
Direct Publish (for mobile apps SDK)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Amazon SNS –
* Create a topic
* Create a subscription (or many)
* Publish to the topic

A

Topic Publish (using the SDK)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

Amazon SNS –
* Create a platform application
* Create a platform endpoint
* Publish to the platform endpoint
* Works with Google GCM, Apple APNS, Amazon ADM…

A

Direct Publish (for mobile apps SDK)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

What is SNS Encryption?

A
  • In-flight encryption using HTTPS API
  • At-rest encryption using KMS keys
  • Client-side encryption if the client wants to perform encryption/decryption itself
27
Q

2 uses for the SNS Access Policies

A
  • Useful for cross-account access to SNS topics
  • Useful for allowing other services ( S3…) to write to an SNS topic
28
Q
  • Push once in SNS, receive in all SQS queues that are subscribers
  • Fully decoupled, no data loss
  • SQS allows for: data persistence, delayed processing and retries of work
  • Ability to add more SQS subscribers over time
  • Make sure your SQS queue access policy allows for SNS to write
  • Cross-Region Delivery: works with SQS Queues in other regions
A

SNS + SQS: Fan Out

29
Q

For the same combination of: event type (e.g. object create) and prefix (e.g. images/) you can only have one S3 Event rule.
What do you use if you want to send the same S3 event to many SQS queues?

A

Fan Out Method

30
Q

With SNS - FIFO Topics, what type of SQS queues can you have as subcribers?

A

SQS FIFO

31
Q

Does Amazon SNS – FIFOTopic have limited throughput?

A

YES! Just like SQS FIFO

32
Q
  • JSON policy used to filter messages sent to SNS topic’s subscriptions
  • If a subscription doesn’t have a filter policy, it receives every message
A

SNS – Message Filtering

33
Q
  • Makes it easy to collect, process, and analyze streaming data in real-time
  • Ingest real-time data such as: Application logs, Metrics, Website clickstreams, IoT telemetry data…
A

Kinesis

34
Q

capture, process, and store data streams

A

Kinesis Data Streams

35
Q

load data streams into AWS data stores

A

Kinesis Data Firehose

36
Q

analyze data streams with SQL or Apache Flink

A

Kinesis Data Analytics

37
Q

capture, process, and store video streams

A

Kinesis Video Streams

38
Q

What are the 4 Kinesis Products?

A

Kinesis Data Streams
Kinesis Data Firehose
Kinesis Data Analytics
Kinesis Video Streams

39
Q

Kinesis Data Streams - Retention

A

between 1 day to 365 days

40
Q

Does Kinesis Data Streams have the ability to reprocess (replay) data??

A

YES

41
Q

Once data is inserted in Kinesis, it ________?

A

can’t be deleted (immutability)

42
Q

Does data that share the same partition go to the same shard?

A

YES

43
Q

What is it called when data shares the same partition and goes to the same shard?

A

Ordering

44
Q

What are 3 examples of Kinesis Data Streams Producers?

A

AWS SDK, Kinesis Producer Library (KPL), Kinesis Agent

45
Q

What are the 2 different types and they’re examples of Kinesis Data Streams Consumers?

A

Write your own: Kinesis Client Library (KCL), AWS SDK
Managed: AWS Lambda, Kinesis Data Firehose, Kinesis Data Analytics,

46
Q

Kinesis Data Streams – Capacity Modes (2)

A

Provisioned mode
On-demand mode

47
Q

Kinesis Data Streams – Capacity Modes:
* No need to provision or manage the capacity
* Default capacity provisioned (4 MB/s in or 4000 records per second)
* Scales automatically based on observed throughput peak during the last 30 days
* Pay per stream per hour & data in/out per GB

A

On-demand mode

47
Q

Kinesis Data Streams – Capacity Modes:
* You choose the number of shards provisioned, scale manually or using API
* Each shard gets 1MB/s in (or 1000 records per second)
* Each shard gets 2MB/s out (classic or enhanced fan-out consumer)
* You pay per shard provisioned per hour

A

Provisioned mode

48
Q
  • Control access / authorization using IAM policies
  • Encryption in flight using HTTPS endpoints
  • Encryption at rest using KMS
  • You can implement encryption/decryption of data on client side (harder)
  • VPC Endpoints available for Kinesis to access within VPC
  • Monitor API calls using CloudTrail
A

Kinesis Data Streams Security

49
Q

Fully Managed Service, no administration, automatic scaling, serverless

A

Kinesis Data Firehose

50
Q

What are 3 consumers (and examples) for Kinesis Data Firehose

A
  • AWS: Redshift / Amazon S3 / OpenSearch
  • 3rd party partner: Splunk / MongoDB / DataDog / NewRelic / …
  • Custom: send to any HTTP endpoint
51
Q

How do you get charged for using Kinesis Data Firehose

A

Pay for data going through Firehose

52
Q

What is Kinesis Data Firehose latency?

A
  • Near Real Time
  • 60 seconds latency minimum for non full batches
  • Or minimum 1MB of data at a time
53
Q

Does Kinesis Data Firehose support many data formats, conversions, transformations, compression

A

YES

54
Q

Does Kinesis Data Firehose support custom data transformations using AWS Lambda

A

YES

55
Q

Where can Kinesis Data Firehose send failed or all data?

A

a backup S3 bucket

56
Q

Kinesis Data Streams vs Firehose

A

Kinesis Data Streams:
* Streaming ser vice for ingest at scale
* Write custom code (producer / consumer)
* Real-time (~200 ms)
* Manage scaling (shard splitting / merging)
* Data storage for 1 to 365 days
* Supports replay capability

Kinesis Data Firehose:
* Load streaming data into S3 / Redshift / OpenSearch / 3rd party / custom HTTP
* Fully managed
* Near real-time (buffer time min. 60 sec) * Automatic scaling
* No data storage
* Doesn’t support replay capability

57
Q

How is data sent into Kinesis?

A

using a Partition Key

58
Q

Does the same key always go to the same shard?

A

YES

59
Q

What is similar to Partition Key in SQS?

A

Group ID

60
Q
  • Consumer “pull data”
  • Data is deleted after being consumed
  • Can have as many workers (consumers) as we want
  • No need to provision throughput
  • Ordering guarantees only on FIFO queues
  • Individual message delay capability
A

SQS

61
Q
  • Push data to many subscribers
  • Up to 12,500,000 subscribers
  • Data is not persisted (lost if not delivered)
  • Pub/Sub
  • Up to 100,000 topics
  • No need to provision throughput
  • Integrates with SQS for fan- out architecture pattern
  • FIFO capability for SQS FIFO
A

SNS

62
Q
A

AWS Cert (16 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions