Applications Flashcards
SQS - Simple Queuing system
gives access to a message queue that can store messages while waiting for a computer to process them
Allows you to decouple the components of an app so they run independently
Messages can contain up to 256kb of text in any format
SQS queue types
Standard
FIFO
Standard queue type
default queue type
allows for nearly unlimited # of transactions per second
guarantee a message is delivered at least once
- may be delivered out of order
- may be delivered more than once
best effort ordering
FIFO queue
exactly once processing
sent order of messages preserved
limited to 300 transactions per second
SQS pull vs push
SQS is pull based, you need an EC2 instance pulling messages from the queue
SQS message retention period
1 minute to 14 days
default: 4 days
Visibility timeout
amount of time message is ‘invisible’ in queue after a reader picks up the message
If job is processed before timeout, job is deleted from queue. If not, it becomes visible and another reader will process it
timeout max: 12 hours
SQS long vs short polling
long polling: will wait until there is something to pull from the queue or timeout is reached
short polling: will return immediately, even if queue is empty
SWF - Simple workflow services
Way of coordinating work across distributed app components
tasks = invocations of processing steps that can be performed by code, web service calls, human actions, and scripts
SWF vs SQS
If there is human interaction -> use SWF
max retention period:
- SQS = 14 days
- SWF = 1 year
API
- SQS = message oriented
- SWF = task oriented
SQS = handle duplicated messages and ensure message only processed once SWF = task assigned only once and Never duplicated
SWF = keeps track of all tasks and events in an app SQS = need to implement own app-level tracking
SWF actors
workflow starter - app that can start a workflow
deciders - control the activity tasks in a workflow execution
activity workers - carry out the activity tasks
SNS - Simple notification service
makes it easy to set up, operate, and send notifications from the cloud
publishes messages from an app and immediately deliver them to subscribers or other apps
instant push-based delivery
SNS notification delivery
direct to mobile devices
via SMS or email to SQS queues or any HTTP endpoint
SNS topic
An access point for allowing recipients to subscribe for identical copies of the same notification
SNS vs SQS
both are messaging systems in AWS
SNS = push SQS = pull/poll
Elastic transcoder
media transcoder in the cloud
convert media files into different formats
Kinesis
platform to send streaming data to
provides capability to load and analyst data
kinesis types
kinesis streams
kinesis fireshose
kinesis analytics
kinesis streams
stores data for 24 hours (default) up to 7 days
stores data in shards
Consumers (EC2 instances) access data in shards
kinesis firehose
no consistent storage; no data persistence
need to process storage as data comes in
kinesis analytics
works with streams and firehose
analyzes data in either streams or firehose on the fly w/in kinesis
Web identify federation
lets you give users access to AWS resources after authenticating with an identity provider like amazon, facebook, or google
Cognito
Amazons web identity federation service
provides temp creds that map to an IAM role
syncs user data for multiple devices
recommended for all mobile apps aws services
Cognito recommended approach
user authenticates w/ facebook/google/etc
gets auth token
sends token to cognito
cognito provides access to AWS environment
Cognito user pools
directories used to manage sign-up and sign-in functionality
email address/password
JWT - JSON web token
generated by a successful authentication
Cognito identity pools
provide temp AWS creds to access AWS services like S3 or dynamoDB
actual granting of access to AWS resources
Cognito syncing
push syncing to push updates and sync user data across multiple devices
Works via SNS
DLQ - dead letter queue
like a lost and found for unclaimed messages
SNS - messages failed to deliver are sent to an SQS queue and held for analysis/reprocessing
SQS - messages sent to SQS that exceed the queues maxReceiveCount
Lambda - when failure occurs, will retry twice. If still failing, will send to DLQ
Fanout patter
send a message to an SNS topic and have multiple SQS queues pull from the single SNS topic
S3 event notifications
allows you to receive notifications when certain events happen in a bucket
Can send to: SQS, SNS, Lambda
Can use filters
May miss an event notification -> enable versioning
S3 events that can trigger event notification
Object created Object removed Object restored RRS Object lost Replication -failure -exceed 15 min
