Merge 2 Flashcards
Where is data in DynamoDB stored?
SSD Storage
Where is DynamoDB spread?
across 3 geographically distinct data centers (not AZs)
Default consistency model for DynamoDB
Eventual Consistent Reads. Strongly consistent reads is the other
Eventual Consistent Reads
Consistency across all copies of data is usually reached within 1 second. repeating a read after a short time should return the updated data (best read performance)
Strongly Consistent Reads
returns the result that reflects all writes that received a successful response prior to the read
If low latency is important
pick Eventual consistent reads. Otherwise, pick strongly consistent reads
DynamoDB pricing
Provisioned throughput capacity, storage
How expensive is DynamoDB?
Expensive for writes, cheap for reads
Use case for DynamoDB
If DB is read-heavy, scalabilty and good performance are important, SQL is not necessary
default read and write capacity units when creating a DynamoDB table
5, 5
can you purchase reserved capacity?
Yes, 1 or 3-year terms
Item
Row
Important metrics to know
Read capacity Units, Write capacity units (both provisioned/consumed)
Is RedShift fully managed?
Yes
RedShift configuration
Single node (160 GB), multi-node
Multi node consists of
Leader node, compute node
Leader node
manages client connections and receives queries
Compute node
store data and perform queries and computations. up to 128 compute nodes
Does RedShift organize data by rows or columns?
Columns, so improved performance
Redshift is fast because
Massively Parallel processing, advanced compression
RedShift pricing
compute node hours,backup, data transfer w/in a VPC
RedShift security
encrypted in transit (SSL), encrypted at rest (AES-256), redshift takes care of key management by default
Redshift availability
1 AZ, can restore snapshots to new AZs in case of outage
Elasticache can be used for
improving application performance/latency/throughput for many read-heavy apps by storing critical pieces of data in memory for low-latency access (social networking, gaming, media sharing, Q/A portals)
Memcached
memory object caching system
Redis
open source in-memory key-value store that support data structures such as sorted sets and lists.
Elasticache supports
Master/slave replication, multi-AZ
If a database is under a lot of stress, which service would you use?
Elasticache if read heavy and not prone to frequently changing
Redshift if reason for stress is OLAP
What is Aurora?
MySQL-compatible, relational database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open-source databases
Performance, Price: Aurora vs MySQL
Aurora: 5x better, 1/10 cheaper
Aurora scaling
start with 10 GB, scales in 10 GB increments to 64 Tb
Compute resources scaling
up to 32 vCPUs and 244 Gb memory
How many copies of Aurora data?
2 per AZ, minimum of 3 AZs, so minimum of 6
How many copies can be lost w/o affecting availability?
2 - write, 3 - read
Aurora is self-healing, what is meant by that?
data blocks and disks are continuously scanned for errors and repaired automatically
2 Aurora replicas
Aurora replicas (15), MySQL Read replicas (5)
Aurora Replica vs MySQL read replica
which has failover?
Aurora read replica has failover, MySql read replica doesn’t
Failover logic
tier 0 is highest
What is SQS?
web service that allows you to access a message queue that can be used to store messages while waiting for a computer to process them.distributed queue system that enables web services apps to quickly and reliably queue messages that one component in the app generates to be consumed by another component
What is a queue?
temporary repository for messages that are awaiting processing
Example of SQS:
Image is uploaded to web app, web app alerts SQS that image has been uploaded and a job needs to be executed on it. app servers may queue SQS system and notice new job that comes in and access that image and perform a task. app servers will update queue and remove message from the queue
What is meant by decoupling/loosely coupling in SQS?
If web server goes down, for example, message is still in SQS and app servers not affected
T/F: Using SQS, you can decouple the components of an app so they run independently, with SQS easing message management between components.
TRUE
Can any component of a distributed app store messages in a fail-safe queue?
Yes
How much text can be contained in a message?
256 KB, any format
How can a component retrieve messages?
with the SQS API
The queue acts as
a buffer between the component producing and saving data and the component receiving the data for processing.
The queue resolves…
…issues that arise if the producer is producing work faster than the consumer can process it, or if the producer or consumer are only intermittently connected to the network
How many times does SQS deliver each message?
at least once
How many readers and writers can interact with the same queue?
multiple
How many components can a single queue interact with?
Many, simultaneously.
Do components need to coordinate with each other to share the queue?
No
SQS is engineered to
always be available and deliver messages
Does SQS guarantee first in first out delivery?
No
Is order important?
No, but you can place sequencing information in each message.
Is SQS pull or push?
pull
sqs visibility timeout
how long message will be visible in the queue
when does visibility timeout start?
when app server picks up the message
when would a message be processed/deleted?
during visibility timeout period
what happens if visibility timeout expires?
message will stay in queue and won’t be deleted and another app server will pull it
T/F: SQS can be integrated with Auto Scaling
True. Useful for when queue gets too large and more app servers can spin up
max visibility timeout period
12 hours
Best design practice
design system so that processing a message more than once does not create errors/inconsistencies
How is message billing based?
64 kb chunks. each chunk billed as one request
Cost for requests per month
First 1M free, after that it’s 50 cents per million requests
How many messages can a request have?
Minimum 1 message, maximum 10 messages, up to maximum 256 kb
What is SWF?
web service that makes it easy to coordinate work across distributed app components
SWF enables apps to be designed as…
a coordination of tasks
Tasks represent
invocations of various processing steps in an app which can be performed by executable code, web service calls, human actions, and scripts
SQS vs SWF retention period
SQS - 14 days, SWF - 1 year
SQS vs SWF - APIs
SQS - message oriented, SWF - task oriented
SQS vs SWF - duplication
SWF - task assigned only once, never duplicated
SQS - need to handle duplicated messages, make sure message is processed only once
SQS vs SWF - tracking
SWF - keeps track of all the tasks and events in an app
SQS - need to implement your own app-level tracking, especially if your app uses multiple queues
SWF Actors
Workflow Starters, Deciders, Activity Workers
Workflow Starters
an app that can initiate (start) a workflow. example: e-commerce website when placing an order or a mobile app searching for bus times
Deciders
control the flow of activity tasks in a workflow execution. if something has finished in a workflow (or fails) a decider decides what to do next
Activity Workers
carry out the activity tasks
What is SNS?
web service that makes it easy to set up, operate, and send notifications from the cloud. provides developers w/ a highly scalable, flexible, and cost-effective capability to publish messages from an app and immediately deliver them to subscribers or other apps
Common use case for SNS
notifies you in case of auto scaling, CloudWatch event
push notifications to
Apple, Google, Fire OS, and Windows devices, as well as Android devices in China w/ Baidu Cloud Push
What can SNS do other than push cloud notifications to mobile devices?
deliver notifications by SMS or email, to SQS queues, or to any HTTP endpoint. can also trigger Lambda functions
When message published to an SNS topic that has Lambda function subscribed to it
Lambda function is invoked with the payload of the published message. the lambda function receives the message payload as an input parameter and can manipulate the info in the message, publish the message to other SNS topics, or send the message to other AWS services
SNS allows you to group multiple recipients using
topics
Topic
access point for allowing recipients to dynamically subscribe for identical copies of the same notification.
One topic can support deliveries to
multiple endpoint types (e.g. group together iOS, Android, SMS)
When you publish once to a topic
SNS delivers appropriately formatted copies of your messages to each subscriber
to prevent messages from being lost
all messages published to SNS are stored redundantly across multiple AZs
SNS benefits
Instantaneous, push-based delivery (EC2 instances do not need to pull) simple APIs and easy integration with apps flexible message deliver over multiple transport protocols pay as you go
SNS vs SQS
both messaging services, SNS - Push, SQS - Pull
SNS pricing
50 cents per million SNS requests, 6 cents per 100k notifications over HTTP, 75 cents per 100 notifications over SMS, $2 per 100k notifications over email
What is Elastic Transcoder?
Media transcoder in the cloud. Converts media files from original source format into different formats that will play on smartphones, PCs, tablets, etc. transcoding presets for popular output formats. pay based on the minutes you transcode, resolution at which you transcode
What is API Gateway?
Amazon API Gateway is a fully managed service that makes it easy for developers to publish, maintain, monitor, and secure APIs at any scale. With a few clicks in the AWS Management Console, you can create an API that acts as a “front door” for applications to access data, business logic, or functionality from your back-end services, such as applications running on Amazon Elastic Compute Cloud (Amazon EC2), code running on AWS Lambda, or any web application.
API gateway architecture
User makes a request through API gateway, which triggers a Lambda function or routes request to EC2 instances. That request is then cached at API gateway, so subsequent requests would return the cached endpoint
What is API caching?
Can enable API caching to cache your endpoint’s response. with caching, you can reduce the number of calls made to your endpoint and also improve the latency of the requests to your API. when you enable caching for a stage, API gateway caches responses from your endpoint for a specified TTL period in seconds. API gateway then responds to the request by looking up the endpoint from the cache instead of making a request to your endpoint
What can API gateway do?
Low cost and efficient, scales effortlessly (w/o autoscaling groups or EC2 servers but can cause security concerns), you can throttle requests to prevent attacks, connect to CloudWatch to log requests
Same origin policy
In computing, the same-origin policy is an important concept in the web app security model. Under the policy, a web browser permits scripts contained in a first web page to access data in a second web page, but only if both web pages have the same origin/domain name
Cross-Origin Resource Sharing (CORS)
CORS is one way the server at the other end (not the client) can relax the same-origin policy.It is a mechanism that allows restricted resources (e.g. fonts) on a web page to be requested from another domain outside of the domain from which the first resource was served
Error: “Origin policy cannot be read at the remote resource”
You need to enable CORS on API gateway
T/F: API gateway has caching capabilities to increase performance
TRUE
T/F: API gateway is low cost and scales automatically.
TRUE
If you are using Javascript/AJAX that uses multiple domains with API gateway,
ensure you have enabled CORS on API Gateway
