AWS Developer Associate - by Thomas Haslett Flashcards
S3 Encryption: Protecting Data In-Transit (KMS)
1) Using an AWS-KMS managed customer (master) key
- Client gets a unique key for each object
2) On Upload:
- Send request to AWS KMS for key
- AWS KMS returns an encryption key
3) On Download:
- Client downloads encrypted object with their cipher blob stored in metadata -> blob to KMS -> get plain text key -> decrypt object
DynamoDB: Scans
1) Reads every item in a table and is operationally inefficient
2) Looks for all items and attributes in a table by defualt
SNS: Subscribers
1) Subscribe to a topic to receive published messages
2) Subscribers are end-points and include:
- Mobile apps
- Web servers
- Email addresses
- Amazon SQS queue
- AWS Lambda
Cloud Formation: Intrinsic Functions
1) Used to pass in values that are NOT available until runtime.
Example: “GetAtt”
SWF: Domains
1) A domain is used to help determine scope of work flows
2) Multiple workflows can live in a domain
3) Workflows cannot interact with workflows in OTHER domains
DynamoDB: Atomic Counters
1) Allows you to increment or decrement the value of an attribute without interfering with other write request
2) Request are applied in the order that they were received
3) Updates are NOT Idempotent: It will update the value each time it is called
SNS: Push Notification Setup Process
1) Needs a device token
2) There are Device Tokens and Registration IDs, depending on the mobile platform.
3) Request credentials from the mobile platform
4) Request Token from the mobile platform
5) Create a platform application object
6) Create a platform endpoint object
7) Publish a messages to the mobile endpoint
Cloud Formation
1) Allows you to create and provision resources in a reusable template fashion
2) Turns your resources into Stacks
3) Allows you to “source control” your infrastructure
4) Templates are in JSON format
S3 Performance: Sequence Pattern Issue
ISSUE: Using sequential object names cause writes to the same partition (overload I/O). KEY NAME is used to decide which partition they key is stored in
SOLUTION: Introduce randomness by using Hex Hash Prefix:
- use a has (like MD5) of a character sequence
- Pick a specific number of characters from that hash to use as the prefix OR reserve ID string
Cloud Formation: Resources
1) This is where you create different resources like S3, EC2, etc.
2) This section is REQUIRED
DynamoDB: Local Secondary Indexes
1) Partition key must be the same, but the sort key is different
2) “Local” because every partition is scoped to a table partition with the same partition key
3) uses the tables provisioned throughput
S3 Encryption: Protecting Data At-Rest (S3-managed)
1) AWS provides server-side encryption before saving data to disk
2) Add the “x-amz-server-side-encryption” request leader to your upload request
3) Uses AES-256
4) Bucket policies can require all objects use server-side encryption
5) Alternatives:
- KMS managed keys
- Customer provided keys
Deleting Versioned Objects
1) All versions remain in the bucket, but S3 inserts a “delete” marker
2) The “delete” marker becomes the current version
3) GET request retrieve the latest version
4) If current version has a “delete” marker, it returns a 404 error
5) You can get previous versions by specifying an ID
Cloud Formation: Template Sections
1) AWSTemplateFormatVersion
2) Description
3) MetaData
4) Parameters
5) Mappings
6) Conditions
7) Resources
8) Outputs
DynamoDB: Scans (benefits)
1) Scans can apply filters to the results to refine values
2) Can return only specific attributes with the “ProjectionExpression” parameters
S3: IAM Policies
1) User policy
2) Can create multiple users and give them the same policy or different policies
3) Policies are attached and can be detached
4) Cannot grant anonymous permissions
Intrinsic Functions: FN::Select
1) Returns a single object from a list of objects by index
S3: Eventual Data Conistancy
1) State reads possible
2) Lowest read latency
3) Highest read throughput
S3 Performance: GET Intensive workloads
1) Use Cloud Front
- Distributes content with lower latency & high transfer rate
- Cache objects
- Fewer direct request to S3
Cognito Sync
1) Sync data across mobile devices and the web
2) Client libraries cache data locally
DynamoDB: ALL
Projection Type
1) All attributes are projected (biggest index - least performant
SWF: Workers
1) Process that performs an activity that is part of the workflow
2) Workers poll SWF for new tasks that they need to perform
3) After receiving a task, the worker will process the task however it is instructed to do so, then report back to SWF
4) Workers CAN consist of a server (code being executed), a human worker can also be used to process as task
DynamoDB: Features
1) Can be used with (via) the AWS console or API
2) Flexible data model with attributes and items
3) Supports different levels of consistency (eventually & strongly)
4) Conditional updates and concurrency control (automatic counter)
5) Pay for what you use
6) Integrates with monitoring
7) Integrates with AWS Big Data (EMR & Redshift)
8) Integrates with other services through “streams”
SNS: Managing Access
1) Access is controlled with policies
2) SNS has it’s own IAM permissions system
3) You can control:
- Who is allowed to publish a topic
- Who is allowed to subscribe to a topic
- and under what conditions
S3: ACLs
1) Used for both buckets & objects
2) Grant read/write permissions to other AWS accounts
3) You cannot grant conditional permissions
4) you cannot explicitly deny permissions
5) An object ACL is the only way to manage access to objects not owned by the bucket owner
6) Uses XML format
S3: URL
1) Every hosted bucket receives it’s own URL
.S3-website-.amazonaws.com
SQS: Core Features
1) Redundant infrastructure
2) Multiple writes & reads
3) Queues can be configured
4) Access Control
5) Delay queues (0-15 seconds)
6) PCI compliant
Cloud Formation: Mappings
1) Lets you map keys to values
DynamoDB: Partition Key
1) A type of primary key
2) Also known as a “hash attribute”
3) Simple primary key composed of an attribute
4) Used to retrieve data
5) Must be unique
Intrinsic Functions: Ref
1) Returns a resource or value based on a logical name or parameter
Error 409
Error 409 = Conflict
1) S3 Bucket already exist
2) Bucket is not empty (when trying to delete)
3) Bucket name already taken
SQS: Message Lifecyle
1) Component “1” sends message “A” to a queue, and then the message is redundantly distributed across SQS servers
2) When component “2” is ready, it retrieves the message from SQS. While message “A” is being processed it remains in the queue, but “Visibility Timeout”
3) Component “2” deletes the message from the queue during that “Visibility Timeout” or it will get deleted
SNS: Access Control Policies
1) The AWS account owner had the only permissions by default
2) ALLOWS override default DENIES
3) Explicit DENIES override ALLOWS
4) Order of policies does NOT matter
5) Can grant access to another account API call “AddPermission”
S3: AWS Limits/Restrictions
1) AWS account can have up to 100 S3 buckets
2) No limit on the number of objects
3) Bucket name must:
- Be a min of 3 characters and a max of 63
- Can only contain lowercase letter, numbers, periods & hyphens
- Must start with a letter or number
- Periods & hyphens cannot follow each other
- Can’t be an an IP address
DynamoDB: Primary Key
1) Is a unique identifier for each ITEM in a table
2) No two items can have the same key
S3: Restoring Version Objects
1) Any earlier version can be restored by:
- Copying a previous version into the same bucket will restore it as the current version
- Permanently deleting the current version
- Copying an earlier version GETs the version and PUTs it in the bucket, giving it a new ID (the new ID is used as current version)
DynamoDB: Queries
1) Allows you to find items using ONLY primary key-values from a table OR secondary index
2) more efficient then SCAN
DynamoDB: INCLUDE
Projection Type
1) Only specified attributes are projected
AMI API: “RegisterImage”
1) Occurs during the FINAL process of creating an AMI
Error 404
Error 404 = Not found
1) Bucket does not exist
2) Key does not exist
SNS Message Data: TopicARN
ARN = Amazon Resource Name
1) ARN fro the topic that this message was published to
Federation (mobile apps)
1) Use
-Facebook
-Amazon
-Google
To authenticate mobile app users and then grant access to temp resources
SQS API: “ChangeMessageVisibility”
1) Changes a messages Visibility Timeout
2) Adds specified time to current remaining time
*Note: Visibility Timeout can be 0-12 hours
Amazon Cognito
1) Creates unique identity for users
2) Authenticates identities with identity providers
3) Supports unauthenticated identities (anonymous)
4) Saves mobile user data
5) Uses credentials obtained to sync data with Cognito sync
SNS Message Data: Subject
1) Subject Parameter
2) Optional parameter
SNS Message Data: Message
1) Message value specified when the notification was published to the topic
2) A string that describes the message
S3: Versioning
1) Allows multiple versions of an object
2) Protects against unintended overwrites and deletions
3) Automatically archives objects
4) Versioning is at the BUCKET LEVEL
5) Configured via console or SDK
6) “off” by default
SNS Message Data: MessageID
1) Universally Unique Identifiers (UUID)
2) Same ID must be used for retries
S3: Bucket Policies
1) Resource-based policy
2) Created via JSON
3) Can grant other AWS accounts or IAM users permissions for the bucket/object
4) SHOULD be used to manage cross-account permissions for all S3 permissions
5) Limited to 20kb in size
DynamoDB: Scans (negatives)
1) The larger the data set in the table, the slower the performance of a scan
2) The more filters on the scan, the slower the performance
3) Return only filtered results
4) Only eventually consistent reads available
SQS: Advantages
1) Components are loosely coupled (decoupled) for high availability, reliability, and scalability
2) Protects against losing data on application failure
DynamoDB: Exceeding Throughput
1) Request exceeding the allocated throughput may be throttled
2) With Global Secondary Indexes, all indexes must have enough Write Capacity OR the write might get throttled (even if the write doesn’t effect index)
3) You can monitor throughput in the AWS Console
SNS: Topics
1) Channel used to send messages and subscribe to notifications
2) Names MUST be unique
3) Names are limited to 256 characters
4) All letters, numbers, hyphens and underscores allowed in name
5) Topics and messages are stored redundantly on multiple servers and data centers
SQS API: “GetQueueURL”
1) Gets a queue’s URL
2) Instead of manually hardcoding a queue’s URL, we can retrieve it with an API call
SWF (definition)
1) Is a task coordination and state management service for cloud applications
Features:
a) Distributed
b) Highly scalable
c) Work with both on-premise and cloud applications
d) A workflow execution can last up to 1 year
e) A workflow can consist of human events
f) Guarantees order in which activities/tasks occur
SQS API: “ReceiveMessageWaitTimeSeconds”
1) Enable long polling on a queue (change default)
2) If value > 0
S3: Static Web Hosting
1) Host static html files
2) Specify index file
3) specify custom erro file
4) Supports domains and redirects
5) Gives a default URL
6) redirects from www.example.com to example.com
7) Route53 integration fro custom domains
8) Bucket names must match domain name
Identities (STS)
A user or “identity” within a service (like facebook, google)
DynamoDB: Keys_ONLY
Projection Type
1) Only the index and primary keys are projected (smallest index, more performant)
DynamoDB: Global Secondary Indexes
1) Partition Key and Sort Key can both be different from those on the table
2) Had it’s own provisioned throughput
S3 Encrypting: Protecting Data In-Transit (client-side master key)
1) Master Keys and unencrypted data are NEVER sent to AWS
2) On Upload:
-S3 client generates random data key and encrypts
-Uploads a “material description” as part of metadata
On Download:
-Client downloads encrypted object
-The metadata tells client which master key to use
-Using that key, client decrypts the data key
-Data key used to decrypt object
S3: Choosing a region
Choose a region for:
1) Optimized latency
2) Minimize cost
3) Address regulatory requirements
*NOTE: Objects stay in a region UNLESS explicitly transfered
Federation
1) Grant temp access to users on a corporate network without having to create IAM creds for each user
2) A list of domain users with a separate list of users in another domain (facebook, LDAP, google)
SNS Message Data: Type
1) Type of the message
(i. e. notifications are type “notifications”)
- Sub confirmation
- Notification
- Unsub confirmation
DynamoDB: Projected Attributes
1) Attributes copied from the table to the index, in addition to the primary key attributes and index key attributes
Intrinsic Function: “FindInMap”
1) Returns the value of a key from a specified mapping
Cloud Formation: Parameters
1) Values you can pass in right before template creation
2) Allows you to customize templates
3) Can have DEFAULT as well as ALLOWED values
Security Token Service (STS)
1) Allows you to grant a trusted user tempting access and limited access to AWS resources
Error 403
Error 403 = Forbidden (no access)
1) Access denied
2) Caused by Role, security group/ACL issue
DynamoDB: WRITE THROUGHPUT with Local Secondary Indexes
1) Adding, updating, or deleting an ITEM in a table also cost write capacity to perform the action on the local index
2) new ITEM/Update ITEM = one write operation in the index
3) If you change the value of an indexed key attribute = two writes
4) Delete = one write
SQS API: “SendMessage”
1) Send message to a queue
2) Can be used to set the message “delay”, as well as message attributes, and the message body
Cross Region Resource Sharing (CORS)
1) Sharing/Accessing resources stored in one bucket with another
2) MUST be enabled (disabled by default)
SNS: Mobile Push Notifications
1) SNS provides the ability to send notifications directly to apps on mobile devices
2) Notifications sent to a mobile device can appear in the app as:
- Message alerts
- Badge updates
- Sound alerts
Intrinsic Functions: FN::Join
1) Concatenates elements, separated by a specific delmiter
DynamoDB: READ THROUGHPUT with Local Secondary Indexes
1) Use the same read/write capacity from parent table
2) IF you read only index keys and projected attributes, then calculations are same as table (calculate using the size of the index entry)
3) IF queried attributes are NOT projected attributes or keys, we get extra latency and read capacity cost
* NOTE: You use read capacity from the Index and every Item from the table
Cloud Formation: Description
1) Helps clearly differentiate between templates
DynamoDB: Grating Temporary Access
1) IAM roles
2) Web Identity federation (enterprise or web)
3) Amazon Cognito
Example: Mobile app needs to store user info
DynamoDB: Secondary Indexes
1) Lets you query data using alternate keys (flexibility)
2) Can provide better performance
3) There are GLOBAL & LOCAL secondary indexes
4) Up to five (5) global & local per table
SWF: Activity Task
1) A task assigned to a worker such as encode a video OR check inventory
S3 Performance (limits)
1) Burst (#of request per second)
IF > 300 put/list/delete
IF > 800 get
THEN contact AWS to prepare/avoid limit issues
2) Consistent high number of request per second
IF > 100 put/list/delete
IF > 300 get
Then follow best practice guidelines to avoid overwhelming the I/O capacity of a partition
DynamoDB: Essentials
1) Can scale up/down without downtime or performance degradation
2) You control performance through read/write capacity
3) Conditional updates & Concurrency Control
4) Change read/write at any time
S3: Versioning (when enabled)
1) Existing objects are unchanged
2) Added objects are given unique version IDs
3) ID set by AWS, cannot manually set or change IDs
SNS Message Data: SigningCertURL
1) The URL to the certificate that was used to sign the message
DynamoDB: Essentials
1) Can scale up/down without downtime or performance degradation
2) You control performance through read/write capacity
3) Conditional updates & Concurrency Control
4) Change read/write at any time
S3: Versioning (when enabled)
1) Existing objects are unchanged
2) Added objects are given unique version IDs
3) ID set by AWS, cannot manually set or change IDs
SNS Message Data: SigningCertURL
1) The URL to the certificate that was used to sign the message
SQS Limitations
1) Message Size: 256kb of text (any format)
2) Up to 120,000 “in-flight” messages
SQS API: “SetQueueAttributes”
1) Sets/Changes an attribute
Example: Attribute.Name:VisabilityTimout
-Sets the default queue visibility timeout
SQS: “VisibilityTimeout”
1) It is used to block other components from processing a message
2) You can choose what the timeout is, and you can extend it
3) Can be controlled via SQS API
4) Limits: 0-12 hours
Identity Store (list)
1) Facebook
2) Active-directory
3) Google
4) Amazon
SNS Message Data: UnsubscribeURL
1) URL that you can use to unsubscribe
API Credentials with ACCESS KEYS
1) Used to sign request
2) Used by:
- AWS SDKs
- AWS CLI
3) Can be disabled & deleted, but not retrieved
4) Can be temporary
5) useful when connecting from outside of AWS (like your computer or application)
S3: Consistent Reads
1) No state reads
2) Potential higher read latency
3) potential lower read throughput
Cloud Formation: RollBack
1) If a Stack fails to create a resource, by default a stack will “rollback”
2) Removal of all created resources after a failed creation, or after cancelling creation
3) Rollback CAN be disabled via API
SQS Short Polling
1) Returns results immediatly, even if the queue is empty
2) It only checks a subnet of servers, which can cause false empty responses
S3: Bucket Permissions
1) Resource based policies
2) Specify who is allows to access
3) What the user can do with those resources
4) AWS gives full permission to the owner of a resource (bucket, object)
5) Resource owners grant access to others, even cross-account
* NOTE: The bucket owner paying the bills can deny access/modify objects regardless of who owns them
DynamoDB: Keys (facts to know)
1) Primary key = partition key = “hash attribute”
2) Sort key = “range attribute”
SNS Message Data: Timestamp
1) The time (GMT) when the notification was published
SQS Message Retention Period
1) Amount of time a message will “live” in a queue if it is NOT deleted
2) 1 minute - 14 days
DynamoDB: Provisioned Throughput
1) Flexibility to change read & write capacity:
- table creation
- or at any time after without downtime/degradation
2) Automatically allocates machine resources
3) Ability to reserve capacity
SNS Message Data: Signature
1) Base64-encoded “SHA1 with RSA” signature
- Message
- MessageID
- Subject
- Type
- Timestamp
- TropicARN values
Error 400
Error 400 = bad request
SQS “ReceiveMessageWaitTime”
1) If set to > 0, long polling in enabled
2) It is the maximum amount of time a long polling call will wait for a message to become available before returning empty
3) Limits: 0-20 seconds
AMI API: “DescribeImages”
1) API call to describe an AMI
Multi-Part Upload (benefits)
1) Can upload independently, in any order, and in parallel
2) If any part fails to upload, you can retransmit that part
3) You can pause/resume uploads
4) You can upload objects as they are being created
5) Object is reassembled after calling “CompletMultiPartUpload” API
Cloud Formation: Meta Data
1) JSON objects that provide details about the template
DynamoDB: Conditional Writes
1) Is Itempodent
2) Helps coordinate writes
3) Checks for condition before proceeding with operations
4) Supported for PutItem, DeleteItem, UpdateItem
5) Specify conditions in “ConditionExpression”: Can contain attribute names, conditional operations, and build-in functions
6) A failed conditional write returns “ConditionalCheckFailedException”
DynamoDB: Limits (api)
1) Up to 10 ReadTable, UpdateTable, and DeleteTable actions running simultaneously
2) A single BatchGetItem can get a max of 100 Items (must be < 16mb in size)
3) A single BatchWriteItem can contain up to 25 PutItems OR DeleteItems request (16mb)
4) Query and Scan results set is limited to 1mb of data per call.
* NOTE: LastEvaluatedKey in the response can be used to retrieve more data
SNS (definition)
1) It is a pub-sub service for messaging
2) Publishes messages to subscribe via Topics
3) Scalable & highly reliable
4) Supported through console, API & SDK
SWF: Decision Task
1) Tells the decider that the state of the workflow execution has changed
2) Allows the decider to determine what the next activity is to be performed
3) Decision tasks occur whenever the state of the workflow changes (“task completed”)
S3: Objects (facts & numbers)
1) Size: 0 bytes - 5 TB
2) Objects larger than 5 GB require multi-part upload API
3) Multi-part uploading is recommended for ALL files larger than 100mb
4) Objects can be encrypted before being saved to disk- AND decrypted when downloaded
SNS: Publishers
1) The owner who creates a topic and controls access to it
You can publish from:
a) CLI
b) Your application (http)
c) SDKs
d) Other AWS Services
Cloud Formation: Outputs
1) Outputs values that you would like to see from console or API call
Example: Return bucket name you just created
S3: Encryption (options)
1) Protect data “in-transit”
- Use SSL or client side encryption
2) Protect data at rest
- Request AWS S3 to encrypt data
SNS Message Data: SignatureVersion
1) Version of the SNS signature used
DynamoDB: Limits (size)
1) 265 tables per region (increase on request)
2) Partition key length: 1 byte - 2048 bytes
3) Sort key length: 1 byte - 1024 bytes
4) Item size: 400kb including attribute name & value
SQS: Dead Letter Queues
1) Queues that other queues can send messages to when those messages could not be successfully processed
2) You can then analyze those messages
DynamoDB: Partition & Sort Key
1) A type of primary key
2) Also known as a “Range attribute” key
3) Composite primary key composed of TWO attributes (partition key & sort key)
4) Two items can have the same partition key, but must have different sort keys
SNS: Message Data
1) JSON formatted key-value pairs
2) Allows developers to grab the message date and parse it
3) POSTs to http/s end points with specific headers
4) Allows developers to verify the authenticity of the message
SNS: Items in a Message Body (list)
1) Message
2) MessageID
3) Signature
4) SignatureVersion
5) SigningCertURL
6) Subject
7) Timestamp
8) TopicARN
9) Type
10) UnsubscribeURL
Cloud Formation: Conditions
1) Check values before deciding what to do
2) Allows you to create different resources in the same template depending on the condition value
Example: Create different environments for production or dev
SQS API: “DeleteMessage”
1) Deletes a message
Identity Broker
1) A service that allows you to take an identity from “point A” and federate it to another identity at “point B”
DynamoDB: Scans (if you must use)
1) You can reduce the “page size” of an operation with the “limit” parameter, to limit how much data you try to retrieve at the same time
2) Avoid scan on mission critical tables
3) Program your application logic to retry any request that receives a response code saying you exceeded provisioned throughput (or increase your throughput)
SQS Long Polling
1) Long Polling doesn’t return a response until there is a message in the queue
2) It checks every server to avoid false empty responses
3) Should be used whenever possible as it can reduce cost and false empty responses
DynamoDB: Essentials
1) Fully managed NoSQL SB
2) Can scale up/down without downtime or performance degradation
3) Manage data, NOT hardware/software
4) Built in monitoring
5) Consistent & fast performance:
- stored on fast SSDs
- You control performance through read/write capacity
- Can spread load across servers & tables
- Replicated across multiple-AZ’s in a region
Cloud Formation: AWSTemplateFormatVersion
Specifies which template you want to use
Cross-Account Access
1) Allows an IAM user in one account access resources in another account
Error 500
Error 500 = Internal Server Error
Intrinsic Functions: FN::GetAZ
1) Gets the AZ for a specific region
DynamoDB: Queries (benefits)
1) Returns the ITEM matching the primary key search
2) Much more efficient because it searches only indexes
3) Returns all attributes of an ITEM, OR only the ones you want
4) Is eventually consistent by default (but you can request consistent)
5) Can use conditional operations and filters to return precise results
Permanently Deleting (S3 versioning)
1) Must specify and ID
2) AWS will then set the next ID # to the “current” version
