S3 Flashcards
- Backup and storage
- Disaster Recovery
- Archive
- Hybrid Cloud storage * Application hosting
- Media hosting
- Data lakes & big data analytics * Software delivery
- Static website
Amazon S3 Use cases
Amazon S3 allows people to store __________ in ___________.
objects (files)
“buckets” (directories)
Buckets must have a ___________ (across all regions all accounts)
globally unique name
What level are Buckets defined at?
region level
Amazon S3 Objects (files) have a ______?
Key
What is the max size of an Object?
5TB (5000GB)
If uploading more than 5GB, must use _____________?
“multi-part upload”
3 things that an object can have other than the body??
- Metadata (list of text key / value pairs – system or user metadata)
- Tags (Unicode key / value pair – up to 10) – useful for security / lifecycle
- Version ID (if versioning is enabled)
2 ways to grant access to a S3 bucket?
User-Based
Resource-Based
Which API calls should be allowed for a specific user
IAM Policies (User-Based)
What are the 3 Resource-Based Amazon S3 – Security?
Bucket Policies
Object Access Control List (ACL)
Bucket Access Control List (ACL)
Bucket wide rules from the S3 console - allows cross account
Bucket Policies (Resource-Based)
Which resouce based security is finer grain and can be disabled
Object Access Control List (ACL) (Resource-Based)
Which resouce based security is less common and can be disabled
Bucket Access Control List (ACL) (Resource-Based)
an IAM principal can access an S3 object if …..
- The user IAM permissions ALLOW it OR the resource policy ALLOWS it
- AND there’s no explicit DENY
S3 Bucket Policies JSON based policies (4)
- Resources: buckets and objects
- Effect: Allow / Deny
- Actions: Set of API to Allow or Deny
- Principal:The account or user to apply the policy to
Bucket Policies JSON based policies - buckets and objects
Resources
Bucket Policies JSON based policies - Allow / Deny
Effect
Bucket Policies JSON based policies - Set of API to Allow or Deny
Actions
Bucket Policies JSON based policies - The account or user to apply the policy to
Principal
Use S3 bucket for policy to:
- Grant public access to the bucket
- Force objects to be encrypted at upload
- Grant access to another account (Cross Account)
Can be set at the account level
Bucket settings for Block Public Access
S3 can host __________ and have them accessible on the Internet
static websites
f you get a _________ error, make sure the bucket policy allows public reads!
403 Forbidden
Amazon S3 - Versioning is enabled at what level???
Bucket Level
2 best practices to version your buckets
- Protect against unintended deletes (ability to restore a version)
- Easy roll back to previous version
Any file that is not versioned prior to enabling versioning will
have version ______
“null”
Suspending versioning does OR does not delete the previous versions
DOES NOT
2 types of Amazon S3 – Replication
- Cross-Region Replication (CRR)
- Same-Region Replication (SRR)
When using S3 replication, you must enable Versioning in _______ AND ________ buckets
source
destination
Can buckets be in different AWS accounts??
YES
Replication copying what kind of synchronized???
asynchronous
DO you need to give proper IAM permissions to S3
YES
Compliance, lower latency access, replication across accounts
CRR
Log aggregation, live replication between production and test accounts
SRR
After you enable Replication, only __________ are replicated
new objects
You can replicate existing objects using __________?
S3 Batch Replication
Replicates existing objects and objects that failed replication
S3 Batch Replication
- Can replicate delete markers from source to target (optional setting)
- Deletions with a version ID are not replicated (to avoid malicious deletes)
Replication DELETE operations
There is no _________ of replication
“chaining”
* If bucket 1 has replication into bucket 2, which has replication into bucket 3
* Then objects created in bucket 1 are not replicated to bucket 3
S3 Storage Classes (7)
- Amazon S3 Standard - General Purpose
- Amazon S3 Standard-Infrequent Access (IA)
- Amazon S3 One Zone-Infrequent Access
- Amazon S3 Glacier Instant Retrieval
- Amazon S3 Glacier Flexible Retrieval
- Amazon S3 Glacier Deep Archive
- Amazon S3 Intelligent Tiering
Can you move between classes manually or using S3 Lifecycle configurations
YES
- 99.99% Availability
- Used for frequently accessed data
- Low latency and high throughput
- Sustain 2 concurrent facility failures
- Use Cases: Big Data analytics, mobile & gaming applications, content distribution…
S3 Standard – General Purpose
- For data that is less frequently accessed, but requires rapid access when needed
- Lower cost than S3 Standard
S3 Storage Classes – Infrequent Access
- 99.9% Availability
- Use cases: Disaster Recovery, backups
Amazon S3 Standard-Infrequent Access (S3 Standard-IA)
- High durability (99.999999999%) in a single AZ; data lost when AZ is destroyed
- 99.5% Availability
- Use Cases: Storing secondary backup copies of on-premises data, or data you can recreate
Amazon S3 One Zone-Infrequent Access (S3 One Zone-IA)
2 types of S3 Storage Classes – Infrequent Access
Amazon S3 Standard-Infrequent Access (S3 Standard-IA)
Amazon S3 One Zone-Infrequent Access (S3 One Zone-IA)
3 types of Amazon S3 Glacier Storage Classes
Amazon S3 Glacier Instant Retrieval
Amazon S3 Glacier Flexible Retrieval (formerly Amazon S3 Glacier)
Amazon S3 Glacier Deep Archive
- Low-cost object storage meant for archiving / backup
- Pricing: price for storage + object retrieval cost
Amazon S3 Glacier Storage Classes
- Millisecond retrieval, great for data accessed once a quarter
- Minimum storage duration of 90 days
Amazon S3 Glacier Instant Retrieval
- Expedited (1 to 5 minutes), Standard (3 to 5 hours), Bulk (5 to 12 hours) – free
- Minimum storage duration of 90 days
Amazon S3 Glacier Flexible Retrieval
- Standard (12 hours), Bulk (48 hours)
- Minimum storage duration of 180 days
Amazon S3 Glacier Deep Archive
- Small monthly monitoring and auto-tiering fee
- Moves objects automatically between Access Tiers based on usage
- There are no retrieval charges
S3 Intelligent-Tiering
5 types of S3 Intelligent-Tiering
Frequent Access tier
Infrequent Access tier
Archive Instant Access tier
Archive Access tier
Deep Archive Access tier
S3 Intelligent-Tiering - (automatic): default tier
Frequent Access tier
S3 Intelligent-Tiering - (automatic): objects not accessed for 30 days
Infrequent Access tier
S3 Intelligent-Tiering - (automatic): objects not accessed for 90 days
Archive Instant Access tier
S3 Intelligent-Tiering - (optional): configurable from 90 days to 700+ days
Archive Access tier
S3 Intelligent-Tiering - (optional): config. from 180 days to 700+ days
Deep Archive Access tier
2 types of Amazon S3 – Lifecycle Rules
Transition Actions
Expiration actions
configure objects to move to another storage class
* Move objects to Standard IA class 60 days after creation
* Move to Glacier for archiving after 6 mont
Transition Actions
configure objects to expire (delete) after some time
* Access log files can be set to delete after a 365 days
* Can be used to delete old versions of files (if versioning is enabled)
* Can be used to delete incomplete Multi-Part uploads
Expiration actions
Can rules be created for a certain prefix
YES
Can rules be created for certain objectsTags
YES
- Help you decide when to transition objects to the right storage class
- Recommendations for Standard and Standard IA
- Does NOT work for One-Zone IA or Glacier
- Report is updated daily
- 24 to 48 hours to start seeing data analysis
- Good first step to put together Lifecycle Rules (or improve them)!
Amazon S3 Analytics – Storage Class Analysis
In general, ______________ pay for all Amazon S3 storage and data transfer costs associated with their bucket
Bucket owners
The requester instead of the bucket owner pays the cost of the request and the data download from the bucket
Requester Pays buckets
- Helpful when you want to share large datasets with other accounts
- The requester must be authenticated in AWS (cannot be anonymous)
S3 – Requester Pays
For S3 Event Notifications, is Object name filtering possible??
YES
How many “S3 events” can you create??
As many as you want
S3 event notifications typically deliver events in _______ but can sometimes take __________?
seconds
a minute or longer
What are the 4 places S3 Event Notifications can send notifications?
SNS
SQS
Lambda Functions
Eventbridge
- Advanced filtering options with JSON rules (metadata, object size, name…)
- Multiple Destinations – ex Step Functions, Kinesis Streams / Firehose…
- Capabilities – Archive, Replay Events, Reliable delivery
S3 Event Notifications with Amazon EventBridge
Your application can achieve at least ________ PUT/COPY/POST/DELETE or _______ GET/HEAD requests per second per prefix in a bucket.
3500
5500
What is the limits to the number of prefixes in a bucket??
No limits
- recommended for files > 100MB, must use for files > 5GB
- Can help parallelize uploads (speed up transfers)
Multi-Par t upload
- Increase transfer speed by transferring file to an AWS edge location which will forward the data to the S3 bucket in the target region
- Compatible with multi-part upload
S3 Transfer Acceleration
- Retrieve less data using SQL by performing server-side filtering
- Can filter by rows & columns (simple SQL statements)
- Less network transfer, less CPU cost client-side
S3 Select & Glacier Select
What are examples of S3 batch operations?
- Modify object metadata & properties
- Copy objects between S3 buckets
- Encrypt un-encrypted objects
- Modify ACLs, tags
- Restore objects from S3 Glacier
- Invoke Lambda function to perform custom action on each object
Perform bulk operations on existing S3 objects with a single request
S3 Batch Operations
What does a S3 Batch Operation job consist of?? (3)
a list of objects
the action to perform
optional parameters
S3 Batch Operations manages (4)
retries
tracks progress
sends completion notifications
generate reports
You can use ___________ to get object list and use ________ to filter your objects
S3 Inventory
S3 Select
You can encrypt objects in S3 buckets using one of 4 methods
Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3)
Server-Side Encryption with KMS Keys stored in AWS KMS (SSE-KMS)
Server-Side Encryption with Customer-Provided Keys (SSE-C)
Client-Side Encryption
- Encryption using keys handled, managed, and owned by AWS * Object is encrypted server-side
- Encryption type is AES-256
- Must set header “x-amz-server-side-encryption”: “AES256”
- Enabled by default for new buckets & new objects
Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3)
- Encryption using keys handled and managed by AWS KMS (Key Management Service)
- KMS advantages: user control + audit key usage using CloudTrail
- Object is encrypted server side
- Must set header “x-amz-server-side-encryption”: “aws:kms”
Server-Side Encryption with KMS Keys stored in AWS KMS (SSE-KMS)
- Server-Side Encryption using keys fully managed by the customer outside of AWS * Amazon S3 does NOT store the encryption key you provide
- HTTPS must be used
- Encryption key must provided in HTTP headers, for every HTTP request made
Server-Side Encryption with Customer-Provided Keys (SSE-C)
- Use client libraries such as Amazon S3 Client-Side Encryption Library
- Clients must encrypt data themselves before sending to Amazon S3
- Clients must decrypt data themselves when retrieving from Amazon S3
- Customer fully manages the keys and encryption cycle
Client-Side Encryption
SSE-KMS Limitation
- If you use SSE-KMS, you may be impacted by the KMS limits
- When you upload, it calls the GenerateDataKey KMS API
- When you download, it calls the Decrypt KMS API
- Count towards the KMS quota per second (5500, 10000, 30000 req/s based on region)
- You can request a quota increase using the Service Quotas Console
Encryption in flight is also called?
SSL/TLS
Amazon S3 – Encryption in transit (SSL/TLS) - HTTPS is mandatory for _______
SSE-C
Amazon S3 – Encryption in transit (SSL/TLS) - HTTPS is ________
recommended
Can you for encryption in Transit??
YES
What does CORS stand for??
Cross-Origin Resource Sharing
The requests won’t be fulfilled unless the other origin allows for the requests, using_________?
CORS Headers
If a client makes a cross-origin request on our S3 bucket, What do we need to do??
enable the correct CORS headers
Which CORS, You can allow for ____________
specific origin or for * (all origins)
Amazon S3 – MFA Delete will be required to ……
- Permanently delete an object version
- Suspend Versioning on the bucket
Amazon S3 – MFA Delete will NOT be required to ……
- Enable Versioning
- List deleted versions
To use MFA Delete, Versioning must be __________ on the bucket
enabled
Only the bucket ______________ can enable/disable MFA Delete
owner (root account)
S3 Access Logs - The target logging bucket ________ be in the same AWS region
MUST
How can you generate pre-signed URLs?
S3 Console, AWS CLI or SDK
What are the pre-signed URL’s expirations?
- S3 Console – 1 min up to 720 mins (12 hours)
- AWS CLI – configure expiration with –expires-in parameter in seconds (default 3600 secs, max. 604800 secs ~ 168 hours)
Examples of Amazon S3 – Pre-Signed URLs
- Allow only logged-in users to download a premium video from your S3 bucket
- Allow an ever-changing list of users to download files by generating URLs dynamically
- Allow temporarily a user to upload a file to a precise location in your S3 bucket
- Adopt a WORM (Write Once Read Many) model
- Create a Vault Lock Policy
- Lock the policy for future edits
(can no longer be changed or deleted) - Helpful for compliance and data retention
S3 Glacier Vault Lock
- Adopt a WORM (Write Once Read Many) model
- Block an object version deletion for a specified amount of time
S3 Object Lock
2 types of Retention Modes for S3 Object Lock
Retention mode - Compliance
Retention mode - Governance
- Object versions can’t be overwritten or deleted by any user, including the root user
- Objects retention modes can’t be changed, and retention periods can’t be shortened
Retention mode - Compliance
- Most users can’t overwrite or delete an object version or alter its lock settings
- Some users have special permissions to change the retention or delete the object
Retention mode - Governance
S3 Object Lock - protect the object for a fixed period, it can be extended
Retention Period
- protect the object indefinitely, independent from retention period
- can be freely placed and removed using the s3:PutObjectLegalHold IAM permission
Legal Hold
Simplify security management for S3 Buckets
Access Points
Each Access Point has (2)
- its own DNS name (Internet Origin or VPC Origin)
- an access point policy (similar to bucket policy) – manage security at scale
We can define the access point to be accessible ________?
only from within the VPC
You must create a __________ to access the
Access Point (Gateway or Interface Endpoint)
VPC Endpoint
The VPC Endpoint Policy __________ allow access to the target bucket and Access Point
MUST
To change the object before it is retrieved by the caller application
AWS Lambda Functions