S3 Flashcards
What type of storage is S3?
Object storage
What are S3 use cases?
Backup and storage
Disaster Recovery
Archive purposes
For hybrid cloud storage
Application hosting
Media Files
Data lakes y big data analytics
Software updates delivery
Hosting static websites
Where does S3 store objects in?
Buckets
What characteristics must a bucket name have?
It must be globally unique
Where are buckets located in aws?
In a specific region
Is S3 a Global Service?
No. Buckets are located in a region.
What is the naming convention for S3 buckets?
No uppercase.
No undesrcore.
Between 3 and 63 characters long.
Not an IP. Must start with lowercase letter or number.
Must not start with xn
Must not end with s3alias
Example name: bucket-leito123
What is the S3 object key?
The full path of the file (object)
Example: s3://my-bucket/my_file.txt
Example of key with directories (not really directories):
s3://my-bucket/my_folder1/another_folder/my_file.txt
Remember, the s3 UI makes it look like there are directories inside a bucket, but in reality there are just different keys, and some are longer and have / slashes, like the previous example: “/my_folder1/another_folder/”
The /my_folder1/another_folder/ path is just the prefix for the object name.
Key = prefix + object name
Can you have folders in s3?
Yes. Folders can be created inside a bucket to organize a buckets’ objects. The folder will be part of the object key.
What is the key prefix?
Part of the key. The prefix is the part of the patch that consists of the folders the object is in.
Key = Prefix + Object
Are there directories in buckets?
No. In the UI it looks like there are directories. But in reality there are just different keys to the objects inside buckets.
What is the max object size?
5TB
What happens when you upload a file larger than 5GB?
You must use the multi part upload
Why can you open objects with the “open” button in the aws management console, and not the public URL?
Because when you open an object with this button, aws verifies your user is the one opening it and it signs the request with your user, which owns the object.
What is an S3 object URL?
Every S3 object has a public access URL. To which you need to enable access. Objects are not public by default.
How does security work in S3?
Security is centered around access to S3 objects.
It can be user based or resource based.
User based security is for bucket objects, and it uses IAM Policies to set which api calls should be allowed for a specific user from IAM
Resource based security are bucket wide rules that you can assign them from the s3 console. These rules can allow a specific user to have access, and it is even cross account. So it can allow users from other AWS Accounts to have access to the bucket
What is resource based security in S3?
Resource based security contains the following:
Bucket Policies: These are bucket wide rules that you can assign from the s3 console. These rules can allow a specific user to have access, and it can even be cross account. So it can allow users from other AWS Accounts to have access to the bucket.
Object ACLs: A finer grained security for bucket objects. These can be disabled.
Bucket ACLs: Less common. And can be disabled.
What is user based security in S3?
User based security is for s3 in general. It uses IAM Policies to set which api calls should be allowed for a specific user from IAM.
What are Object ACL?
A finer grained security for bucket objects.
What is a principal?
That to which a permission or role applies to. A user or a resource
A principal can access an s3 object if?
If the user IAM permissions allow it, OR, if the resource policy allows it, AND, there is no explicit deny.
Can you use encryption in s3?
Yes. You can encrypt objects using encryptions keys.
What are S3 bucket policies made of?
JSON document. Very similar to IAM Policies.
How can you force objects to be encrypted at upload in s3?
With a bucket policy.
What can you use a bucket policy for?
Force objects to be encrypted.
Grand a bucket public access.
Grant access to another account to a bucket (Cross Account).
How do you grant an EC2 instance access to an S3 Bucket?
With an IAM Role.
How do you grant a user in another account access to my bucket?
With a bucket policy
What are default scurity settings for a bucket?
Block all public access is on. You need to disable it to allow public access to your bucket. This will block any access granted by bucket policies or ACLs.
How does an s3 bucket policy look like?
Its like an IAM policy, with a statement. Here the principal is a , which means anyone. The action is s3:GetObject which is view. And the resource is the bucket with a /, which means any object inside the bucket. The resource could be arn:aws:s3:::leito-bucket/coffee which will only make public the coffee picture within the bucket.
{
“Version”: “2012-10-17”,
“Id”: “Policy1727063555175”,
“Statement”: [
{
“Sid”: “Stmt1727063553042”,
“Effect”: “Allow”,
“Principal”: “”,
“Action”: “s3:GetObject”,
“Resource”: “arn:aws:s3:::leito-bucket/”
}
]
}
What is s3 hosting?
You can host static websites in s3.
S3 websites look like this:
https://bucket-name.s3-website.aws-region.amazonaws.com
What is an s3 website composed of?
A bucket with files, html files, pictures, etc.
How do you enable static website hosting in s3?
Enter your bucket and in properties you can enable it.
It lets you add a home page html, which will be in your bucket, and an error html too. These htmls can point to pictures or other objects in the bucket.
You will have to enable public access to all the objects in the bucket that the website uses
What is S3 versioning?
You can have multiple versions of objects in a versioning enabled bucket.
What happens when you overwrite a key in a versioning enabled bucket?
It automatically makes another version
What happens when you delete a key in a versioning enabled bucket?
It doesn’t delete. It adds a “delete marker” and lets you restore the deleted version.
What is a version rollback?
You have an object in version 3, you can go back to version 2 easily.
Do you lose versions with “suspend versioning”?
No. You just stop versioning files, you dont lose previous versions.
What version do objects that are previous to enabling versioning, and that have not been versioned have?
Null.
What happens when you upload a file with the same name to an s3 bucket?
If you don’t have versioning, the object is overritten. If you have versioning, a new version is created.
What is version id?
A unique identifier for a version of an object
How do you roll back an object change in s3?
You delete the latest version of the object.
What happens when you delete an object in an s3 bucket with versioning enabled?
When you delete an object, it adds a delete marker on it, and hides it from the object list. It doesn’t actually delete it.
If you want to permanently delete an object, you have to delete all versions of it, with the versions toggle enabled.
You can permanently delete only certain versions of an object.
What is a delete marker?
It’s like a version of an object S3 adds when you delete an object with versioning enabled. It hides the object from the list and “protects” and makes unavailable its versions.
How do you undo deleting an object in an s3 bucket with versioning enabled?
By deleting the delete marker.
What are the 2 possibilities with s3 replication
Cross region replication and Same region replication
What are some s3 cross region replication use cases?
To grant lower latency access to data.
For replication across aws accounts
For company compliance
What are the characteristics of s3 replication?
Asyncronous
From an origin bucket to a destination bucket, in same or different region, and in same or different aws account.
Must enable versioning in origin and destination buckets
CRR or SRR option
What are some s3 same region replication use cases?
For log aggregation (Process of consolidation log data from various sources of an environment).
Live replication from production to test accounts. (For test environments).
What are s3 replication caveats?
When you enable replication, only new objects are going to be replicated.
How do you replicate pre-existing objects once you enable replication for an s3 bucket?
By using S3 batch replication.
Can you chain s3 replication? (Meaning replicate from bucket1 to bucket2 and from bucket2 to bucket3)
No. S3 does not allow replication of a replicated object.
Does S3 replication replicate delete markers?
By default it doesn’t, but you can enable an option in the replication rule to replicate delete markers too.
What is asyncronous replication?
Its continuous replication but with a delay related to source of replica.
Are deletes replicated in s3 replication?
No. Only delete markers optionally, but never permanently deletes.
What are all the S3 storage classes?
General Purpose
Infrequent Access
One Zone Infrequent Access
Glacier instant retrieval
Glacier flexible retrieval
Glacier Deep Archive
Intelligent Tiering
What are S3 lifecycle configurations?
A way to move your s3 through storage classes
What is S3 durability?
It’s Eleven 9s for all s3 storage classes
What is S3 availability?
Varies depending of storage class. S3 GP has 99,99% availability. (53 minutes a year of downtime).
What s3 storage class is used for frequently accessed data? What are it’s characteristics?
General Purpose.
It has low latency and high throughput.
No retrieval time
What are S3 standard general purpose use cases?
Big Data analytics, gaming apps, mobile apps, content distribution.
What s3 storage class is used for unfrequently accessed data? What are it’s characteristics?
Infrequent access (standard or one zone)
It’s for data that is infrequently accessed, but requires rapid access when needed. (No retrieval time, same as general purpose).
It’s cheaper than general purpose, but as a counterpart has a cost for retrieval of files. That’s why it’s best for infrequent access, so you can save costs.
What are S3 standard infrequent access use cases?
Disaster recovery, backups.
What are S3 one zone infrequent access use cases?
Secondary backups from on premises, for data you can recreate in case of loss.
What are the differences between standard and one zone infrequent access?
It’s similar but one zone is cheaper.
Has lower availability per year.
It’s single AZ so in case of an AZ disaster, you will lose all data.
standard ia is for disaster recovery and backups, one zone is for secondary backups, or data you can recreate if the AZ fails
How do s3 Glacier options work?
It’s for archival purposes. You have a minimum storage duration before you can delete what you upload. You pay for storage and for object retrieval.
Instant retrieval you can access the data in milliseconds, and you have a minimum storage duration of 90 days.
Flexible Retrieval you have different options:
1 to 5 minutes, 3 to 5 hours, or 5 to 12 hours retrieval time. 5 to 12 hours option is free of retrieval cost. You also have a minimum storage duration of 90 days.
Glacier Deep Archive you commit to a minimum storage duration of 180 days. And you have to wait 12 or 48 hours to retrieve data.
What s3 storage class is best used for long term storage? What are it’s characteristics?
Glacier Deep Archive. You commit to a minimum storage duration of 180 days. And you have to wait 12 or 48 hours to retrieve data.
What’s the difference between Glacier instant retrieval and flexible retrieval?
Both have a minimum storage duration of 90 days, but instant retrieval is instant as its name suggests, and flexible retrieval has options of retrieval time, the fastest being between 1 and 5 minutes.
Flexible retrieval is cheaper.
Which s3 storage class allows you to move objects between tiers? How does it work?
Intelligent tiering.
It costs a monthly fee to use. But there are no retrieval charges when using intelligent tiering.
It moves objects automatically between “access tiers” based on usage.
Access tiers are special for intelligent tiering, similar to storage classes, going from frequent access tier which is the default to deep archive access tier.
Intelligent tier moves unaccessed objects to lower tiers. An object unaccessed for 30 days for example, will be moved from frequent to infrequent access tier.
What is minimum storage duration?
The minimum amount of time you have to commit your storage to stay in glacier storage classess.
What are access tiers in s3?
The different tiers of storage in intelligent tiering storage class.
In which storage classes you dont pay for retrieval?
In General Purpose and Intelligent Tiering.
In which storage classes you dont have a minimum storage duration?
In General Purpose and Intelligent Tiering.
How does retrieval fee work?
You are charged for downloading a file. You are charged per GB. Larger files are more expensive.
How do you manage an object with lifecycle configuration?
You create a lifecycle rule for a bucket, that will transition objects to a different storage class after x days of being created.
What can you use lifecycle rules for?
To move objects to different storage classes after a certain time.
To delete objects after some time
To delete old versions of files, move old versions of files.
To delete incomplete multi part downloads
What can you apply lifecycle rules to?
A complete bucket or a prefix of a bucket.
Also to object tags
What does S3 analytics do?
It will give you recommendations (only for standard and standard IA) to help you decide when to transition objects between storage classes.
It creates a daily csv report.
What are all the possible lifecycle rule actions?
Move current versions of objects
Move noncurrent versions of objects
Expire current versions (adds delete marker and makes the expired version noncurrent)
Permanently delete noncurrent versions
What do you pay for in s3?
Storage in buckets
Data transfered associated with buckets.
What are requester pays?
A type of bucket in which the requester of an object pays for the download instead of the owner of the bucket.
What are event notifications?
An s3 feature that allows you to receive notifications when certain things happen in your s3 bucket. For example an object is created, deleted, replicated, etc.
What is a common use case for s3 event notifications?
You can use for example notifications for .jpg file creations in a bucket to set up automated creation of thimbnails of those jpg files. This works with SNS, SQS or Lambda functions.
How can S3 work with Eventbridge?
All s3 event notifications are also visible to the Amazon Eventbridge service. You can then set up rules in eventbridge to send these events to over 18 different services.
What is the prefix of an object in s3?
The part of the Key that encompases everything between after the bucket and before the file.
Example: s3://bucket/folder1/sub1/file.jpg
In this case the prefix is: /folder1/sub1/
How does multi part upload work in s3?
It uploads files divided in multiple parts which increases bandwidth.
S3 then unites them in a single file in bucket after upload finishes.
What is s3 transfer acceleration?
When you have a file ie in usa, and you want to upload it to a bucket in ie australia, transfer acceleration will first put it in an edge location in usa and then send it to your bucket in australia through the private aws network.
What feature can help you you speed up file downloads in s3?
S3 byte-range fetches. Allow you to make request for specific byte ranges of a file, and by making requests in parallel you can speed up GETs.
What feature can help you you retrieve only partial data of a file in s3 bucket? What could it be useful for?
S3 byte-range fetches. Allow you to make request for specific byte ranges of a file. For example you can request only the header of a file to get only certain information of a file instead of having to download it.
What are batch operations use cases in s3?
For bulk object operations, like:
Encrypt all unencrypted objects in a bucket
modify object metadata or properties
copy objects between s3 buckets
Modify tags
Modify ACLs
What is S3 storage lens?
A metrics feature thay analyzes s3 in your account to find anomalies, possible cost efficiency improvements, apply data protection best practices.
What are S3 storage lens for summary metrics use cases?
Identify growing buckets or not used buckets
What are S3 storage lens for cost optimization metrics use cases?
Identify buckets with old incomplete multipart uploads
Identify objects that could be transitioned to lower cost storage classes.
What are S3 storage lens for access management metrics use cases?
To identify what are your bucket object ownership settings
What are S3 storage lens for data protection metrics use cases?
Identify buckets that are not following your data protection best practices (Not encrypted, not replicated, no versioning, etc)
You are looking to get recommendations for S3 Lifecycle Rules. How can you analyze the optimal number of days to move objects between different storage tiers?
s3 analytics
It will give you recommendations (only for standard and standard IA) to help you decide when to transition objects between storage classes.
It creates a daily csv report.
What are the different types of encryption supported in S3?
You have server side encryption and client side encryption.
Client side encryption is when you encrypt everything before uploading it to S3.
SSE-S3: Server side encryption with s3-managed keys
SSE-KMS: Server side encryption with AWS KMS
SSE-C: Server side encryption with customer provided keys
Which S3 encryption method is enabled by default?
SSE-S3: Server side encryption Wwith s3-managed keys
When do you use SSE-S3 over SSE-KMS?
When you don’t want to manage the keys yourself, and you want to let S3 manage everything.
When do you use SSE-KMS over SSE-S3?
You have control of the keys with the KMS service, instead of letting S3 manage the keys on the background.
You choose which keys to use for which objects.
You can create your own keys and audit key usage.
What is a disadvantage of using SSE-KMS in S3?
You can experience throtling of your bucket transfer speed (slowness) in some high throughput cases because of the high amount of api calls being made between the S3 bucket and the KMS Server every time a file is uploaded or downloaded
When you upload, S3 calls the GenerateDataKey API.
And when you download, it calls the Decrypt API.
How does encryption in flight work in S3?
You have encryption in flight by uploading and downloading data from S3 buckets using HTTPS.
How do you enforce encryption in flight in S3?
Attach a bucket policy to your S3 bucket, with a statement that denies any getobject operations if transport encryption is disabled.
Effect: Deny
Principal: *
Action: s3:getobject
Resource: bucket
condition: bool “aws:SecureTransport”: “false”
How do you enforce a specific kind of encryption in an S3 bucket?
By creating a bucket policy that only allows API calls with the specific kind of encryption you want to use.
This would be a deny effect with a condition that says you dont have a header of aws:kms for example.
What is CORS?
It’s cross origin resource sharing.
When you make a request to a webserver, and that webserver needs to take resources from a different webserver, for example example.com and other.com. It gives in the HTML the direction to the browser to get the resources, like images, from the second webserver. This direction is marked with the origin example.com, so the cross origin other.com will allow your request and give you the images you requested through the HTML you got from example.com
The other.com webserver needs to be configured to use CORS and allow the origin example.com for the request to be successful.
This data in HTML for CORS comes in the “CORS Headers”.
What is CORS in S3?
It’s a web browser security function that allows you to retrieve images from a bucket, when the request is being made from another s3 bucket that is allowd as an origin in the CORS headers of the first s3 bucket.
If have bucket A as a web server, and in your html index you want to display content from bucket B. What do you need to do?
You need to point to the bucket B resources in your html file for bucket A.
And you need to allow bucket A URL as a cors origin in bucket B, so that the fetch request works and you can display bucket b content in bucket a html index page.
Who can enable MFA Delete in an S3 bucket?
Only the root account. And only through CLI, for which you need MFA enabled.
How do you delete objects when mfa delete is enabled?
Only with the CLI or SDK. And using MFA. You cant through management console.
What do S3 access logs do?
It’s a full log of an s3 bucket, that is stored in another s3 bucket.
The logging bucket will be in the same region.
What is a precaution to take when enabling logging in s3 buckets?
Never set the log bucket to be the same bucket as the logged bucket. This will create a logging loop and your bucket will grow exponentially.
What is a pre signed URL?
An URL you generate for access to a file in an s3 bucket, this URL grants permissions temporarily to the file in the bucket even if you dont have access to it.
This includes downloading or uploading a file. Depending the use case.
Which permissions does a pre signed URL grant to whomever uses it?
The same permissions of the user that generated the pre signed URL
How do you create a pre signed URL?
You select a bucket object and you create a pre signed url to share it. You state the amount of minutes or hours it will be available.
What are pre signed url use cases?
Allowed only logged in users to download a premium video from your s3 bucket
Allow an ever changing list of users to download files by generating urls dinamically
Allow temporarily a user to upload a file to a precise location in your s3 bucket.
What is glacier vault lock?
It’s WORM model: Write once read many.
You create a vault lock policy, and then lock the policy, so it can no longer be edited. You do this per bucket
Once an object is inserted into your glacier vault, the object can never be deleted.
Helpful for compliance and data retention
What is object lock?
You block an object version deletion for a specific amount of time.
You set a retention period
What are the 2 kinds of object lock and their differences?
Compliance mode and governance mode.
In compliance mode there is no way to delete object locked objects. Even the root account can’t delete them. Neither change object lock settings.
In governance mode special users with the s3 governance permission can delete the object.
In short, compliance is very strict and governance is a bit more lenient.
What is a legal hold in s3?
It’s an object lock option that protects an object from deletion indefinitely.
It’s for legal procedures and avoiding accidental deletion of objects related to this, You enable and disable it if you have the legal hold permission.
What are access points in s3?
With access points you define access point policies that grant read or write access to users to one or many determined prefixes (the prefix is the route of the files)
Users will be granted access through IAM to the needed access points instead of each specific prefix, thus simplifying management of permissions in s3 buckets.
Access points have a dns name to which you define internet or private vpc access
What are s3 access points good for?
Simplifying access management to your s3 buckets and objects.
What is object lambda?
Through the use of an s3 access point and a lambda function, you can change the contents of an s3 bucket object, without actually changing the original object, and instead making an object lambda, which is a “redacted” or “enriched” version of the original.
This is useful in cases where you have an analytics or marketing app and you want go give it modified versions of the objects. In the case of redacted, objects modified to have less data. And in the case of enriched, objects with added data in comparison to the original.
TLDR: The lambda function through access points redacts or enriches objects as the objects are retrieved by an app.
The lambda function has access to the access point prefix of the original objects.
The app only has access to the “s3 object lambda access point”, which contains the redacted object.
In cases where you enrich objects with a lambda function, you would use a database to get the enriching data.
What are object lambda use cases?
To redact personal information data from objects.
To convert data, ie: from xml to json etc.
Resizing and watermarking images (Enriching), using caller specific details, such as the user who requested the object.
