Object Storage and CDN Flashcards
What type of storage is S3?
Object based i.e. allows you to upload files. Not suitable to install an OS on.
File size range
0 bytes to 5 TB
How much storage in S3?
Unlimited
Files are stored in
Buckets
When you register a bucket
S3 is a universal namespace, so names must be unique globally
S3 bucket naming in N. Virginia:
http: //bucket.s3.amazonaws.com
http: //s3.amazonaws.com/bucket
S3 bucket naming in other regions:
http: //bucket.s3-aws-region.amazonaws.com
http: //s3-aws-region.amazonaws.com/bucket
Consistency for PUTS of new objects
Read After Write
Consistency for overwrite PUTS and DELETES
Eventual
S3 Storage Classes/Tiers
S3 Standard (immediately available, frequently accessed data)
S3 – Infrequently Accessed (immediately available, infrequently accessed data)
Reduced Redundancy Storage (easily reproducible data)
Glacier (archival, extremely cheap)
Core fundamentals of S3
Key (name) Value (data) Version ID Metadata Subresources (ACLs)
Versioning
Stores all versions of an object (including all writes and even if you delete an object), great backup tool
Versioning costs
Pay for each version, since storage accumulates
Once versioning is enabled
it cannot be disabled, only suspended. Only way to disable versioning is to delete and recreate the bucket
Does versioning integrate with Lifecycle Rules?
Yes
Does versioning have MFA Delete?
Yes, it provides an additional layer of security
Cross Region Replication requirements:
versioning enabled on both source and destination bucket. only new files automatically replicated as well as all versions. permissions also replicated. if you restore object or delete individual versions, not replicated, chaining not allowed
Lifecycle Management
Can be used with versioning, but not required. Can be applied to current and previous versions.
Lifecycle Management Actions
Transition to Standard - IA (128 kb and 30 days after creation date)
Archive to Glacier (30 days after IA, if relevant. 60 days after creation date, or you can move to Glacier day after object uploaded to S3)
Permanently Delete
Edge Location
location where content will be cached. separate from an AZ/Region
Origin
origin of all files that the CDN will distribute. can either be an S3 bucket, EC2 instance, an ELB, or Route53
Distribution
name given to the CDN which consists of a collection of edge locations
Distribution types
Web Distribution (websites), RTMP (media streaming)
Are edge locations read only?
No, you can write to them too
How long are objects cached?
TTL
Can you clear cached objects manually?
Yes, but you will be charged
By default, all newly created buckets are
PRIVATE
You can set up access control to your buckets using
Bucket Policies, ACLs
Access Logging
S3 buckets can be configured to create access logs which log all request made to the S3 bucket. This can be done to another bucket
Encryption types
In Transit, At Rest
In Transit protocol
SSL/TLS
At Rest
Server Side Encryption:
-S3 Managed Keys (SSE-S3)
-AWS Key Management Service, Managed Keys (SSE - KMS)
-Server Side Encryption with Customer Provided Keys (SSE - C)
Client Side Encryption
S3 Managed Keys (SSE-S3)
each object encrypted with a unique key employing strong multifactor encryption. as additional safeguard, encrypts the key itself with a master key that rotates regularly (AES-256). AWS manages this
AWS Key Management Service, Managed Keys (SSE - KMS)
similar to SSE-S3, has benefits + costs. allows use of an envelope key (key that protects encryption key). allows for audit trail
Server Side Encryption with Customer Provided Keys (SSE - C)
You manage keys yourself, S3 manages encryption/decryption
Client Side Encryption
You encrypt data on client side, upload to S3
Storage Gateway Types
File Gateway - for flat files, stored directly on S3
Volume Gateway
-Stored Volumes: Entire dataset stored on site and asynchronously backed up to S3
-Cached Volumes: Entire dataset stored on S3 and most frequently accessed data is cached on site
Gateway Virtual Tape Library (VTL): used for backup and uses popular backup apps like NetBackup, Backup Exec, Veam etc
Stored Volumes good for
connectivity issues, low latency
Snowball types
Snowball (pure storage)
Snowball Edge (storage + compute)
Snowmobile
Snowball Can
Import to S3, Export from S3
You can speed up transfers to S3 using
S3 transfer acceleration. costs extra, and has the greatest impact on people who are far away
S3 static websites
You can use S3 to host static websites, serverless, very cheap + scales automatically, cannot host dynamic sites
successful write to S3
HTTP 200
How to load files to S3 much faster?
enable multipart upload
