S3 Flashcards
Simple Storage Service S3 intro
Provides secure, durable, scalable, object storage to store data anywhere on web.
• Object based (0-5TB). Upload files, not OS or anything.
• Largest single upload (PUTS) is 5GB.
• Unlimited Storage/ Stored in bucket
• Global service (no region)
• Universal name space (must be unique name because it creates a web address).
• HTTP200 code if upload successful.
Choose an objects region. Once it is in region it will never leave unless you specifically transfer.
Object
• Object basically a file. ○ Key (name of object) ○ Value (data of the file in bytes) ○ Version ID (can have multiple version of files). ○ Metadata ○ Sub-resources § Access Control List (ACL) Permissions of object. Can lock objects at object level or bucket level. § Torrents
Storage Classes
Standard; IA; One Zone-IA; Intelligent Tiering; Glacier; Glacier Deep
S3 standard
• S3 Standard: 11 9’s of durability, stored redundantly across multiple devices in facilities. Can sustain loss of 2 concurrent facility failures.
○ 0-50TB/GB/month: 0.023
○ 50-500TB/GB/month: 0.022
○ 500+TB/GB/month: 0.021
S3 Infrequently Accessed
• S3-IA (infrequently accessed): Data access less frequently but rapid access. Lower storage fee but charged retrieval fee.
○ All/GB/month: 0.0125
S3 IA One Zone
• S3 One Zone - IA: Same as IA but lower cost and don’t require AZ data resilience.
○ All/GB/Month: 0.001
S3 Intelligent Tiering
Storage class that optimize costs by auto moving data to most cost-effective access tier without performance impact or overheard.
○ For a small monthly fee.
○ Stores in 4 access tiers
§ 2 low latency for frequent/infrequent access
§ 2 opt-in archive tiers for asynchronous/rare access.
○ Process
§ Auto uploaded to S3 Standard (frequent).
§ Not accessed in 30 days moves to IA.
§ If archive activated then 90 days of no access move to Archive
§ 180 days of no access move to deep archive.
§ If you want to receive from archive have to use RestoreObject. If objects are accessed from archive it will move it back to Frequent access.
○ Monitoring all storage/1000 object/month: 0.0025
○ Frequent
§ 0-50TB/GB/month: 0.023
§ 50-500TB/GB/month: 0.022
§ 500+TB/GB/month: 0.021
○ Infrequent
§ All Storage/GB/month: 0.00125
S3 Glacier
• S3 Glacier: For data archiving. Can store any amount for very cheap. Retrieval from minutes to hours.
○ All/GB/month: 0.004
• S3 Glacier Deep Archive: lowest cost storage with 12 hour retrieval time.
○ All/GB/month: 0.00099
S3 Charged
How you’re charged:
• Storage
• Requests (the more requests the more expensive)
• Storage management pricing (different tiers)
• Data transfer
• S3 Transfer Acceleration
• Cross Region Replication Pricing
S3 Transfer Acceleration
Enables fast transfer of files over long distances between your end users and an S3 bucket.
1. Takes advantage of AWS CloudFront's globally distributed edge locations. 2. As the data arrives at an edge location, data is routed to AWS S3 over an optimized network path. 3. Users upload files to edge locations (location where content will be cached). 4. These files are then sent using AWS network to the S3 bucket. 5. This is done to improve speed and performance. 6. Edge location are both read and write compatible. 7. Cached objects are stored for their Time To Live (TTL), after which they are deleted from cached memory. 8. Clearing a cached object before the TTL expires will incur cost.
S3 Cross Region Replication
Simply put, replicating your bucket in another region. It improves availability and is useful for disaster recovery.
1. Versioning must be enabled on both the source bucket (bucket with the object) and the destination bucket. 2. Once turned on, it cannot be turned off only suspended. 3. Cross region replication must be across different regions.
S3 Security
Secure using • ACL/Bucket policies • Object policies (individual files) • IAM/Users/Groups can view. • Has access logs. • Encryption ○ In transit (HTTPS) using SSL or TLS ○ At Rest stored data. § Server Side □ S3 managed keys SSE-S3 □ AWS Key Management System KMS (both customer/amazon) SSE-KMS □ Customer provided keys SSE-C § Client side
S3 Limitations using KMS
• If using SSE-KMS to encrypt you must keep KMS limits in mind.
○ Uploading you call GenerateDataKey in KMS API
○ Downloading you call Decrypt in KMS API
○ Count’s towards KMS quota.
§ 5,500; 10,000;30,000/second depending on region
§ Cannot request quote increase
Versioning
- Stores all versions of object. Good backup tool.
- Versioning cannot be disabled but can be suspended and have to delete old objects manually.
- MFA delete capability.
- Every time you add new object it will list it under the original object name. But if old object public new object won’t be public by default.
○ In Object-Management-lifecycle configuration
○ Create a rule
○ Which tier to go to after which period of time
S3 Object Lock
• S3 Object Lock
○ Write Once, Read Many (WORM). Prevents modification/delete for fixed time
○ Use to meet regulatory requirements
○ Governance mode
§ Can’t overwrite or delete object version after its lock without specifics perms.
§ Can still grant some users to alter retention settings.
○ Compliance mode
§ Can’t be overwritten or delete by any users, not even root for retention period.
§ S3 stores timestamp in object metadata to prevent deletion.
○ Legal hold
§ Prevents object version being overwritten/deleted but without retention period. In effect until this is removed (s3:PutObjectLegalHold)
S3 Glacier Vault Lock
• Glacier Vault Lock
○ Easily deploy and enforce compliance controls for individual S3 glacier vaults with vault lock policy.
○ Can specify controls such as WORM
