S3 Flashcards
S3
Simple storage service
File size range that can be stored in S3
0 to 5Tb
S3 is a universal namespace
s3-region-amazon/buckername
What indicate that upload is successful
http code 200
What does S3 object contain
Key, value, versionID, Metadata, Subresources(ACL, Torrent)
Data consistency for S3
1) Read after write 2) Eventual consistency for overwrite
S3 Guarantees from amazon
Bulit for 99.99 availability, 99.9 guaranteed availability, 11 9’s durability
S3 features
Tiered storage, Lifecycle Mgmt, Versioning, Encryption, MFA for delete, Secure using ACL and bucket policies
S3 standard
99.99 availability, 11 9’s durability. Designed to sustain loss of 2 facilities concurrently
S3- IA
Infrequently accessed
S3 One Zone IA
lower cost option
S3 Glacier
Super cheap. retrieval time can beminutes to hours
S3 Glacier deep archive
12 Hours retrieval time
Charge for S3
Storage, Requests, Storage Mgmt Pricing, Data transcfer Pricing, Transfer acceleration, cross regioon replication
Transfer acceleration
Users upload to nearest edge location instead of S3 bucker.
Intelligent tierring
Can be turned on at object level or bucket level
cloud front is part of
networking and content delivery
cloud front
is a global service not a regiional service
edge location
where content is cached. It is separate to an AWS region /AZ
Origins for cloud front
S3 bucket, EC2 instance, load balancer, Route 53
Distribution
Name for a collection of edge locations
web distributions
typically used for website
RTMP distribution
Used for media streaming
Edge location are not read only
Transfer acceleration
You can invalidate cached objects
But you will be charged
snowball
petabyte scale data transport solution
snowball features
simple, fast secure and as little as 1/5th the cost of high speed internet
snowball flavors
50TB and 80TB
snowball multilayer security
tamper resistent enclosures, 256 bit encryption, industry standard Trusted platform module(TPM) designed to ensure security and full chain of custody
aws snowball edge
100TB, also support compute runs lambda functions. Kind of having a mini aws at remote locations.
AWS storage gateway
connects on-premise sw appliance with cloud based storage to provide seemless and secure storage for you IT infrastructure.
AWS storage gateway
connects on-premise sw appliance with cloud based storage to provide seemless and secure storage for you IT infrastructure.
scalable cost effective storage
storage gateway device
physical or virtual flavors. Can run on Type1 or Type2 hypervisors
3 type of storage gateway
File gateway(NFS&SMB), Volume Gateway iSCSI(storage volumes, cached volumes), Tape gateway
Volume gateways
presents app with iSCSI protocol. Captures changed blocks and stored in the cloud ad amazon EBS snapshots.
stored volumes
Provides application with low latency storage for entire data set.
cached volumes
Only recently read data and written data. low onprem storage requirements
availability of S3-OneZone-IA
99.5
s3 cost
the key driver here is cost, so an awareness of cost is necessary to answer this. Full S3 is quite expensive at around $0.023 per GB for the lowest band. S3 standard IA is $0.0125 per GB, S3 One-Zone-IA is $0.01 per GB, and Legacy S3-RRS is around $0.024 per GB for the lowest band. Of the offered solutions SS3 One-Zone-IA is the cheapest suitable option. Glacier cannot be considered as it is not intended for direct access, however it comes in at around $0.004 per GB.
SSO
Using SAML (Security Assertion Markup Language 2.0), you can give your federated users single sign-on (SSO) access to the AWS Management Console.
origin Access Identity
An Origin Access Identity on the other hand, is a virtual user identity that is used to give the CloudFront distribution permission to fetch a private object from an S3 bucket
3500 puts per second
Until 2018 there was a hard limit on S3 puts of 100 PUTs per second. To achieve this care needed to be taken with the structure of the name Key to ensure parallel processing. As of July 2018 the limit was raised to 3500 and the need for the Key design was basically eliminated.
iam policy documents
You will need to configure Users and Policy Documents only once, as these are applied globally.
How many S3 buckets can I have per account by default
100
Your proposed upload exceeds the maximum allowed object size
Design your application to use the Multipart Upload API for all objects.
Power User Access allows
Access to all AWS services except the management of groups and users within IAM.
Can you specify a regiohn whne you create your Amazon S3 bucket
Yes. Within that region your objects are redundantly stored on multiple devices across multiple AZ’s
Amazon S3 does not provide object locking
if you need this you need to build it into your app or use versioninig
success
- use the Content-MD5 header. When you use this header, Amazon s3 checks the object against hte provided MD5 value and if they do not match returns an error
headers
starts with x-amz
multi-object delete
Amazon S3’s new Multi-Object Delete gives you the ability to delete up to 1000 objects from an S3 bucket with a single request.
if s3 object file is csv of json you can write sql queries to get elements of the file by using
SQL queries
with s3 ACL we can setup cross account access
useful when orgs has multiple
3 types of s3 permissions
IAM, ACl, bucket control list
difference between a bucket policy and and IAM policy
who is the actual resource? IAM the policy applies to user, group, role
with bucket policy the policy applices to a bucket
Encryptioni is on by default in Glacier and it cannot be turned off.
You don’t need to encrypt the data before sending
Any put above 100Mb is good candidates for multipart
You can send just the parts that failed.
If you have to store anything more than 5Gb you have to to use Multipart upload.
WIth multipart upload you can store upto 5TB
Transfer acceleration
Enable transfer acceleration for your bucket and use
s3-accelerate instead of s3 in
bucket.s3.amazonaws.com
storage gateway( 3 types volume[gateway cached and gateway stored], file and tape)
You download a VM image and install in your data center
volume gateway types
gateway cached and gateway stored
gateway stored
All data is stored locally in storage volumes. Gateway will periodically take snapshots of the data as incremental backups and stores them on Amazon s3
Note individual files can be accessed only after the snapshot is mounted as a EBS volume.
File gateways
unlike storage gateway files are stored in s3 and individual files can be accessed.
Which messaging service uses standard APIs and protocols such as JMS, NMS, AMQP, STOMP, MQTT, and WebSocket
Amazon MQ
How big can a SQS text message be in size
The maximum is 256 KB. (As a side note, the minimum SQS message size is 1 byte).
How many subscribers can receive a message from SNS
10 million subscribers per topic is a soft limit
Besides CloudFront, what are two ways API Gateway can block DDOS attacks from reaching your backend
Request Throttling, Caching API Responses
