Storage - S3, EBS, EFS, Cloudfront, Storage GW, Snowball, ++ Flashcards
All things storage.
-Use if you need more than 10,000 IOPS -Can provision up to 20,000 IOPS per volume
- General purpose SSD
- Provisioned IOPS
- Throughput optimized HDD (ST1)
- Cold HDD (SC1)
- Magnetic Standard - Legacy
- Provisioned IOPS
Designed for IO intensive apps such as large relational or NoSQL databases a) General purpose SSD b) Provisioned IOPS c) Throughput optimized HDD (ST1) d) Cold HDD (SC1) e)Magnetic Standard - Legacy
b) Provisioned IOPS
T or F If a spot instance is terminated by EC2, you will not be charged for a partial hour of usage. However, if you terminate the instance yourself, you will be charged for the complete hour in which the instance ran.
True
_____ allows you to create storage volumes and attach them to EC2 instances
EBS - elastic block storage
once attached, you can create a filesystem on top of these volumes, run a database, or use them in any other way you would use a block device.
EBS volumes
____ volumes are placed in a specific availability zone, where they are auto replicated to protect you from the failure of a single component.
EBS volumes
EBS volumes types
-General purpose SSD -Provisioned IOPS -Throughput optimized HDD (ST1) -Cold HDD (SC1) -Magnetic Standard - Legacy
ratio of 3 IOPS per GB with up to 10,000 IOPS and the ability to burst up to 3,000 IOPS for extended period of time for volumes at 3334 GB and above. a) General purpose SSD b) Provisioned IOPS c) Throughput optimized HDD (ST1) d) Cold HDD (SC1) e)Magnetic Standard - Legacy
a) General purpose SSD
Lowest cost per GB of all EBS volume types that is bootable. Ideal for workloads where data is accessed infrequently and applications where the lowest storage cost is important. a) General purpose SSD b) Provisioned IOPS c) Throughput optimized HDD (ST1) d) Cold HDD (SC1) e)Magnetic Standard - Legacy
e) Magnetic Storage
-Big data -data warehouses -log processing -can’t be boot volume a) General purpose SSD b) Provisioned IOPS c) Throughput optimized HDD (ST1) d) Cold HDD (SC1) e)Magnetic Standard - Legacy
c) Throughput optimized HDD (ST1)
- Lowest cost storage for infrequently accessed workloads
- file server
- can’t be boot volume
a) General purpose SSD
b) Provisioned IOPS
c) Throughput optimized HDD (ST1)
d) Cold HDD (SC1)
e) Magnetic Standard - Legacy
d) Cold HDD (SC1)
Types of compliance in AWS
- Service Organization Controls (SOC) 1/International Standard on Assurance Engagements (ISAE) 3402, SOC 2, and SOC 3
- Federal Information Security Management Act (FISMA), Department of Defense Information Assurance Certification and Accreditation Process (DIACAP), and Federal Risk and Authorization Management Program (FedRAMP)
- Payment Card Industry Data Security Standard (PCI DSS) Level 1
- International Organization for Standardization (ISO) 9001, ISO 27001, and ISO 27018
What languages does Elastic Beanstalk support?
PHP, Java, Python, Ruby, Node.js, .NET, and Go.
Name some EBS facts
- persistent block-level storage volumes
- each volume is automatically replicated within its Availability Zone
- low-latency performance
How does storage gateway work?
It provides low-latency performance by maintaining a cache of frequently accessed data on-premises while securely storing all of your data encrypted in Amazon S3 or Amazon Glacier.
Why use Dynamo DB?
fast and flexible NoSQL database service for all applications that need consistent, single-digit millisecond latency at any scale. great fit for mobile, web, gaming, ad-tech, Internet of Things, and many other applications.
What is CloudTrail
web service that records AWS API calls for an account and delivers log files for audit and review.
Common use cases for S3
Backup and archive for on-premises or cloud data Content, media, and software storage and distribution Big data analytics Static website hosting Cloud-native mobile and Internet application hosting Disaster recovery
S3 storage classes
general purpose, infrequent access, and archive.
How does block storage operate?
Block storage operates at a lower level—the raw storage device level—and manages data as a set of numbered, fixed-size blocks.
How does file storage operate?
File storage operates at a higher level—the operating system level—and manages data as a named hierarchy of files and folders.
What protocols do block storage use? SAN - Storage Area Network
iSCSI or Fiber Channel
What protocols does file storage use? NAS - Network Attached Storage
Common Internet File System (CIFS) Network File System (NFS)
What protocol does S3 use?
Application Program Interface (API) built on standard HTTP verbs
An S3 ______ contains both data and metadata
object
Objects reside in containers called ______
buckets
How are S3 objects identified?
unique user-specified keys (filename)
Amazon S3 objects are automatically replicated on multiple devices in multiple facilities within a region. T or F?
True
Amazon S3 automatically partitions buckets to support very high request rates and simultaneous access by many clients. T or F?
True
Which storage option provides network-attached shared file storage (NAS storage) using the NFS v4 protocol.
Amazon Elastic File System (AWS EFS)
Which storage option provides block level storage for Amazon Elastic Compute Cloud (Amazon EC2) instances.
EBS
Bucket names can contain:
63 lowercase letters, numbers, hyphens, and periods.
How many buckets can you have per account by default?
100
Best practice
It is a best practice to use bucket names that contain your domain name and conform to the rules for DNS names. This ensures that your bucket names are your own, can be used in all regions, and can host static websites.
What sizes can S3 objects be?
0 bytes to 5TB
How many objects can a single bucket store?
Unlimited
What is included in system metadata?
the date last modified, object size, MD5 digest, and HTTP Content-Type.
When can you create user metadata on an object?
Only at the time the object is created.
A S3 key consists of what?
up to 1024 bytes of Unicode UTF-8 characters, including embedded slashes, backslashes, dots, and dashes.
What is the URL format of S3?
http://mybucket.s3.amazonaws.com/jack.doc http://mybucket.s3.amazonaws.com/fee/fi/fo/fum/jack.doc
Is there a file or folder hierarchy in S3?
There is no actual file and folder hierarchy. A key may contain delimiter characters like slashes or backslashes to help you name and logically organize your Amazon S3 objects, but to Amazon S3 it is simply a long key name in a flat namespace. For convenience, the Amazon S3 console and the Prefix and Delimiter feature allow you to navigate within an Amazon S3 bucket as if there were a folder hierarchy. However, remember that a bucket is a single flat namespace of keys with no structure.
The S3 API includes:
Create/delete a bucket Write an object Read an object Delete an object List keys in a bucket
What type of API does S3 use?
REST (Representational State Transfer) API. uses standard HTTP or HTTPS requests to create and delete buckets, list keys, and read and write objects.
How does REST work in S3?
REST maps standard HTTP “verbs” (HTTP methods) to the familiar CRUD (Create, Read, Update, Delete) operations. Create is HTTP PUT (and sometimes POST); read is HTTP GET; delete is HTTP DELETE; and update is HTTP POST (or sometimes PUT).
Best practice
Always use HTTPS for Amazon S3 API requests to ensure that your requests and data are secure.
What are some of the high level interfaces people use to interact with S3 instead of the REST interface itself?
These include the AWS Software Development Kits (SDKs) (wrapper libraries) for iOS, Android, JavaScript, Java, .NET, Node.js, PHP, Python, Ruby, Go, and C++, the AWS Command Line Interface (CLI), and the AWS Management Console.
What does durability mean according to AWS?
Durability addresses the question, “Will my data still be there in the future?”
What does availability mean according to AWS?
Availability addresses the question, “Can I access my data right now?”
how many 9s are Amazon’s S3 storage DURABILITY of objects over a given year designed for?
99.9999999999% - 11 total 9s Amazon S3 achieves high durability by automatically storing data redundantly on multiple devices in multiple facilities within a region. It is designed to sustain the concurrent loss of data in two facilities without loss of user data. Amazon S3 provides a highly durable storage infrastructure designed for mission-critical and primary data storage.
how many 9s are Amazon’s S3 storage AVAILABILITY of objects over a given year designed for?
99.99% - 4 total 9s
If high durability is not required, what is the best storage to use?
RRS - Reduced Redundancy Storage
What durability does RRS offer?
99.99% with a lower cost of storage
Best Practice
Even though Amazon S3 storage offers very high durability at the infrastructure level, it is still a best practice to protect against user-level accidental deletion or overwriting of data by using additional features such as versioning, cross-region replication, and MFA Delete.
Why is S3 considered an eventually consistent system?
your data is automatically replicated across multiple servers and locations within a region, changes in your data may take some time to propagate to all locations. As a result, there are some situations where information that you read immediately after an update may return stale data.
What is meant by an eventually consistent system?
Eventual consistency means that if you PUT new data to an existing key, a subsequent GET might return the old data. Similarly, if you DELETE an object, a subsequent GET for that object might still read the deleted object. In all cases, updates to a single key are atomic—for eventually-consistent reads, you will get the new data or the old data, but never an inconsistent mix of data.
For PUTs to new objects….
Amazon S3 provides read-after-write consistency.
for PUTs to existing objects (object overwrite to an existing key) and for object DELETEs…
Amazon S3 provides eventual consistency.
Types of controls put on S3
coarse-grained access controls (Amazon S3 Access Control Lists [ACLs]), and fine-grained access controls (Amazon S3 bucket policies, AWS Identity and Access Management [IAM] policies, and query-string authentication).
S3 ACLs allow you to grant:
READ, WRITE, or FULL-CONTROL at the object or bucket level. ACLs are a legacy access control mechanism, created before IAM existed. ACLs are best used today for a limited set of use cases, such as enabling bucket logging or making a bucket that hosts a static website be world-readable.
Differences between IAM policies and S3 policies:
S3: They are associated with the bucket resource instead of an IAM principal. They include an explicit reference to the IAM principal in the policy. This principal can be associated with a different AWS account, so Amazon S3 bucket policies allow you to assign cross-account access to Amazon S3 resources.
What does a policy in effect do in s3?
you can specify who can access the bucket, from where (by Classless Inter-Domain Routing [CIDR] block or IP address), and during what time of day.
Can IAM policies be associated directly with IAM principals?
yes
What does a prefix and delimiter parameters do for S3?
lets you organize, browse, and retrieve the objects within a bucket hierarchically. Typically, you would use a slash (/) or backslash () as a delimiter and then use key names with embedded delimiters to emulate a file and folder hierarchy within the flat object key namespace of a bucket.
What are the S3 storage classes?
Standard Intelligent-Tiering (S3 Intelligent-Tiering) Standard – Infrequent Access (Standard-IA) One Zone-Infrequent Access (S3 One Zone-IA) Reduced Redundancy Storage (RRS) Amazon Glacier Glacier Deep Archive (S3 Glacier Deep Archive)
Amazon S3 Standard (S3 Standard)
S3 Standard offers high durability, availability, and performance object storage for frequently accessed data. Because it delivers low latency and high throughput, S3 Standard is appropriate for a wide variety of use cases, including cloud applications, dynamic websites, content distribution, mobile and gaming applications, and big data analytics. S3 Storage Classes can be configured at the object level and a single bucket can contain objects stored across S3 Standard, S3 Intelligent-Tiering, S3 Standard-IA, and S3 One Zone-IA. You can also use S3 Lifecycle policies to automatically transition objects between storage classes without any application changes.
Amazon S3 Standard (S3 Standard) Key features
Low latency and high throughput performance Designed for durability of 99.999999999% of objects across multiple Availability Zones Resilient against events that impact an entire Availability Zone Designed for 99.99% availability over a given year Backed with the Amazon S3 Service Level Agreement for availability Supports SSL for data in transit and encryption of data at rest S3 Lifecycle management for automatic migration of objects to other S3 Storage Classes
Amazon S3 Intelligent-Tiering (S3 Intelligent-Tiering)
The S3 Intelligent-Tiering storage class is designed to optimize costs by automatically moving data to the most cost-effective access tier, without performance impact or operational overhead. It works by storing objects in two access tiers: one tier that is optimized for frequent access and another lower-cost tier that is optimized for infrequent access. For a small monthly monitoring and automation fee per object, Amazon S3 monitors access patterns of the objects in S3 Intelligent-Tiering, and moves the ones that have not been accessed for 30 consecutive days to the infrequent access tier. If an object in the infrequent access tier is accessed, it is automatically moved back to the frequent access tier. There are no retrieval fees when using the S3 Intelligent-Tiering storage class, and no additional tiering fees when objects are moved between access tiers. It is the ideal storage class for long-lived data with access patterns that are unknown or unpredictable. S3 Storage Classes can be configured at the object level and a single bucket can contain objects stored in S3 Standard, S3 Intelligent-Tiering, S3 Standard-IA, and S3 One Zone-IA. You can upload objects directly to S3 Intelligent-Tiering, or use S3 Lifecycle policies to transfer objects from S3 Standard and S3 Standard-IA to S3 Intelligent-Tiering. You can also archive objects from S3 Intelligent-Tiering to S3 Glacier.
Amazon S3 Intelligent-Tiering (S3 Intelligent-Tiering) Key features:
Same low latency and high throughput performance of S3 Standard Small monthly monitoring and auto-tiering fee Automatically moves objects between two access tiers based on changing access patterns Designed for durability of 99.999999999% of objects across multiple Availability Zones Resilient against events that impact an entire Availability Zone Designed for 99.9% availability over a given year Backed with the Amazon S3 Service Level Agreement for availability Supports SSL for data in transit and encryption of data at rest S3 Lifecycle management for automatic migration of objects to other S3 Storage Classes
Amazon S3 Standard-Infrequent Access (S3 Standard-IA)
S3 Standard-IA is for data that is accessed less frequently, but requires rapid access when needed. S3 Standard-IA offers the high durability, high throughput, and low latency of S3 Standard, with a low per GB storage price and per GB retrieval fee. This combination of low cost and high performance make S3 Standard-IA ideal for long-term storage, backups, and as a data store for disaster recovery files. S3 Storage Classes can be configured at the object level and a single bucket can contain objects stored across S3 Standard, S3 Intelligent-Tiering, S3 Standard-IA, and S3 One Zone-IA. You can also use S3 Lifecycle policies to automatically transition objects between storage classes without any application changes.
Amazon S3 Standard-Infrequent Access (S3 Standard-IA) Key features:
Same low latency and high throughput performance of S3 Standard Designed for durability of 99.999999999% of objects across multiple Availability Zones Resilient against events that impact an entire Availability Zone Data is resilient in the event of one entire Availability Zone destruction Designed for 99.9% availability over a given year Backed with the Amazon S3 Service Level Agreement for availability Supports SSL for data in transit and encryption of data at rest S3 Lifecycle management for automatic migration of objects to other S3 Storage Classes
Amazon S3 One Zone-Infrequent Access (S3 One Zone-IA)
S3 One Zone-IA is for data that is accessed less frequently, but requires rapid access when needed. Unlike other S3 Storage Classes which store data in a minimum of three Availability Zones (AZs), S3 One Zone-IA stores data in a single AZ and costs 20% less than S3 Standard-IA. S3 One Zone-IA is ideal for customers who want a lower-cost option for infrequently accessed data but do not require the availability and resilience of S3 Standard or S3 Standard-IA. It’s a good choice for storing secondary backup copies of on-premises data or easily re-creatable data. You can also use it as cost-effective storage for data that is replicated from another AWS Region using S3 Cross-Region Replication. S3 One Zone-IA offers the same high durability†, high throughput, and low latency of S3 Standard, with a low per GB storage price and per GB retrieval fee. S3 Storage Classes can be configured at the object level, and a single bucket can contain objects stored across S3 Standard, S3 Intelligent-Tiering, S3 Standard-IA, and S3 One Zone-IA. You can also use S3 Lifecycle policies to automatically transition objects between storage classes without any application changes.
Amazon S3 One Zone-Infrequent Access (S3 One Zone-IA) Key Features:
Same low latency and high throughput performance of S3 Standard Designed for durability of 99.999999999% of objects in a single Availability Zone† Designed for 99.5% availability over a given year Backed with the Amazon S3 Service Level Agreement for availability Supports SSL for data in transit and encryption of data at rest S3 Lifecycle management for automatic migration of objects to other S3 Storage Classes