Storage

Question 1

EBS

Answer 1

• Network drive you attach to ONE instance only.
• Linked to specific AZ, the only way to move is do snapshot and restore
• Volumes can be resized
• Best performance when EBS and instance type are well matched

Question 2

EBS Volume Types

Answer 2

• gp2: General (cheap)
• • 3 IOPS/GB, min 100 IOPS, burst to 3000 IOPS, max 16000 IOPS
• • 1GB - 16TB. +1 TB = + 3000 IOPS
• IO1. Provision IOPS (expensive)
  – Min 100 IOPS, Max 64000 IOPS (Nitro) or 32000 (Other)
  – 4GB - 16TB, Size of volume and IOPS are independent
  For databases
• ST1. Throughput optimized HDD
  – 500GB - 16TB, 500 MB/s throughput
  For data analtics
• SC1 Cold HDD
• • 250GB - 16TB, 250MB/s throughput

Question 3

EBS RAID COnfigurations

Answer 3

• RAID 0, distributed - faster but in case of failure half is lost
• RAID 1, replicated - same speed, in case of failure no data loss

Question 4

EBS Snapshots

Answer 4

• Incremental
• Use IO so don’t run while application is using a lot of traffic
• Stored in S2 (Not visible)
• Not necessary to detach volume to snapshot, but recommended
• Can copy across region for DR
• Can create AMI from snapshot
• EBS volumes restored by snapshots need to be pre- warmed (fio or dd command to read entire volume)
• Can be automated using Amazon Data Lifecycle Manager

Question 5

Local EC2 Instance Store

Answer 5

- Physical Disk, very high IOPS
up to 7.5TB, stripped can reach 30TB
- Block storage
- Cannot be increased in size
- Risk of data loss of hw fails
- Ephemeral, lose, stop or terminate EC2,  instance lose storage
- Survives reboots
- Good for buffer, cache, scratch 
data
- Manual backups

Question 6

EFS

Answer 6

• Linux based only, POSIX, NSF4
• Good for data sharing, cms
• Control access using SGs
• Encryption at rest with KMS
• Only one VPC, but can create one mount target per AZ for redundancy

Question 7

EFS Scale

Answer 7

• 1000s concurrent NFS client, 10GB+/s throughput

- Grow to petabyte scale

Question 8

EFS Performance

Answer 8

• General
• Max IO
  Set at EFS creation time

Question 9

EFS Throughput

Answer 9

Bursting, linked to FS size

Provisioned IO, expensive high throughput to size ratio

Question 10

EFS VPC peering

Answer 10

EC2 can be in another VPC and connected using VPC peering

Question 11

EFS on-prem

Answer 11

• Can be connected using Direct Connect and/or VPN

- Can be accessed using Mount Target IPv4, hostname not supported

Question 12

S3 vs DynamoDB

Answer 12

No indexing facility on S3

1. Use S3 event to notify lambda
2. Lambda reads from S3 using by fetch and inserts metadata and indexed data into DynamoDB

Question 13

S3 vs EFS

Answer 13

S3 is not good for POSIX or file locking use EFS instead

Question 14

S3 Replication

Answer 14

• For latency, for DR, for security
• Cross Region
• Same Region
• Can combine with lifecycle policies
• Must enable S3 bucket versioning

Question 15

S3 Event Notifications

Answer 15

• Delivery in seconds, but can take up to minutes
• If two events same time non versioned object, possible only one event will be fired
• To ensure event for every successful write enable versioning

Question 16

S3 CW

Answer 16

• When CloudTrail enabled, records all bucket level API calls by default
• Can enable object level by enabling CloudTrail on bucket

Question 17

S3 Baseline, Performance

Answer 17

• 3500 PUT/COPY/POST/DELETE per second per prefix

- 5500 GET/HEAD per second per prefix

Question 18

S3 Performance Optimizations Upload

Answer 18

• Multi-part upload, parallel uploads
• • recommended for > 100MB
• • must use for > 5GB
• • retries chunk not whole
• Transfer Acceleration
• • Compatible with multi-part upload
• • Go to edge location, from there use fast AWS network to upload to regional bucket

Question 19

S3 Performance Optimizations Download

Answer 19

• S3 Byte-Range Fetches
• • Parallelise gets by requesting specific byte
• • re-request byte chunk not whole, better resilience
• • Can be used to retrieve only a part of the file, ex. head

Question 20

S3 Select & Glacier Select

Answer 20

• Optimise data transfer size by doing server side filtering using SQL
• Less network transfer, less CPU cost client-side
• Cheaper

Question 21

S3 Cloudfront

Answer 21

• Require that your users access your private content by using special CloudFront signed URLs or signed cookies.
• Require that your users access your content by using CloudFront URLs, not URLs that access content directly on the origin server (for example, Amazon S3 or a private HTTP server). Requiring CloudFront URLs isn’t necessary, but we recommend it to prevent users from bypassing the restrictions that you specify in signed URLs or signed cookies.