Storage and Databases Flashcards
Types of block level storage
- instance stores
- Amazon Elastic Block Store (EBS)
What is an instance store volume?
local storage physically attached to the host running the EC2 instance
What is the downside of an instance store volume?
All data are deleted when the instance is stopped. Because an instance can be restarted on another physical host.
What is instance store volume good for?
temporary storage
What is Amazon EBS?
Elastic Block Store - volumes (physical hard drives) attached to an EC2 instance.
What is EBS good for?
Incremental backups - snapshots.
What is Amazon S3?
Simple Storage Service, object store
What are S3 limits?
- single object limit is 5TB
- total volume - unlimited
Storage tiers
- S3 standard
- S3 infrequent access (IA)
- one-zone IA
- Amazon S3 Glacier
What is stored in object store
data, metadata, key
S3 Standard storage tier - features
- 99.999999999% probability of being intact after 1 year
- data is stored in at least 3 facilities
- good for static website hosting
S3 IA
Infrequent Access
- lower storage price, higher retrieval price
- rapid access when needed
- minimum 3 AZs, backups, disaster recovery
What is Amazon S3 Glacier good for?
audit data that need to be archived
What is WORM policy?
Write once, read many
What is Amazon S3 Glacier?
storage vaults for data that don’t need to be accessed rapidly (from minutes to hours)
What is Amazon S3 Intelligent Tiering?
Service for automatically moving data between tiers according to lifecycle policies.
What is Amazon S3 Intelligent Tiering good for?
Data with unknown or changing access patterns.
What do you pay for with S3 Intelligent Tiering?
a monthly monitoring and automation fee per object
Types of Amazon Glacier
- Instant Retrieval
- Flexible Retrieval
- Deep Archive
Amazon Glacier Flexible Retrieval - retrieval policy
from minutes to hours
Amazon Glacier Instant Retrieval - retrieval policy
minutes
Amazon Glacier Deep Archive - retrieval policy
12 hours, 1-2 a year
EBS vs. S3
EBS:
- up to 16TB
- block store - delta updates possible
- requires an instance
- use case: large vide file which is edited
- not web enabled
S3:
- unlimited
- object store - discrete objects, no delta updates possible (WORM)
- use case: photo analysis website
- serverless
- web enabled
What is Amazon EFS?
Elastic File System - a managed filesystem that scales automatically
EFS vs. EBS
EFS
- not a drive, but a linux filesystem
- regional resource
- scalable
EBS
- a drive (SSD or HDD)
- must be in the same AZ as an EC2 instance it’s connected to
- not scalable
EFS use case
a filesystem shared among multiple servers that run analytics on data located on this filesystem
What is Amazon RDS?
Amazon Relational Database System
RDBMS supported by RDS
PostgreSQL, MySQL, Oracle, MSSQL, etc.
Ways to use RDBMS on AWS.
- lift&shift an existing database to EC2
- use a managed service - Amazon RDS
- Amazon Aurora
Benefits of Amazon RDS
- automated patching
- backups
- redundancy
- failover
- disaster recovery
- can be integrated with other services, e.g. lambda
Amazon Aurora - SQL engines
MySQL, PostgreSQL
Amazon Aurora - benefits
- cheaper - 1/10 cost of commercial dbs
- 6 replicas across facilities
- up to 15 read-replicas
- continuous backups to S3
- point-in-time recovery
- up to 5x faster than MySQL, 3x than PostgreSQL
What is Amazon DynamoDB?
non-relational database, serverless, key-value (no schema)
features of DynamoDB
- scalable
- very performant (ms response time)
- massive throughput
- PB size potential
- granular API access
- redundant storage across AZs, mirrors data across drives
What is Amazon Redshift?
data warehousing as a service (big data)
What is data warehousing use case?
- historical analysis - looking backwards at data that don’t change in real time
- variety of different data
Amazon Redshift specs
- multiple PBs, massively scalable
- 10x more performant than traditional dbs
What is AWS DMS?
AWS Data Migration Service. Secure and easy migration from on premise to cloud.
Benefits of AWS DMS
- source db is operational during migration - downtime is minimized
- source and target can be of different types
Two DB migration types
- between homogenous DBs
- between heterogenous DBs
Homogenous DB migration
Compatible schema, data types, db code, e.g. MySQL -> Amazon RDS for MySQL.
Source can be: on premise, Amazon EC2, Amazon RDS, target: Amazon EC2, Amazon RDS.
One step: user creates a migration task using AWS Database Migration Service.
Heterogenous DB migration
Two steps: 1) convert schema structures, db code using AWS schema conversion tool, 2) migrate using AWS DMS.
Database migration use cases
- dev and test db migration (copy prod to dev or test)
- db consolidation
- continuous db replication - for disaster recovery or due to geographical separation
Types of dbs supported by DMS
relational, non-relational, other data stores
Amazon Document DB
Document db service that supports MongoDB workloads.
Good for content management system: catalogues, user profiles.
Amazon Neptune
A graph DB.
Good for social networking and recommendation engines, fraud detection, knowledge graph.
Anazon Quantum Ledger Database (QLDB)
An immutable system of record - no entry can be changed or removed - complete history of changes.
Good for supply chain, banking, financial records.
Amazon Managed Blockchain
distributed ledger
Amazon Elasticache
Service which adds a layer of optimization to a db. Offers optimization of read times for common requests from mili- to microseconds.
Supports Redis or Memcached engines.
Amazon DAX
DynamoDB Accelerator - in-memory native caching layer to improve read times.
