Solutions Architect Flashcards
IAM users
identity used for anything requiring long-term aws access
Principal
Person or Application that interacts with an IAM through a Request
ARNs(Amazon resource name)
Uniquely identify resources within any AWS accounts
IAM user account limits
5,000 users per account
IAM member limits
Up to 10 groups
IAM groups
containers for users
Groups are not what?
True identities
Can’t be referenced as principal in policy
IAM user
Single principal
IAM role
multiple users. represents a level of access
sts:AssumeRole
IAM Role is involved
Function as a service product
AWS Lambda
Lambda execution role
Lambda service that trusts AWS Lambda
-assumes the sts:AssumeRole and gets tokens for CloudWatch and S3
Break Glass For Key
Emergency Role situation in AWS
Single Sign-on or > 5000 identities
On-premise:
Existing Identities, Active Directory
Off-premise:
external accounts can’t be used directly w/S3
ID Federation
Having a small number of roles to manage and external identities can use these roles to access the AWS resources
Web Identity Federation
Needing millions of users to be authenticated into DynamoDB
Web Identities
No was credentials on the app
Uses existing customer logins
Scales to 100,000,000’s of accounts
Cross-Account Access
Allows an IAM user in one account access resources in another account
i.e.: 1,000’s identities to assume role to get to S3 bucket
AWS Organization
Management Account
SCPs
Service Control Policies:
- They don’t give permissions, are boundaries
- Control what an account CAN/CANNOT do via identity policies
- Appliable to organizations, OU’s, or individual
- Members can be effect, Management accounts can’t
What two policies are assigned to an IAM Role?
Permissions, Trust
Within AWS policies, what is always priority?
Explicit Deny
Which are features of IAM groups?
Admin groupings of IAM users, Can hold identity permissions
Which are true for IAM roles?
Roles can be assumed, When assumed - temporary credentials are generated
What 3 features are provided by AWS organizations?
Consolidated billing
AWS account restrictions using SCP
Account organizations via OU’s
What is CloudTrails functionality?
Account wide auditing and API logging
Is it possible to restrict what account root user can do?
If AWS organizations are used.. but not the management account
Role switching
Assuming role in another AWS account to access the account via console UI
Valid IAM policy types
S3
Private by DEFAULT
S3 Bucket policies
- form of resource policy
- like identity policies, but attached to bucket
- resource perspective permissions
- ALLOW/DENY same or diff accounts
- ALLOW/DENY anonymous principles
Access Control Lists(ACLs)
- In objects and buckets
- subresource
- legacy
- inflexible and simple permissions
Identity
- Controlling different resources
- You have preference for IAM
- Same account
Bucket
- Just controlling S3
- Anonymous OR Cross-Account
ACLs
NEVER, unless you MUST
Normal access for Static Website Hosting
via AWS APIs
Static Website Hosting allows access via
HTTP; blogs
Documents in Static Website Hosting
Index and Error documents are set
Website endpoints created in
Static Web Hosting
Bucketname Matters
Custom Domain via R53 MUST match
S3 offloading hosting
Take all media to static hosting in S3
S3 out-of-band page hosting
Can be accessed via cellular network
-can be used as a backup page in case a web app is down
” * “
Applies to all policies within “Principal”
Object versioning
Default is Disabled
Once enabled, cannot be Disabled, but can be suspended
id = null
Once changed, version id will be updated.
Delete Marker
Hides past versions of an object
MFA Delete
Enabled in versioning configuration
MFA required to change bucket versioning state
Required to delete versions
Serial number (mfa) + code passed w/API calls
Single PUT Upload
Single data streeam to S3
If stream fails, upload fails, and requires full restart
Speed & reliability = limit of 1 stream
Multipart Upload
Data is broken up
Min data size = 100MB for multipart
10,000 max parts, 5mb -> 5gb
Last part can be smaller than 5mb
Parts can fail and be restarted
Transfer rate = speeds of all parts
S3 Transfer Acceleration
When Edge Locations support S3 buckets in remote regions
Encryption at rest
To deter from physical tampering
e.g; password. ‘a very expensive paperweight’
Encryption in transit
Data is encrypted before leaving laptop and upon arrival
Ciphertext
encrypted data
KMS
Keys never leave KMS - Provides FIPS 140-2 (L2)
Regional & Public Service; separate product in each region
Create, store, manage cryptographic keys
Symmetric & Asymmetric keys
Cryptographic operations
- Backing Key ( and previous backing keys)
- Aliases
CMK
Customer Master Key managed by KMS
Logical - ID, date, policy, desc & state
Backed by physical key material; generated or imported
Up to 4kb of data
Isolated to a region & never leave
Support rotation
DEKs (Data Encryption Key)
GenerateDataKey through KMS, works on > 4kb
KMS does not store the DEK beyond generating them
- Plaintext version
- Ciphertext version
- Discard
- Store encrypted key w/data
Key Policies
Key policies ( resource )
Every CMK has one
Key Policies + IAM policies
S3 encryption
Buckets aren’t encrypted, objects are
Client-side encryption
server-side encryption
SSE-C
Server side encryption with customer-provided keys
SSE-S3
Server side encryption with amazon S3-managed keys
- AES256
- Stored persistently, data is at rest
SSE-KMS
Server side encryption w/customer master keys (CMKs) stored in was key management service
Method: Client-Side
Key Management: YOU
Encryption Processing: YOU
Method: SSE-C
Key Management: YOU
Encryption Processing: YOU
Method: SSE-S3
Key Management: S3
Encryption Processing: S3
Method: SSE-KMS
Key Management: S3 & KMS
Encryption Processing: S3
Extras: Rotation Control, Role Separation
Bucket Default Encryption
Bucket header: x-amz-server-side-encryption
S3 standard
S3 storage class
Objects are replicated across 3+ AZs in the AWS region
AZ-A: 99.9% durability for 10mil objects, 1 object loss per 10k years
AZ-B: replication over 3 AZ’s & content-md5 checksums & cyclic redundancy checks (CRCs) used to detect & fix data corruption
S3 standard API
Stored in HTTP/1.1 200 OK response by S3 API endpoint
S3 standard billing
Billed for data stored. No specific retrieval fee, no minimum duration, no minimum size
S3 standard-IA
S3 storage class
Objects are replicated across 3+ AZs in the AWS region
AZ-A: 99.9% durability for 10mil objects, 1 object loss per 10k years
AZ-B: replication over 3 AZ’s & content-md5 checksums & cyclic redundancy checks (CRCs) used to detect & fix data corruption
S3 standard-IA billing
per gig data retrieval fee, overall cost increases
S3 standard-IA need to know
should be used for long-lived data which is important or irreplaceable or data access is infrequent.
Do not: use for small files or temporary data, data that is constantly accessed,
S3 one Zone-IA
- Used for long-lived data (non-critical & replaceable) and where access is infrequent
- Doesn’t provide multi-AZ resilience model of Standard or Standard-IA. 1 used w/in region
- Min duration of 30 days
- Min capacity of 128kb/object
S3 Glacier
Objects cannot be made publicily accessible, requires retrieval process
Data is retrieved to S3 standard-IA temporarily
Expedited(1-5 minutes)
Standard(3-5 hours)
bulk(5-12 hours)
First byte latency = minutes or hours
S3 Glacier Deep Archive
40kb min size
180 Day min Duration
Requires retrieval process
Restore time longer than Glacier
Standard(12 hours)
Bulk(up to 48 hours)
S3 Intelligent Tiering
Tiers: Frequent Access, Infrequent Access, Archive, Deep Archive
(Monitoring & Automated Migration)
- automatically moves objects not accessed for 30 days to low cost infrequent access tier, archive, deep archive
- used for long-lived data, w/changing or unknown patterns
- cost per 1k objects. frequent access tier costs same as S3 standard, infreqent same as Standard-IA.
- Archive/Deep Archive comparable to glacier equivalents
S3 Lifecycle Configuration
Set of Rules
For Bucket or groups of objects
Transaction Actions & Expiration Actions
S3 Lifecycle Configuration - Transitions
S3 Standard
- > S3 STandard-IA
- > S3 Intelligent Tiering
- > S3 One Zone-IA
- > S3 Glacier
- > S3 Glacier Deep Archive
For exam: small object has to remain for min 30 days on S3 standard before to S3 Standard-IA or S3 One Zone-IA
S3 Replication - CRR (Cross-Region Replication)
Source bucket & desination bucket are different regions
S3 Replication - SRR (Same-Region Replication)
Source bucket & destination bucket are same regions
S3 Replication options
- All objects or subset
- Storage class - default is to maintain
- Ownership - default is that they will be owned by the source account
- Replication Time control (RTC): SLA for buckets to be in sync
S3 Replication Considerations for test
- Replication is NOT retroactive & versioning needs to be ON
- One-way replication from source -> destination
- Unencrypted, SSE-S3 & SSE-KMS (w/extra config)
- Source bucket owner needs permissions to objects
- Won’t replicate system events, Glacier or Glacier Deep Archive
- DELETES are NOT replicated (to prevent against malicious deletions)
Why replication?
- SRR (log aggregation
- SRR (PROD and TEST sync)
- SRR (Resilience w/strict sovereignty)
- CRR (Global resilience improvements)
- CRR (latency reduction)
S3 Presigned URL
iamadmin -> 1. Generate presignedURL from Bucket, 2. Bucket sends presignedURL
- GET & PUT operations
- whoever uses presignedURL is using as if they were iamadmin
S3 Presigned URLs Exam
- You can create URL for object you have NO ACCESS TO
- When using URL, permissions match identity that generated it
- Access denied could mean the generating id NEVER had access, or doesn’t know
- Don’t generate w/a role, URL stops working when temporary credentials expire
S3 Select & Glacier Select
- can store objects (up to 5TB)
- You often want to retrieve ENTIRE object
- Retrieving 5TB, takes time, uses %TB
- Filtering at client side doesn’t reduce this
- Service lets you use SQL-like statements
- to select part of object, pre-filtered by S3
- CSV, JSON, Parquet, BZIP2 compression for CSV & JSON
S3 Event notifications
- Notification generated when events occur in bucket
- can be delivered to SNS, SQS, & Lambda functions
- Object created (Put, Post, Copy, CompleteMultiPartUpload)
- Object delete (*, Delete, DeleteMarkerCreated)
- Object restore (Post (Initiated), Completed)
- Replication (OperationMissedThreshold, OperationReplicatedAfterThreshold,OperationNotTracked,OperationFailedReplication)
Event Notifications
CREATE, DELETE, RESTORE, REPLICATE
CloudWatch Logs
- Public Service - usable from AWS or on-premises
- Store, Monitor, & access logging data
- AWS Integrations - EC2 VPC flow logs, lambda, ClouTrail, R53, more
- Can generate metric logs based on metric filter
Logging streams
Log events from the same source
-Log event format: YYYYMMDDHHMMSS message
CloudTrail Event
Logs API calls & activities
CloudTrail Event History
Stored for 90 days by default
CloudTrail Event Default
Enabled by default - no cost for 90 days
CloudTrail customization
Create 1 or more trails
CloudTrail Management
Management of Events & Data Events
CloudTrail regions
Global Services - one region or all regions
CloudTrail & CloudWatch
Manage CloudTrail events through CloudWatch logs
CloudTrail Exam Essentials
- Enabled by default, for 90 days, no s3
- Trails are how you configure S3 & CWLogs
- Management events only by default
- IAM, STS, CloudFront => Global Service Events
- NOT REAL TIME - there is a delay, ~15 minutes
VPC Considerations
- Size of VPC
- Are there Networks we can’t use…
- Avoid ranges that other services use
- VPC structure - tiers & resiliency zones
VPC AWS considerations
- VPC minimum /28 (16 IP’s), maximum /16 (65456 IP’s)
- Avoid common ranges - avoid future issues
- Reserve 2+ networks per region being used per account
Custom VPC
- Regional service - all AZs in the region
- Isolated network
- Nothing IN or OUT w/out explicit configuration
- Flexible configuration - simple or multi-tier
- Hybrid networking - other cloud & on premises
- Default or dedicated tenancy!
VPC CIDRs
- IPv4 cidr blocks & public ips
- 1 primary private ipv4 cidr block
- min /28 (16 IP) max /16 (65,536 IPs)
- Optional secondary ipv4 blocks
- Optional single assigned ipv6 /56 cidr block
DNS in VPC
- Provided by R53
- VPC ‘Base IP + 2’ address
- enableDnsHostnames - gives instances DNS names
- enableDnsSupport - enables DNS resolution in VPC
VPC Subnets
- AZ resilient
- Subnetwork of a VPC - w/in a particular AZ
- 1 subnet => 1 AZ, 1 AZ => 0+ subnets (EXAM Q)
- IPv4 CIDR is a subset of the VPC CIDR
- Cannot overlap w/other subnets
- Optional IPv6 CIDR (/64 subset of the /56 VPC - space for 256)
- Subnets can communicate w/other subnets in the VPC
Subnet IP Addressing
- Reserved IP addresses (5 in total)
- 10.16.16.0/20(10.16.16.0 => 10.16.31.255)
- Network address (10.16.16.0)
- ‘Network +1’ (10.16.16.1) - VPC Router
- ‘Network +2’ (10.16.16.2) - reserved (DNS*)
- ‘Network +3’ (10.16.16.3) - reserved future use
- Broadcast address 10.16.31.255 (last IP in subnet)
VPC Router
- A highly available router - every VPC has one
- Every subnet ‘network +1’ address
- Routes traffic between subnets
- controlled by ‘route tables’ each subnet has one
- VPC has Main route table - subnet default
Internet Gateway (IGW)
- Region resilient gateway attached to a VPC
- 1 VPC = 0 or 1 IGW, 1 IGW =0 or 1 VPC
- Runs from w/in AWS public zone
- Gateways traffic btw VPC & Internet or AWS public zone (S3…SQS…SNS…etc)
- Managed - AWS handles performance
Using an IGW
- Create IGW
- Attach IGW to VPC
- Create custom RT
- Associate RT
- Default Routees => IGW
- Subnet allocate IPv4
IPv4 Addresses w/a IGW
Never touches the actual servers inside the VPC
-IGW changes address of packet between instance and server
Bastion Host/Jumpbox
- Bastion Host = Jumpbox
- Instance in a public subnet
- Incoming management connections arrive there
- Access internal VPC resources
- Often the only way IN to a VPC
Stateful Firewall
Intelligent to identify REQUEST & RESPONSE components of connection being related
Stateless Firewall
Doesn’t understand the ‘state’ of connections
-Remember the ‘response ephemeral ports’, not well known APP port
NACL (Network Access Control Lists)
NACLs filter traffic crossing the subnet boundary INBOUND or OUTBOUND
- Connections w/in a subnet aren’t impacted by NACLs
- STATELESS; 1 x INBOUND & 1 x OUTBOUND
Custom NACL
- Can be created for a specific VPC & are initially associated w/NO SUBNETS; result -> ALL TRAFFIC IS DENIED
- 1 INBOUND rule: implicit(*) DENY
- 1 OUTBOUND rule: implicit(*) DENY
NACL exam
- STATELESS: REQUEST & RESPONSE seen as different
- Only impacts data crossing subnet boundary
- NACLs can EXPLICITLY ALLOW & DENY
- IPs/CIDR, Ports & Protocols - no logical resources
- NACLs cannot be assigned to AWS resources - only subnets
- Use w/Security Groups to add explicit DENY(Bad IPs/Nets)
- Each subnet can have ONE NACL(Default or custom)
- NACL can be associated w/MANY SUBNETS
VPC Security Groups (SG)
- STATEFUL - detect response traffic automatically
- ALLOWED (IN/OUT) REQUEST = allowed response
- NO EXPLICIT DENY……only ALLOW or implicit DENY
- …..can’t block specific bad actors
- Supports IP/CIDR & logical resources
- Attached to ENI’s, not instances
(SG) Self References
- “SG source” is same as “anything with the SG attached”
- Using “self reference” means “anything with this SG attached”
- Scales w/ ADDS & REMOVES from the SG
NAT
Set of processes - remapping SRC or DST IPs
- IP masquerading - hiding CIDR blocks behind one IP
- Public IPv4 addresses are running out
- Gives Private CIDR range OUTGOING internet* access
NAT architecture
- (Not publicly routable) Route table => NATE Gateway => VPC router = Internet Gateway/ Public Internet
NAT Gateways
- runs from public subnet
- uses elastic IPs (static IPv4 public)
- AZ resilient service (HA in that AZ)
- For region resilience - NATGW in each AZ….
- …RT in for each AZ w/NATGW as target
- Managed, scales to 45 Gbps, $ duration & data volume
What about IPv6?
- NAT isn’t required for IPv6
- all IPv6 addresses in AWS are publicly routable
- Internet Gateway works w/ALL IPv6 IPs directly
- Exam Q: NAT Gateways DON’T WORK WITH IPV6
- ::/0 Route + IGW for bi-directional connectivity
- ::/0 + Egress-only internet gateway - outbound only
Resilient Gateway Architecture; w/o SSH Agent Forwarding
-Private Subnet (Internal Test) a. public part is added an ‘authorised key’ on SSH servers <=> Public Subnet (Bastion) a. no private key <=> Public Internet
Resilient Gateway Architecture; w/ SSH Agent Forwarding
-Private Subnet (Internal Test) a. private key remains on client at all times, authentication requests are forwarded <=> Public Subnet (Bastion) a. SSH client connects agent forwarding) <=> Public Internet a. ssh-agent service (ssh-add)
Hardware Assisted Virtualization
The Hypervisor has knowledge of the virtualization
SR-IOV
Enhanced networking in EC2
EC2 architecture
- Virtual Machines (OS + Resources)
- Run on EC2 hosts
- Shared Hosts or Dedicated Hosts
- Hosts = 1 AZ -AZ fails, Host Fails, Instances fail
EC2 Exam Tips
Look for availability zones in answer
EC2 Types
- Raw cpu, memory, local storage capacity & type
- Resource ratios
- storage & data network bandwidth
- system architecture/vendor
- additional features and capabilities
EC2 categories
General purpose - default- diverse workloads, equal resource ratio
Compute organized - media processing, HPC, scientific modeling, gaming, machine learning
Memory Optimized - processing large in-memory datasets, some database workloads
Accelerated computing - hardware GPU, field programmable gate arrays (FPGAs)
Storage optimized - sequential & random IO - scale-out transactional databases data warehousing, elastic search, analytics workloads
“R5dn.8xlarge”
- Instance family - “R”
- Instance generation - “5”
- Additional capabilities - “dn”
- Instance type - “R5dn.8xlarge”
- Instance size - “8xlarge”
EC2 Key Terms Part 1
- Direct (local) attache storage - storage on the EC2 host
- Network attached storage - volumes delivered over the network (ebs)
- Ephemeral storage - temporary storage
- Persistent storage - permanent storage - lives on past the lifetime of the instance
EC2 Key Terms Part 2
-Block storage - VOLUME presented to the OS as a collection of blocks…. no structure provided.
MOUNTABLE. BOOTABLE.
-File storage - presented as a file share…. has structure.
MOUNTABLE. NOT BOOTABLE.
-Object storage - collection of objects, flat.
NOT MOUNTABLE. NOT BOOTABLE.
Storage Performance
IO(block) size X IOPS = Throughput
- IO: 16k, 64k, 1MEG
- IOPS: revolutions per second
- Throughput: Rate of data, XX MB/S, “65 megabits per second”
Elastic Block Store (EBS)
- Block storage - raw disk allocations (volume) - can be encrypted using KMS
- ….instances see block device & create file system on this device (ext3/4, fs)
- storage is provisioned in ONE AZ (resilient in that AZ)
- Attached to *one EC2 instance(or other service) over a storage network
- ….DETACHED and REATTACHED, not lifecycle linked to one instance…persistent
- Snapshot(backup) into S3. Create volume from snapshot(migrate btw AZs)
- Different physical storage types, different sizes, different performance profiles
- Bill based on GB-month (some cases for performance)
EBS - General Purpose SSD - GP2
-Volumes: 1GB, up to 16TB
EBS - HDD-based
- st1; cheap, throughput optimized
- sc1; cheaper, cold HDD
Instance Store Volumes
- Block storage devices
- Physically connected to 1 EC2 host
- Instances on that host can access them
- Highest storage performance in AWS
- Included in instance price….
- only ATTACHED AT LAUNCH
Exam Q’s
- Instance store volumes are local on a EC2 host
- Instance volumes are only added AT LAUNCH
- Instance volumes lost on instance move, resize, or hardware failure
- High performance
- You pay for it anyway - included in instance price
- INSTANCE STORE VOLUMES ARE TEMPORARY
Instance store vs EBS
- Persistence .. EBS (avoid instance store)
- Resilience .. EBS (avoid instance store)
- Storage isolate from instance lifecycle .. EBS
- Resilience w/App in-built replication … it depends
- High performance needs … it depends
- Super high performance needs … instance store
- Cost … instance store (often included)
- Cheap = ST1 or SC1
- Throughput .. streaming … ST1
- Boot ……..NOT ST1 or SC1
- GP2/3 - up to 16,000 IOPS
- IO1/2 - up to 64,000 IOPS (*256,000)
- RAID0 + EBS up to 260,000 IOPS (io1/2-BE/GP2/3)
- More than 260,000 IOPS - INSTANCE STORE
EBS Snapshots
- snapshots are incremental volume copies to S3
- First is a full copy of ‘data’ on the volume
- Future snaps are incremental
- Volumes can be created (restored) from snapshots
- Snapshots can be copied to another region
EBS snapshots/volume performance
- new EBS volume = full performance immediately
- snaps restore lazily - fetched gradually
- requested blocks are fetched immediately
- force a read of all data immediately
- fast snapshot restore (FSR) - immediate restore
- up to 50 snaps per region. set on the snap & AZ
EBS encryption
- customer managed;
- EC2 instance <=> EC2 host (plaintext stored at rest)
- aws/ebs;
- EC2 instance <=> EC2 host (customer master key + data encryption key), ciphertext stored at rest, any snapshots created will share DEK
EBS exam
- Accounts can be set to encrypt by default - default CMK
- Otherwise choose a cmk to use
- Each volume uses 1 unique DEK
- Snapshots & future volumes use the same DEK
- Can’t change a volume to NOT be encrypted
- OS isn’t aware of encryption….no performance loss
EC2 Network & DNS architecture
Primary ENI:
- MAC address
- Primary IPv4 Private IP
- 0 or more secondary IPs
- 0 or 1 Public IPv4 Address
- 1 elastic IP per private IPv4 address
- 0 or more IPv6 addresses
- Security groups
- Source/destination check
Secondary ENI:
-^^
EC2 Network/DNS Exam
- Secondary ENI + MAC = licensing
- Multi-homed (subnets) management & data
- Different security groups - multiple interfaces
- OS - DOESN’T SEE PUBLIC IPV4
- IPv4 Public IPs are dynamic … stop/start = CHANGE
- Public DNS = private IP in VPC, public IP everywhere else
AMI lifecycle
- LAUNCH : AMI -> Instance; BOOT /dev/xvda DATA /dev/xvdf
- CONFIGURE : Instance BOOT /dev/xvda DATA /dev/xvdf -> customizations
- CREATE IMAGE : Instance BOOT /dev/xvda DATA /dev/xvdf -> AMI (referenced w/ block device mapping)
- LAUNCH: AMI -> Instance BOOT /dev/xvda DATA /dev/xvdf
AMI Exam tips
- AMI =One region, only works in that one region
- AMI baking …. creating an AMI from a configured instance + application
- An AMI can’t be edited …launch instance, update configuration & make new AMI
- Can be copied btw regions ( includes its snapshots)
- Remember permissions .. default = your account
EC2 Purchase Options - On Demand
- On-Demand instances are isolated
- Multiple customer instances run on shared hardware
- Instances of different sizes run on same EC2 hosts (consuming defined allocation of resources)
- Per-second billing while instance is running.
- Associated resources such as storage consume capacity; all are billable, regardless of instance state
On Demand pricing
- Default purchase option
- No interruption
- No capacity reservation
- Predictable pricing
- No upfront cost
- No discount
- Short term workloads
- Unknown workloads
- apps which can’t be interrupted
Spot pricing
-When AWS sells unused EC2 host capacity for up to 90% discount; spot price is based on spare capacity at a given time
When to not use spot instances?
Never use SPOT for workloads which can’t tolerate interruptions
Spot benefits
- Non time critical
- Anything which can be rerun
- Burst capacity needs
- Cost sensitive workloads
- Anything which is stateless
Options - Reserved
Reservations are for 1 year or 3 year terms, you pay for the entire term
- No reservation; full per/s price
- Matching instance; reduced or no p/s price
- Unused reservation; still billed
- Partial coverage of larger instance
Dedicated Hosts
Customer pays for the host
- Software licensing based on sockets/cores
- Software host affinity links instances to hosts
Dedicated Hosts vs Shared vs Instances
- Shared; on-demand, no host exposure, per sec charges
- Host; Pay for the host, no instance charges, capacity management required
- Instance; you don’t own, or share host,…..extra charges for instances, but dedicated hardware
Scheduled Reserved Instances
- Ideal for long term usage which doesn’t run constantly
- Batch processing daily for 5 hours starting @ 23:00
- Weekly data, sales analysis, every Friday for 24 hrs
- Doesn’t support all instance types or regions. 1,200 hours per year & 1 year term minimums
- 100 hours of EC2 per month
Capacity reservations
- Regional reservation provides billing discounts for valid instances launched in any AZ in that region
- Zonal reservations only apply to one AZ providing billing discounts and capacity reservation in that AZ
- Flexible but, don’t reserve capacity w/in AZ - risk during major faults w/limited capacity
- Full price & no capacity reservation
- On-demand capacity reservations can be booked to ensure you always have access to capacity in AZ when you need it - but at full on-demand price. No term limits - but you pay regardless of if you consume it.
EC2 savings plan
- Hourly commitment for 1 or 3 year term
- Reservation of general compute $ amounts ($20 per hour for 3 years)
- Specific EC2 savings plan - flexibility on size & OS
- Compute products, currently Ec2, fargate, & lambda
- Products have an on-demand rate & savings plan rate
- Resource usage consumes savings plan commitment @ reduced savings plan rate
- Beyond your commitment … on-demand is used
Vertical scaling
- Each resize requires a reboot - DISRUPTION
- Larger instances often carry a $ PREMIUM
- Upper cap on performance - instance size
- No application modification required
- Works for all applications - even monoliths
Horizontal scaling
- Sessions, sessions, sessions
- Requires application support OR off-host sessions
- No disruption when scaling
- No real limits to scaling
- Less expensive - no large instance premium
- able to be more granular
Horizontal vs Vertical Scaling Exam
- Horizontal: 1 BOB 2 BOB 4 BOB
- Vertical: bob.small bob.medium bob.large
Instance Metadata
- EC2 service provides data to instances
- Accessible inside all instances
- http://169.254.169.254, always want the ‘latest’ ‘meta-data’
- http://169.254.169.254/latest/meta-data/
- Environment
- Networking
- Authentication
- User-Data
- NOT AUTHENTICATED or ENCRYPTED
Elastic container service (ECS)
- ECS cluster; Network O
- Service definition(Container definition(Task definition, task role))
- Container definition: image & ports
- Task definition: security(task role), container(s), resources
- Task role: IAM role which the TASK assumes
- Service: how many copies, HA, restarts
ECS - EC2 mode
-ASG encompasses AZA & AZB, ECS handles cluster from Registry
ECS - Fargate Mode
- Serverless option
- Container images uses Registry
- Containers are hosted on Fargate Shared Infrastructure, no visibility of other customers
EC2 vs ECS(EC2) vs Fargate
- If you use containers….. ECS
- Large workload; price conscious: EC2 mode
- Large workload; overhead conscious: Fargate
- Small/Burst workloads; Fargate
- Batch/Periodic workloads; Fargate
What cluster modes are available within ECS?
Network only (Fargate)
EC2 Linux + Networking
EC2 Windows + Networking
EC2 Bootstrapping
- Allows EC2 Build automation
- User Data - accessed via meta-data IP
- http://169[.]254[.]169[.]254/latest/user-data
- Anything in user data is executed by the instance OS
- ONLY on launch
- EC2 doesn’t interpret, OS needs to understand user data
Bootstrapping architecture
-AMI -> Instance
User data key points
- Opaque to EC2… its BLOCK DATA
- NOT secure - don’t include passwords or long term credentials in it
- User data limited to 16KB in size
- Can be modified when instance stopped
- ONLY EXECUTED ONCE AT LAUNCH
AWS::CloudFormation::Init
EC2 Instance Roles
- IAM Role allows EC2 Service to assume it (IAM Role)
- Credentials are inside meta-data
- iam/security-credentials/role-name
- Automatically rotated - always valid
- should always be used rather than adding access keys into instance
- CLI tools will use ROLE credentials automatically
SSM Parameter Store
- Storage for configuration & secrets
- String, StringList, & SecureString
- License codes, Database strings, full configs & passwords
- Hierarchies & Versioning
- Plaintext & cipher text
- Public Parameters - Latest AMIs per region
Logging on EC2
- CloudWatch is for metrics
- CloudWatch logs is for logging
- Neither natively capture data inside an instance
- CloudWatch agent is required…
- …plus configuration and permissions
What are the EC2 placement group options?
- Cluster: Pack instances close together
- Spread: Keep instances separated
- Partition: Groups of instances spread apart
What are features of the cluster placement groups?
*When you want to achieve the absolute highest performance possible in EC2
- Note: have to be launched into a single AZ
- All members have direct connections to each other
- Lowest latency and max PPS possible in AWS
What are the data specs for cluster placement groups?
- 10Gbps p/ stream
- 5Gbps normally
Do cluster placement groups share resources?
-Same rack, sometimes additionally the same host
Cluster Placement Groups
- Can’t span AZ’s, one only - locked when launching first instance
- Can span VPC peers - but impacts performance
- Requires a supported instance type
- Use the same type of instance(not mandatory)
- Launch at the same time(not mandatory, highly recommended)
- 10Gbps single stream performance
What are common use cases for cluster placement groups?
Performance, fast speeds, low latency(high performance compute)
Spread placement groups
- Provides infrastructure isolation & resilience
- …..each instance runs from a different rack
- each rack has its own network and power source
- 7 instances per AZ (HARD LIMIT)
- Not supported for dedicated instances or hosts
- Use case: small number of critical instances that need to be kept separated from each other
What are partition placement groups?
*designed for infrastructure that has 7+ instances per AZ but you still need ability to separate those instances into separate fault domains
-Similar architecture to spread placement groups
Do partitions share their racks in partition placement groups?
-Each rack has its own rack - no sharing between partitions
*How are instances placed in partition placement groups?
Instances can be placed in a specific partition, or auto placed
-Only allowed up to 7 partition per AZ
*Do partition placement groups have topology?
Contains impact of failure to part of an application. Is also great for topology-aware applications including;
-HDFS, HBase, Cassandra
What is an EC2 Dedicated host?
-An EC2 Host dedicated to you
Specific instance families include a1, c5, m5
Do dedicated EC2 hosts charge you?
No instance charges, you pay for the host
-On demand & reserved options are available
What hardware is on an EC2 dedicated host?
Host has physical sockets and cores;
- it dictates how many instances can be run on that host
- licensed software based on physical sockets or cores, can utilize visibility of the hardware
How do dedicated EC2 hosts work?
Designed for a specific family & instance size.
What are limitations for EC2 dedicated hosts?
- AMI limits - RHEL, SUSE Linux, & Windows AMIs are not supported
- Amazon RDS instances are NOT supported.
- Placement groups are not supported for dedicated hosts
- Hosts can be shared w/other ORG accounts through the RAM(Resource Access Manager)
What is enhanced networking?
A featured used to improve the overall performance of EC2 networking. Required for any high-end performance features like cluster-placement groups
What is SR-IOV within enhanced networking?
Enhanced networking using SR-IOV (SingleRoot-IOVirtualization).
NIC is virtualization aware.
Benefits of enhanced networking?
- No charge, available on most EC2 types.
- Higher I/O & lower host CPU usage
- More bandwidth
- Higher packets-per-second(PPS)
- Consistent lower latency
What is EBS and how is it used?
- EBS = Block storage over the network
- Historically network was shared .. data & EBS
- EBS optimized means dedicated capacity for EBS
- Most instances support & have enabled by default
- Some support, but enabling costs extra
What is a hosted zone?
- R53 Hosted Zone is a DNS DB for a domain
- Zones are globally resilient (multiple DNS servers)
- Are created w/domain registration via R53, can be created separately
- Host DNS records (A, AAAA, MX, etc)
- Hosted zones are the DNS system references - Authoritative for domain
What are R53 specifics?
- DNS DB (zone file) hosted by R53(public name servers)
- Accessible from public internet & VPCs
- Hosted on “4” R53 name servers specific to zone
- use “NS records” to point to the NS
- Resource records(RR) created w/in hosted zone
- Externally registered domains can point at R53 public zone
What are R53 private hosted zones?
- The same as a Public hosted zone…..just not public
- Associated w/VPC’s
- Only accessible in those VPCs
- Use different accounts supported via CLI/API
- Split-view(overlapping public & private) for PUBLIC & INTERNAL use w/the same zone name
R53 CNAME records
- CNAME maps NAME to another NAME
- CNAME is invalid for naked/apex
- with just CNAME - catagram.io => ELB would be invalid
R53 ALIAS records
- ALIAS records map a NAME to an AWS resource
- Can be used for both naked/apex and normal records
- For non apex/naked - functions like CNAME
- There is no charge for ALIAS requests pointing at AWS resources
- For AWS services - default to picking ALIAS
- Should be the same “type” as what the record is pointing at
- API Gateway, CloudFront, Elastic Beanstalk, ELB, Global Accelerator & S3
Route53 Failover routing
-If target of health check is healthy the primary record is used
Route 53 Mutli Value Routing
- Supports multiple records with the same name
- Supports up to 8 healthy records, that are returned
- Improves availability; NOT a replacement for load balancing
- Each record is independent & can have an associated health check
- records which fail health checks won’t be returned when queried
R53 Weighted Routing
- total weight (100)
- ‘0’ weight means record is never returned unless all are ‘0’ then all are considered
- each record is returned based on its record weight vs total weight
- if chosen record is unhealthy, process of selection is repeated until health record is chosen
When is weighted routing used?
For simple load balancing or testing new software versions
What is latency-based routing? When is it used?
- When you’re trying to optimize for performance & user experience
- AWS maintains a db of latency between users & general location and the regions tagged in records
- routing supports one record w/same name in each AWS region
- record that is returned is the one which offers the lowest estimated latency & is healthy
What is geolocation routing? How is it used?
- Similar to latency, only instead of latency the location of customers and location of resources are used to influence resolution decisions
- When creating records, you tag the records with the location
- Four types: “continent”, “country”, “subdivision”, “default”
- when user makes resolution request, IP check verifies location of the user
Resolution request only returns relevant records
Geolocation Routing
What is geoproximity routing? How is it used?
- Geoproximity aims to calculate the distance between the customer & the record, and return an answer with the lowest distance
- Similar to latency policy however it works on distance
- routing is distance based (including bias)
Geoproximity Routing
- “+” or “-” can be added to rules.
- “+” increases region size & decreases neighboring regions
- records can be tagged w/an AWS region or lat & long coordinates
Route 53 Interoperability
- 2 jobs - Domain registrar & domain hosting
- R53 can do both, OR either domain registrar or domain hosting
- Accepts your money (domain registration fee)
- allocates 4 name servers (NS) (domain hosting)
- service creates a zone file (domain hosting) on above NS
- service communicates w/registry of the TLD (Domain registrar)
- sets NS records for domain to point at the 4 NS above
Route53 - Both Roles
Route 53 - Registrar Only
Route 53 - Hosting Only
What is ACID and BASE?
- ACID and BASE are DB transaction models
- CAP theorem - Consistency, Availability, Partition Tolerant (resilience) - choose 2
- ACID = Consistency
- BASE = Availability
Name the parts of an A.C.I.D transaction:
ATOMIC, CONSISTENT, ISOLATED, DURABLE → RDS ….. limits scaling
- ALL or NO components of transaction succeeds or fails
- Transactions over from db from one valid state to another = nothing in=between is allowed
- If multiple transactions occur at once, they don’t interfere w/each other, each executes as if the only one
- Once committed, transactions are durable. Stored on non-volatile memory, resilient to power outages or crashes
Name the parts of a B.A.S.E transaction:
BASICALLY AVAILABLE, SOFT STATE, EVENTUALLY CONSISTENT → DynamoDB* …. consistency
- READ & WRITE operations are available ‘as much as possible’ but without any consistency guarantees - kinda, maybe
- DB doesn’t enforce consistency, this is offloaded onto the application/user
- If we wait long enough, reads from the system will be consistent
Reasons for using databases on EC2?
- Access to the DB instance OS
- Advanced DB option tuning….(DBROOT)
- ….. vendor demands
- DB or DB version AWS don’t provide…
- Specific OS/DB combination AWS don’t provide
- Architecture AWS don’t provide (replication/resilience)
- Decision makers who ‘just want it’
Reasons you should not use databases on EC2?
- Admin overhead - managing EC2 & DBHost
- Backup/ DR Management
- EC2 is single AZ
- Features - some of AWS DB products are amazing
- EC2 is ON or OFF - no serverless, no easy scaling
- Replication - skills, setup time, monitoring & effectiveness
- Performance…AWS invest time into optimization & features