AWS Flashcards
Check the ECR repository?
aws ecr describe-images –repository-name telematics/gmonstarparser
if image tags are needed:
aws ecr describe-images –repository-name telematics/gmonstarparser –image-ids imageTag=”2021.06.14-RC-1.0.47”
How to add tags to AWS ECR image?
MANIFEST=$(aws ecr batch-get-image –repository-name dockerhub/amd64/ubuntu –image-ids imageTag=16.04 –query ‘images[].imageManifest’ –output text)
aws ecr put-image –repository-name dockerhub/amd64/ubuntu –image-tag “test_date +%y%m%d.%H%M” –image-manifest “$MANIFEST”
How to log into kubernetes using AWS credentials?
export KUBECONFIG=/opt/kubeconfig/kube.config
aws eks update-kubeconfig –cluster-name fc-dev
Check tags of an AWS ECR Image
aws ecr describe-images –repository-name fcpls/xyz –image-id imageTags=dev.1234
IAM Group
IAM Users
IAM Role
Role - assumed by trusted entities - used for delegation.
Root user - has complete administration access.
Max 5000 users per account
Group can’t be nested.
IAM Users can assume a role.
All permissions implicitly denied.
Authentication methods
Signing certificate can be used for authentication for some services.
AWS Security Token Service
request limited privilege credentials for IAM/federated users.
global service
sts.amazonaws.com
all regions enabled.
Billing Alarm
Only root user can update anything related to billing.
Billing preferences > Receive Billing Alerts > CloudWatch > Specify conditions > SNS
Locations
Region
Availability Zone
Edge Location - Cache
Regional Edge Location - between AWS and Edge
Global networkVPC
Logically Isolated network assigned to user within a region.
172.31.0.0/16
Default 1 VPC is created in all accounts.
Default route table and internet gateway added.
3 subnets created in each AZ.
Route 53
Domain Name service
DNS management
Traffic Check
Avilability Check
Domain name registeration
EC2
Elastic Compute Cloud
Genral purpose - T, M
Reserved instance - 1 - 12 or 12 - 36 months Spot Instances Savings Plan Dedicated Hosts Scheduled Instances Capacity Reservations
Limits - increase with Support ticket
EBS
Block storage used for EC2
io1
gp2
AMIs
Amazon Machine Image = Launch Permissions + Snapshots + Block dedvice Mapping
EC2 Instance Metadata
Information that can be used to configure the application on the instance.
http://169.254.169.254/latest/meta-data/
EC2 User Data
Execute commands to configure the instance at the time of launch.
http://169.254.169.254/latest/user-data/
EC2 Instance Status Check and System Status Check
Instance - customer resposibility
System - AWS responsibility
For system - stop and then start the instance - this will change the host machine.
EC2 detailed Monitoring
frequency 1 minute from 5 minutes.
IP Addresses
Public IP - lost when instance stopped.
Private IP - lost when instance is terminated
Elastic IP - Charged when not used. Associated with the private IP. Can be transferred to other resources.
EC2 instance doesnt know anything about the public IP.
Internet Gateway
1:1 NAT Gateway
Translates Public IP to Private IP.
Jumphost/Bastion - Agent Forwarding
ssh-add -K xyz.pem
ssh -A ec2-user@
ssh ec2-user@
Pageant for windows machine.
NAT instances and Gateways
Network Address Translation
Instance managed by customer Manual scaling Not HA Assign Security Group Needs Elastic IP Enable Port Forwarding - disable, Source/Destination Check
Placement Groups
- Cluster - close together, tightly coupled.
- Partition - spread across logical partitions, groups of partitions, do not share resource. (Max 7 per AZ)
- Spread - Reduce correlated failure - distinct underlying hardware.
Elastic Load Balancing
Application layer 7 http https path, host, query string, parameter, sourceIP based routing Targets - IP, Lambda, Containers Sticky sessions
Network
layer 4 tcp udp tls
High performance, low latency, TLS offloading
static IP
UDP, IP address as target.
Connection timeout cant be defined
Assign elastic IPs for each node in an AZ
Classic
Layer 4 or layer 7
Old/deprecated
Connection Timeout can be defined
S3
Object storage service: Key Version ID Value Metadata Subresources Access Control Information
Name has to be unique
Region specific
EC2 Access to S3
EC2 Assume Role
Instance setting - attach role - created in previous step.
Auto Scaling for Load Balancer
CloudWatch monitors the parameters
e.g. CPU > 80% > Scaling group
- Create Launch configuration group (Can’t be modified)
- Create Auto Scaling group
i. keep group at initial size OR
ii. Scaling policies:
a. ALB request per count
b. Average CPU
c. Average network IN
d. Average network OUT - Scheduled action to increase instances.
The cooldown period is a configurable setting for your Auto Scaling group that helps to ensure that it doesn’t launch or terminate additional instances before the previous scaling activity takes effect so this would help. After the Auto Scaling group dynamically scales using a simple scaling policy, it waits for the cooldown period to complete before resuming scaling activities.
Scaling policies for Load Balancer
- Target Tracking Scaling
- Simple scaling policy
- Step scaling policy
i. create an alarm
ii. specify action when alarm is triggered
Note: Step scaling the policy can continue to respond to additional alarms, even while a scaling activity or health check replacement is in progress.
Launch Templates and Launch Configuration
LC can’t be modified once created.
To change, create a copy and modify it.
LT is versioned. To edit, a new version of template can be created.
Auto Scaling health checks
Configure ASG to use ELB health checks. Otherwise if ELB cant connect to instance, ASG won’t create new instance.
Cross zone load balancing
Disabled, doesn’t consider if there are more instances in an AZ, both AZs get equal traffic.
If enabled, all instances get same amount of traffic, irrespective of AZ.
ELB for private instances
Configure ELB on Public subnet, then create target group, for corresponding AZ private subnet.
Note, ELB in Public subnet in AZ A cannot serve traffic to Instance in private subnet in AZ B.
Proxy Protocol Header
X-Forwarder Header
PPH - works on layer 4
XFH - works on layer 7
Used for passing the source IP address to the application.
Security groups
Stateful in nature.
All rules are evaluated.
Attached to ENI, so we can define rules with source source and destination as other security groups.
Can only define allow rules, can’t define deny rules.
SG membership can be changed while instance is running.
NACL
Network Access Control List
Attached to Subnet, mandatory to have at least 1 NACL on each Subnet.
List of rules are processed in order and first match is used.
Can define allow or deny rule.
Stateless
Doesn’t affect traffic if both instances are in same subnet.
Default NACL has everything allowed. Custom NACL has all deny.
Used to block traffic, i.e. malicious IPs.
VPC Peering Connections
Connecting 2 VPCs to be able to connect over private network.
Doesn’t support transitive connections.
Transit Gateway
Central gateway to direct multiple CIDRs over multipel networks.
VPC Endpoint Services
In destination VPC, we create an Endpoint service, and then on the source VPC we create an Endpoint.
Interface endpoint is an elastic network interface that allows a private IP address in a subnet to connect VPC resources to a number of AWS services, such as CloudFormation, Elastic Load Balancers (ELBs), SNS, and more.
Gateway endpoint is a target for a route in a route table to connect VPC resources to S3 or DynamoDB. Traffic is then routed from instances in a subnet to one of these two services.
Virtual Private Gateway
Customer Gateway
VPC side of VPN connection - Virtual Private Gateway
Customer side of VPN connection - Customer Gateway
Router
Connects all the networking components and are giverned by router table rules.
Egress only Internet Gateway
Stateful gateway to provide egress onlu access for IPV6 traffic
VPC
from /16 to /28
cant be changed once created.
CIDRs can’t overlap.
Subnet
should be within VPC CIDR can't be updated once created. Automatically connects to default route table. Can't have multiple AZ CIDR can't overlap with other subnet. first 4 IP and last IP are reserved. 172.31.10.0/24 - Network 172.31.10.1 - Gateway/Router 172.31.10.2 - DNS server 172.31.10.3 - Future use 172.31.10.255 - Network broadcast
Connecting to VPC
AWS Managed VPN AWS Direct Connect - not encrypted. AWS Direct Connect + VPN - encrypted AWS VPN Cloud Hub Software VPN Transit VPC VPC Peering AWS PrivateLink VPC Endpoints
VPC Sharing
Subnets can be shared with other AWS accounts within same AWS Org.
VPC Flow logs
Capture IP Traffic to and from network. Stored in CloudWatch Logs. 1. VPC 2. Subnet 3. Network Interface
CORPS
5 pillars
Cost Optimization Operation Excellence Reliability Performance Security
Route 53
Possible to have Domain registered in 1 account and have hosted zone in another AWS account.
Domain can be migrated to other providers using support team.
Hosted zone is collection of records.
Public zone
Private hosted zone. - needs enablednshostname and enablednssupport
Alias is R53, no charges, it can point to domain apex records as well. i.e. amazon.com
A record - 169.254.169.254
CNAME - my.amazon.com
Traffic Flow - create routing configurations - for complex scenarios where multiple policies are required.
Resolver - used in hybrid cloud
Route 53 Health Checks
- check instance health by connecting to it.
- Pointing to Endpoints
- Status of other health checks.
- Status of Cloudwatch Alarms.
Endpoints can be IP or domain names.
Route 53 Routing Policies
- Simple - multiple, round robin, no health check
- Failover - active-passive
- Geolocation
- Geoproximity
- Latency
- Multivalue - upto 8 responses.
- Weighted
AWS GLobal Accelerator
Improves availability and performance of applications.
provides statis IP Addresses that act as fixed entry points to application.
Uses Edge locations - they are associated with regional AWS resources/endpoints
AWS Global Accelerator uses the vast, congestion-free AWS global network to route TCP and UDP traffic to a healthy application endpoint in the closest AWS Region to the user.
This means it will intelligently route traffic to the closest point of presence (reducing latency). Seamless failover is ensured as AWS Global Accelerator uses anycast IP address which means the IP does not change when failing over between regions so there are no issues with client caches having incorrect entries that need to expire.
S3
Key Based object store
Files - 0 - 5TB
Single PUT max size 5GB
Object >100MB can use multipart.
Follows Read after Write consistency for PUT for new objects.
Eventual consistency for overwrite PUT and DELETE
= Key, Value, VersionID, Metadata, ACL.
100 buckets per account
Unlimited objects per bucket
Region specific
Nested bucket is not possible
S3 Features
Transfer Accelration Requester Pays Tags Events Static Web Hosting Bit Torrent
S3 Sub-resources
- LifeCycle
- Website
- Versioning
- ACL
- Bucket Policies
- Cross Origin Resource Sharing (CORS)
- Logging
S3 Storage Classes
1. Standard 2, Intelligent Tiering 3. Standard IA 4. One Zone IA 5. GLacier 6. Deep Archive
CloudFront:
- what components?
- what are the origins?
- types?
- geo-restriction uses.
Distributes content with low latency and high data transfer speeds
Global service with ingress to upload objects and egress to distribute content
= Regional Edge Cache + Edge Locations
Origin - S3, EC2, ELB, R53
- Web Distribution - http/s; add, update, delete objects
- RTMP - streaming
CloudFront geo restriction feature to do one of the following:
• Allow your users to access your content only if they’re in one of the countries on a whitelist of approved countries.
• Prevent your users from accessing your content if they’re in one of the countries on a blacklist of banned countries.
OAI Origin Access Identity
Used to restrict access to content on S3. - only CloudFront can connect to S3.
Cloud Front costs are?
1. Data OUT to Internet 2, Data OUT to origin 3. Number of HTTP/S requests. 4. Invalidation requests 5. Dedicated IP
EBS vs Instance Store
Elastic Block Store
Doesnt need to be attached to EC2.
Cannot attach to multiple instances.
Replicated across multiple locations within AZ.
EC2 and EBS have to be in same AZ
Note:
Instance Store - non-persistent block storage
Can be root or additional volume.
Located physically on the host running the instance.
Can be used for buffers and caches.
Instance-backed EC2 instances can’t be stopped. - Data will be lost.
EBS types
Max IOPS per instance is 80,000
io1 high performance SSD 4GB to 16TB 64,000 Max IOPS 1000 MB/s Max t/p
gp2 general purpose SSD 1GB to 16TB 16,000 Max IOPS 250 MB/s Max t/p
st1 troughput optimized HDD 500GB to 16TB 500 Max IOPS 500 MB/s Max t/p
sc1 cold HDD 500GB to 16TB 250 Max IOPS 250 MB/s Max t/p
EBS vs snapshots
EBS stored on multiple disks in an AZ while snapshots are stored in S3.
Snapshots are easier to store.
Snapshots do not provide granular backup solution.
EBS are AZ specific, while snapshots are region specific.
How to copy AMI to another account
On AMI configuration enter the Account number that is allowed.
Then on the KMS key policy permit the the Account number to grant/read permissions.
Then on the destination account, we will see the AMI.
EFS - Elastic File System
File storage - this is not block storage.
Its fully managed service.
NFS v4.1 protocol
Pay for what is used.
Multi AZ, and the mount-points can be in 1 or many AZs
Using DirectConnect or VPN, can be mounted from on-premises syste,s.
EFS File Sync Agent - existing file systems can be synced with EFS
EFS is elastic - grows ans shrinks as data is added/removed.
Multiple EC2 can connect to 1 EFS
Option - general purpose or Max IO
Provides strong consistency and file locking
Data stored across multiple AZ
Data at rest and transit - TLS 1.2
EFS Access control
IAM is uised for administration of EFS itself.
For files and directories - POSIX compliant user and groups are used.
EFS security groups act as firewall.
Amazon FSx
Fully managed 3rd party file systems.
Native compatibility of third part file systems = windows based storage, high performance computing, machine learning, electronic design automation.
Types:
- Windows File Server
- Lustre for compute intensive workloads.
Amazon FSX Windows File Server
Built on windows server, with native features used by MIcrosoft. - SMB, NTFS, AD integration
Uses SSD
Applications: Home directories, media workflows
Supports, ACL, shadow copies, user quotas
NTFS can be accesses from multiple instances using SMB protocol.
Support access via AWS Direct Connect/AWS VPN
Encryption at Rest and in-transit
ISO, PCI-DSS, SOC, HIPAA
Data is replicated within an AZ.
Optional multi AZ - active standby setup.
Supports Distributed File System Replication (DFSR) in both Single-AZ.
Amazon FSX for Lustre
Data is stored on s3 long term
Data is presented via fast scalable file system interface.
Accesible over Direct Connect and VPN
Storage Gateway - File Gateway
Virtual on-premises file server - store and retrieve files as objects in S3
OnPremises and EC2 can use this file storage.
Flat files directly stored in S3
SMB(v2-v3 Windowws) or NFS(v3-v4 Linux) based.
Max file size 5 TB
Note: The file gateway should be used to replace the NFS file systems as it uses NFS.
Storage Gateway - Volume Gateway
Block based volumes. iSCSI based Cached Volume - data stored in S3, cache is on-site - 32TB per volume max - 1PB per gateway max Stored Volume - data is stored on-site - backup async s3 - 16TB per volume max - 512TB per gateway max Tape Gateway
Note: The AWS Storage Gateway volume gateway should be used to replace the block-based storage systems as it is mounted over iSCSI
Elastic Container Service
Scalable, high performance - container management service - supports only Docker Container.
No additional charge for ECS.
AWS specific platform - different from EKS
Types
- Fargate - serverless
- EC2 - responsibility of customer
ECS Tasks:
- task definition is reqd to run containers on ECS
- task definition - text file in JSON format - max 10 containers.
ECS Cluster:
- logical grouping of containers.
- service provide auto scaling.
- region specific
ECS Container Agent
- required on EC2 instance
- ECS optimized AMIs are present.
ECS Auto Scaling
- target tracking and step scaling policies
Cluster Auto Scaling
- Capacity Provider
- Managed Scaling
- Managed instance termination protection
Lambda
Serverless code execution
Triggerred by events
Need to define the memory
CPU power is proportional to memory
Can access other services.
64MB to 3008 MB in 128 MB increment.
Functions larger than 1536MB get multiple CPU threads
Max timeout of Lambda 15mins-900secs.
Default timeout 3secs.
Event sources maintain mapping configuration - source defines which lambda to call.
While DynamoDB and Kinesis - need lambda to poll them.
Manage and deploy serverless applications using AWS Serverless Application Model. SAM
Amazon API Gateway
Collection of resources and methods integrated with backend HTTP endpoints/Lambda functions. Uses CloudFront Network Manages - traffic management - authorisation - access control - monitoring - api version management. Features - Metering - Security - Resiliency - Operations monitoring - Lifecycle Management
RDS
relational DB Online Transactional Processing OLTP For structured, relational data store. Automated backup Push button scaling, replication and redundancy Types: 1. Aurora 2. MySQL 3. Oracle 4. SQL Server. 5. Postgres
MultiAZ for synchronous replication - automatic failover.
Read replicas used for heavy read workload - async replication
Encryption at rest. - results in snapshot, backups, instance storage, read replicas being encrypted.
Existing DB can’t be encrypted. But using a snapshot a DB can be created which can be encrypted during creation process.
Read Replica on a different region will have to be encrypted using the key in that region.
Upto EBS volume size backup is free.
RDS can only be scaled up for compute and storage.
Scaling compute can cause downtime.
Read Relicas in different region can be promotoed to DB in case of disaster.
Automated backups need to be enabled for Read Replicas to be enabled.
RR available for MySQLm Postgres, Maria, Oracle, Aurora
In case of failover in MultiAZ, RR also take the new connection and connect to new primary.
Max 5 RR.
AWS Database Migration Service
Migrate databse to AWS quickly and securely.
Schema Conversion Tool SCT
- homogenous (mysql to mysql)
- heterogenous (mysql to oracle)
Amazone Aurora
Fully managed.
Scales in 10GB increments - 32 CPU 244GB
2 copies of data maintained in 3 AZs
Can handle loss of 2 copies of data w/o affecting write
Can handle loss of 3 copies w/o affecting read
Amazon Aurora Replicas and Replication
- Aurora Replica (15) in-region
- MySQL Replica (5) cross-region
- this replica can again be replicated to local aurora replicas.
Global Database - provides best replication performance.
Traditional binlog replication is also available.
- this replica can again be replicated to local aurora replicas.
Secondary region can be promoted to full read/write quickly.
Amazon Aurora Multi-Master
only MySQL-compatible.
Used for scaling out write performance with multiple AZ.
Have read after write consistency.
Amazon Aurora Serverless
On-Demand, Autp-Scaling
MySQL and PostgreSQL compatible editions.
Automatically starts up, shuts down, and scales up or down.
Only for DB storage and DB capacity, IO while its active
Amazon DynamoDB
NoSQL DB service
MultiAZ and cross region replication
Push button scaling without incurring downtime.
Eventually consistenct, can be configured to stangly consistent.
Can use DynamoDB Transactions for ACID.
Synchronous Replication to 3 AZs
Is Schema-less
Read model:
- Eventually consistent (Max read throughput)
- Strongly consistent (reflects all writes that received a successful response.)
Auto-Scaling - Dynamically adjust provisioned throughput capacity in response to actual traffic patterns.
Pricing:
- On-demand - data reads and writes
- Provisioned - Specify reads and writes and configure AutoScaling separately.
Amazon DynamoDB Streams and DAX
Keeps a list of item level changes in 24 hours.
Can trigger Lambda based on events.
DAX - fully managed HA, in-memory cache for DynamoDB - 10x performance.
Millions of requests per second.
Code changes not required to use DAX - cache invalidation, data population, cluster management.
Amazon DynamoDB Global Tables
Multi-master DB and replication.
DDB takes care of all task of creating identical tables in different regions and propagate data.
Best for massively scaled applications and globally dispersed users.
Amazon ElastiCache
Redis and Memcache. For OLAP (Online Analytics Processing) Memcache for simple cache. - large node multiple core and threads - scales by adding removing nodes. - ideal front-end for DBs - doesn't support multiAZ no snapshots - no persistence Redis for advanced features - no multithreading - scaling through shards - multi AZ using read replicas in same region. - encryption - HIPAA compliance - clustering - complex data types - HA - Pub/Sub capability - Geospacial Indexing - Backup and restore. - persistence
Usage:
- Web session store
- Database caching
- Leaderboard
- Streaming data dashboards.
Amazon Redshift
Analyzing data using SQL and BI tools. Fully managed data warehouse. Clustered Peta byte scale data warehouse Online Analytics Processing (OLAP) type of DB Parallel processing and columnar data stored. Three copies of data: - the original - replica on compute nodes - backup on s3 (continuous backup to s3)
Runs on single AZ, but snapshot can be restored to another AZ,
Amazon Kinesis
Collect, process, analyze real-time, streaming data.
Has shards, each shard can process 1000 records/sec
Default limit 500 shards.
Record=Parition key, sequence num, data blob
Transient data store default retention 24 hours, max 7 days
Amazon Kinesis Data Streams
Enables build custom app to process streaming data
Real-time processins of streaming big data
For rapidly moving data off data producers
Stores data for later processing by applications.
Scaled using shards.
(FireHose delivers data directly to AWS services.)
Producers
- Streams API
- Producer Library (KPL)
- Kinesis Agent
Consumers - Amazon Kinesis Stream Applications
Amazon Kinesis Data Firehose
Captures, transforms, and loads streaming data
Enables real-time analytics with existing business intelligence tools.
Data streams can be source to Firehose.
Synchronously replicates data across 3 AZs as it is trnsported to detinations.
Destination:
- S3
- Redshift
- Elasticsearch
- Splunk
No Shards here - fully managed
Amazon Kinesis Data Analytics
Process, Analyze real-time, streaming data. Use SQL to process Kinesis Data streams. Use cases - Generate time series analytics - Feed real time dashboards - Real time alerts and notifications
Can ingest from Kinesis Stream/Firehose
Amazon EMR
Uses Hadoop Framework running on EC2 and S3 Processes huge amounts of data Supports Apache Spark, HBase, Presto, Flink Uses: - log analysis - financial analysis - extract - translate
Cluster is a collection of EC2 instances provisioned by EMR to run Steps
Steps - programmatic task for processing data.
Amazon Athena
Interactice query servie to analyze data on S3
Serverless
Uses managed Data Catalog(AWS Glue) to store information and schemas about databases.
Uses Presto for SQL support
Works with CSV, JSON, ORC, Apache Parquet, Avro
Removes the need to have ETL(Extract Transform Load) jobs to prepare data for analysis.
Amazon Glue
ETL service that automates time consuming steps
Automatically discovers and profiles data via Glue Data Catalog
Works with data lakes(S3), data warehouses (Redshift), data stores (RDS)
Amazon SNS:
- what are subscribers?
- what are the transport protocols?
- what are the targets?
Simple Notification Service Setup Operate and send notification Instantaneous push based delivery Multiple transport protocols Event Notification, monitoring, workflow systems, time sensitive information updates,
Subscribers:
- HTTP
- HTTPS
- JSON
- Email-JSON
- SQS
- Application
- Lambda
Transport Protocols:
- HTTP/HTTPS
- Email/EMail-JSON
- SQS
- SMS
- Lambda
- Platform application endpoint
- Kinesis Data Firehose
Amazon Step Functions
Coordinates components of distributed applications as a series of steps in a visual workflow
Create tasks, sequential steps, parallel steps, and branching paths or timers.
Build and run state machines to execute steps of the application.
Amazon SWF
Simple Workflow Service for easy coordination of work across distributed application components.
Distributed async systems as workflows.
Sequential and Parllel processins.
Best for human enabled workflows liek order fulfilment system
Amazon SQS
Message queues that store messages waiting to be processed.
Reliable, highly scalable, hosted queue
distributed and decoupled.
Is Pull based, and not Push based
Messages can be 256 KB in size.
Kept in queue for 1 minute to 14 dats. default - 4 days
Visibility timeout - it process confirmation doesnt come within this time - the message becomes visible again. MAX is 12 hours
Polling
- Short (default)
- doesnt wait for messages to appear in queue
- receivemessagewaittime = 0
- hight cost as more polls are done.
- Long
- fewer requests - less cost
- SQS waits until message is available in queue before sending a response.
- ReceiveMessageWaitTime = 20sec. (20 secs = MAX)
Queue type:
- Standard
- attempts to preserve order
- scalable
- at least once delivery
- 120,000 in-flight messages per queue
- FIFO
- preserves exact roder
- exaclt once processing.
- 20,000 in-flight messages per queue
- 3000 messages/sec when batching
- 300 messages/sec when not batching
Queue name can be 80 characters.
Messages can be retained for 4 days to 14 days
Max message size is 256 KB
PCI DSS level 1 and HIPAA
IAM policies for controling access.
Amazon MQ
Managed message broker service for Apache Active MQ
CloudFormation
Provision infrastructure
Infrastructure as Code
Logical IDs - reference resource within a templace
Physical IDs - reference to resources created
- Templates - JSON/YAML instructions
- Stacks - entire environment described by template
- Change Sets - summary of proposed changes.
Beanstalk
Deploy applications on EC2
PaaS
Devs wanting to just upload the code, and Beanstalk takes care of all the other tasks.
AWS CloudWatch
Monitoring service for AWS Cloud resources
Collect and track metrics, collect and monitor log files and set alarms
Access Control through IAM
CloudWatch logs -
monitor, store and access logs.
Source - Ec2, CloudTrail, Route 53, etc.
Used for real-time monitoring or long term log retentions.
By default, logs retained indefinitely.
Cloudwatch logs metric filters can evaluate Cloudtrail logs for specific terms/values
CloudWatch metrics retention:
- <60secs - 3 hours
- =60secs - 15 days
- =300secs - 63 days
- =3600secs - 455 days
AWS CloudTrail
Creating trail - can be stored on S3 By default trail is stored for 90 days Info: - identity - time of API - source IP - request parameters - response elements
AWS Config
Resource inventory, configuration history and configuration change to enable security and governance
AWS Cognito
Lets Users sign-up, sign-in and access control
Provides Authentication, AUthorization and user management
Users can sign in directly, with username/password, or through third party as facebook, amazon, google.
Components:
- User Pools - directories for sign-up and sign-in
- Indentity Pools - lets us grant access to users to other aws resources.
Cognito works with external identity providers that support SAML and OpenID connect, social Identity providers.
AWS KMS
Highly available keystorage.
CMK/Customer Master Key can be generated locally or on AWS CloudHSM cluster.
These keys are protected by hardware security modules.
Multi-tenant AWS service
Highly avalable and durable
FIPS 140-2 Level 2
AWS CloudHSM
Regulatory compliance for Data Security by using a Hardware Security Module instance within AWS.
Tamper resistance hardware device.
Single Tenant
Customer managed Durability
FIPS 140-2 Level 3
AWS WAF
Web Application Firewall
Web application firewall service - protects against common exploits.
Tightly integrated to CloudFront - rules run on all AWS edge locations.
A new version of the AWS Web Application Firewall was released in November 2019. With AWS WAF classic you create “IP match conditions”, whereas with AWS WAF (new version) you create “IP set match statements”. Look out for wording on the exam.
The IP match condition / IP set match statement inspects the IP address of a web request’s origin against a set of IP addresses and address ranges. Use this to allow or block web requests based on the IP addresses that the requests originate from.
AWS WAF supports all IPv4 and IPv6 address ranges. An IP set can hold up to 10,000 IP addresses or IP address ranges to check.
AWS Shield
Expanded DDoS attack protection 24/7 support from DDoS response team. Always-on detection - Standard - for all AWS users - Advanced - higher levels of protection.
AWS Single Sign-On SSO
Centrally manage access to multiple AWS accounts
Built in integrations to business applications Salesforce, Box, Office 365
Manage user identities in SSO Identity Store or connect to MS AD
Integrated with AWS Organizations, so users can be granted access to the accounts from AD
AWS Managed Microsoft AD
Fully Managed AWS service
Best choice when users are more than 5000
Runs on Windows Server
Works with SharePoint, Microsoft SQL server and .NET apps
AWS Simple AD
Cheap AD compatible service with common directory features Standalone, fully managed, If users are less than 500. Powered by SAMBA 4 AD compatible server Features: - Manage user accounts - apply group policies - Kerberos-based SSO - Supports joining Linux or Windows.
Note: MFA and Trust Relationshops are nto supported.
AWS AD Connector
Directory gateway for redirecting directory requests to on-premise AD
Eliminates the need for directory synchronization
Connects existing on-premise AD to AWS
Sizes:
- Small for 500 users
- Large for 5000 users.
VPC should be connected to on-premise network via VPN/Direct Connect
Note: Must have an existing AD, supports MFA by existing RADIUS-based MFA infrastructure.
AWS IAM Policy Evaluation Logic
- Identity-based policies - attached to identities
- Resource-based policies - attached to resources
- IAM permissions boundaries - Max permissions that an identtiy-based policy can grant to an IAM entity or role.
- AWS Organizations service control policies (SCPs) - Maximum permissions for an organization or organizational unit
- Session policies - advanced policies that is passes as parameters when temporary session is created programatically.
IAM Instance Profile
Container for IAM role that is used to pass role information to EC2, when it starts.
Can only have 1 role.
Note, a role can be in multiple instance profiles.
AWS IAM Cross-Account Access
- Log into the AWS account A.
- User should have access to assume the role in account B
- Assume role for the destination
- Role in account B should have the trust defined
- Role in account B should have the permissions defined.
VM Import / Export
Migrate virtual machines from on-premises to EC2.
Supports, Windows and Linux VMs, from VMWare ESX, Microsoft Hyper-V, and Citrix Xen
Steps:
- Export VM to OVA/VMDK/VHD/RAW
- Upload to S3
- Import image to EBS-Backed AMI
- Create Instance from AMI
AWS DMS Database Migration Service
Supports homogenous and heterogenous migration.
Data is continuously replicated to reduce downtime
Fully managed migration process
Can be used together with Schema Conversion Tool for converting schemas during heterogenous migrations.
Sources supports: Oracle SQL server, MySQL, MariaDB, PostgreSQL, Db2, LUW, SAP, MongoDB, Amazon Aurora
AWS Server Migration Service SMS
Agentless service for migrating on-premises and cloud based VMs to AWS.
Source platform can be VMWare , HyperV or Azure.
SMS connector is installed on source
Server volumes are replicated and saved as AMIs which can be launched as EC2.
AWS Snowball and Snowmobile
When large amounts of data to be sent to AWS.
Uses secure storage for physical transportation
Snowball client is installed on local computer and is used to identify, compress, encrypt and transfer data.
256 bit encryption and tamper resistant enclosures.
Snoball - 80TB/50TB Petabyte scale
Snowball Edge - 100TB - onboard storage and compute
Snowmobile - exabyte scale - 100 PB per snowmobile
AWS Datasync
Datasync software agent connects to on-premises NA storage systems.
Uses NFS or SMB protocols.
Synchronizes data in AWS with scheduled transfers (TLS encrypted)
Destination can be S3, EFS, FSX WFS
Imporves performance of data transfers by 10x
Permissions and metadata are preserved.
Pay per GB transferred.
What are Spot Block Instances?
Max duration 6 hours.
Once started, they will run uninterrupted for selected duration.
What is S3 pre-signed URL?
All S3 objects by default are private. Only the object owner has permission to access these objects. However, the object owner can optionally share objects with others by creating a presigned URL, using their own security credentials, to grant time-limited permission to download the objects.
When you create a presigned URL for your object, you must provide your security credentials, specify a bucket name, an object key, specify the HTTP method (GET to download the object) and expiration date and time. The presigned URLs are valid only for the specified duration.
Anyone who receives the presigned URL can then access the object. For example, if you have a video in your bucket and both the bucket and the object are private, you can share the video with others by generating a presigned URL.
Explain Envelope Encryption.
Envelope encryption offers several benefits:
• Protecting data keys
When you encrypt a data key, you don’t have to worry about storing the encrypted data key, because the data key is inherently protected by encryption. You can safely store the encrypted data key alongside the encrypted data.
• Encrypting the same data under multiple master keys
Encryption operations can be time consuming, particularly when the data being encrypted are large objects. Instead of re-encrypting raw data multiple times with different keys, you can re-encrypt only the data keys that protect the raw data.
• Combining the strengths of multiple algorithms
In general, symmetric key algorithms are faster and produce smaller ciphertexts than public key algorithms. But public key algorithms provide inherent separation of roles and easier key management. Envelope encryption lets you combine the strengths of each strategy.
How to provide permissions to ECS task to DynamoDB?
To specify permissions for a specific task on Amazon ECS you should use IAM Roles for Tasks. The permissions policy can be applied to tasks when creating the task definition, or by using an IAM task role override using the AWS CLI or SDKs. The taskRoleArn parameter is used to specify the policy.
What is EFA?
Elastic Fabric Adapter
An Elastic Fabric Adapter is an AWS Elastic Network Adapter (ENA) with added capabilities. The EFA lets you apply the scale, flexibility, and elasticity of the AWS Cloud to tightly-coupled HPC apps. It is ideal for tightly coupled app as it uses the Message Passing Interface (MPI).
