Fundamentals: IAM EC Flashcards
region
cluster of data centres.
Most AWS services are region-skoped
you can use a service in a specific region. If you use the same service in another region, you will not have your data being replicated or synchronized. So you will have to recreate your data in a different region
availability zones
each region can have many of them, 3, 2 or 6 usually. 6 is max
ap-southeast-2a
ap-southeast-2b
each is one or more discrete data centres with redundant power, networking and connectivity
they are separate from each other and isolated from disasters, but they are connected with high-bandwidth, ultra-low latency networking
Anything that ends with a letter is an AZ
EC2 vs IAM
EC2 regional service
IAM global service, no selection of region required
region table
shows which services are available in which region
IAM
Identity and access management: users, groups, roles
root account
account with which you create your AWS account, should never be used afterwards or shared, has the most power
only for initial setup
IAM policies
written in JSON, define what - users - groups - roles can or cannot do
a user can belong to one or more groups
users inherit permissions
from groups. We can add permissions to both users and groups, but it is better to assign them only to groups, these will be inherited by users anyway
roles are only
for internal usage within AWS resources and services. Roles are given to machines whereas Users are for physical persons
IAM has a global view meaning
users, groups and roles are defined across regions
MFA
multifactor authentication for root account and users, recommended
managed policies
pre-defined policies created by IAM
least privilege principle
best practice to give minimal permissions for users to perform their jobs
IAM federation
big enterprises usually integrate their own repo of users (Active Directory) with IAM, so users can log into AWS with their company credentials
uses SAML standard
one IAM user
per physical person
one IAM role
per application
EC2 capabilities
- launch virtual machines in the cloud
- store data on virtual drives (EBS)
- distribute load across machines (ELB = load balancing)
- scaling the services using an auto-scaling group (ASG)
AMI
Amazon machine image
Subnet in EC2 config says
if you’ve created your own VPC, you can define your own IP address range and create subnets, create subnets, configure route tables, and configure network gateways.
So if you’ve done this - in the dropdown you’ll see a list of configured subnets to choose from (subnet = range of IP addresses in your VPC that can be used to isolate different EC2 resources from each other or from the internet. Network gateway has IP through which the subnet is accessible from the outside world)
If you haven’t done that, Amazon has assigned a default VPC to you, and in this list you can select an AZ (availability zone) you are going to have VM in
Each subnet resides in only one AZ
VPC
virtual private cloud
if you’ve created your own VPC, you can define your own IP address range and create subnets, create subnets, configure route tables, and configure network gateways.
Storage of EC2 machine is
EBS volume
To ssh into the launched machine
- you need to configure security group defined with type ssh before launching the machine
- you need to define a key pair just before launching to use it to access
EC2 instance connect
allows you to connect to your EC2 instance directly from browser in any OS
security groups
- the fundamental of network security on AWS
- control how traffic is allowed into or out of our EC2 machines
- acting as Firewall on EC2 instances
security groups has these 3 things
inboud rules
outbound rules
tags (just key-value pairs)
here you can for ex., define inbound rule to allow connection on port 22, then you can ssh into your instance
you can find the link to the security groups associated with your EC2 instance
if you select the instance in the list and got tab Description below. Also available on the left panel under Network and Security
security groups regulate
- access to ports
- authorized IP ranges - IP4 and IP6
- control inbound traffic (from other to the instance)
- control outbound (from instance to the other)
relationship between security groups and instances
is many to many
one SG can be attached to many instances
one instance can have multiple SGs
SGs are outside the EC2 instance, not running on it, it’s an outside firewall
security groups are locked to down
to the combination of region and VPC
so if you switch to another region or another VPC - you have to recreate your SGs
it’s good to maintain one SG for ssh access
by default all inbound traffic is
blocked
by default all outbound traffic is
authorized
Communication between EC2 instances and security groups
security groups can reference each other
if we want EC2 instance2 to be able to connect to EC2 instance1 in the same or different VPC, we can
- create SG2 and attach to instance2
- create a security group SG1 with inbound rule that references security group SG2 (in field Source we put id of SG2 instead of IP)
- add SG1 to instance1
the same works for outbound
The peer VPC can be a VPC in your account, or a VPC in another AWS account, but not in another region
IP4 and IP6
AWS supports both
public IP
- the machine can be identified on the internet WWW
- must be unique across the whole web. Two machines cannot have the same
- can be geo located easily
in AWS you can see public IP of instance on Description tab
private IP
- the machine can only be identified on the private network
- the IP must be unique within the private network
- two different private networks can have the same IPs
- Machines connect to the internet using an internet gateway (proxy)
- only a specified range of IPs can be used as private IP
in AWS you can see private IP of instance on Description tab. It doesn’t change when you start and stop the instance
elastic IP
- when you stop and start your EC2 instance, it can change its public IP
- if you need to have a fixed public IP, you need an elastip IP
- an elastic IP is a public IP4v IP that you own as long as you dont delete it
- you can have 5 elastic IPs on your account (you can ask for more, but uncommon)
- you can mask a failure of an instance or software by rapidly remapping the address to another instance in your account (uncommon)
instead of using elastic IPs
- avoid using elastic IPs
- reflect poor architectural decisions
- instead use a random public IP and register a DNS name to it
- use Load Balancer and no public IP at all (best!)
Under Network and Security –> Elastic IPs
to create elastic IP
- create elastic IP
go to Network and Security –> Elastic IPs
- we can use one from Amazon pool
- you can use your own if you have one on your AWS account
- customer owned pool - associate it with your EC2 instance
Actions –> Associate elastic IP address
EC2 user data
is used to automate tasks such as:
- installing updates
- installing software
- downloading common files from the internet
run with root permissions, any command will have sudo permissions
the script is only run once at the instance first start
Create EC2 user data
when you create new EC2 instance, in Step3 (Configuring instance details) –> Advanced details –> User data
always add as first line #!/bin/bash
You pay for an EC2 instance compute component
only when it’s in running state
not in stopped state
You are getting a permission error exception when trying to SSH into your Linux Instance
the key is missing permissions chmod 0400
You are getting a network timeout when trying to SSH into your EC2 instance
your security group is misconfigured
Security groups can reference all of the following except:
DNS name
allowed: IP, CIDR, Security Group
Types of EC2 instances
- on-demand instances:
- reserved
- convertible reserved
- schedule reserved
- spot instances
- dedicated instances
- dedicated hosts
on-demand instances
- short workload
- predictable price = pay exactly for what you use, billing per second, after the first minute. You pay for the time between start and stop
- the highest cost, but no upfront payment
- but no long-term commitment
- short-term needs, uninterrupted workloads or
- when you can’t predict how the application will behave
- good for elastic workloads
reserved instances
for minimum 1 year
- long workloads
- up to 75% discount compared to on-demand
- pay upfront for long term commitments
- reserve specific instance type
- recommended for steady state usage applications (database)
convertible reserved instances
long workloads with flexible instances
instead of saying that you want an M4X for one year, say that you want something for one year
a bit more expensive than simple reserved (up to 54% discount)
schedule reserved instances
require for a fraction of day, month or year
for ex., every Thursday between 3 and 6 p.m.
(again - for at least a year)
spot instances
- least reliable
- short workloads resilient to failure (batch jobs or image processing that you can retry or data analysis that you can use with distributed computing)
- but not running a critical job or databases to be up and running for years
- for cheap, can lose instances
- discount of up to 90% compared to on-demand
dedicated instances
- no other customers will share your hardware
- but may share hardware with other instances in the same account
- no control over instance placement: you can only move the instance placement from hardware after doing a start and a stop
dedicated hosts
- book an entire physical server
- control EC2 instance placement
- visibility into the underlying sockets / physical cores of the hardware,
- which is great for licensing purposes - for software that has a complicated licensing model (BYOL) - some licenses will bill you on the number of physical cores in the hardware or number of sockets
- for companies that have strong compliance and regulatory needs so that you don’t share your hardware with anyone else
- reservation is for 3 years
- more expensive
Combination of instance types
reserved for baseline capacity (for ex., web application that you know should run)
and
for anything unpredictable, based on demand or for peaks - mix of on-spot on on-demand instances based on if you can have failure or not on your workload
dedicated hosts and instances: enables the use of dedicated physical servers
yes - yes
dedicated hosts and instances: per instance billing
no - yes
dedicated hosts and instances: per host billing
yes - no
dedicated hosts and instances: visibility of sockets, cores, host id
yes - no
dedicated hosts and instances: affinity between host and instance
yes - no
dedicated hosts and instances: targeted instance placement
yes - no
dedicated hosts and instances: automatic instance placement
yes - yes
dedicated hosts and instances: add capacity using an allocation request
yes - no
how reserve a spot instance
- we define a max spot price that we are wiling to pay
- get instance while its price is below
- the hourly spot price varies based on offer and capacity
- when price goes above, you can choose to stop or terminate the instance with a 2 minutes grace period. If you choose to stop you can restart it later when the price goes down
there is another strategy called Spot Block
Spot Block (spot instances)
block a spot instance for a specified time frame (1-6 hours) without interruptions
how to terminate stop instances
- with spot request you define
- how many instances you want,
- the max price you are wiling to pay,
- the launch specification (AMI)
- from-until (or infinity)
- request type: one-time (once the instances are launched, the request just goes away) or persistent (request willl be valid from-until; if your instance is stopped, request goes back into action and will be revalidated later, i. e. the instance will be restarted when possible)
if you cancel the persistent request, it’s not going to terminate any launched instances
so if you want to terminate the instances, first cancel the request. Otherwise, it will re-launch
spot fleet
a set of spot instances and optionally of on-demand
the fleet will try its best to meet the target capacity with price constraints
- define possible launch types, OS, availability zones
- the fleet will choose the best and most approprate launch pool for you. When it’s reached your budget or the desired capacity, it will stop launching. The lowest price
Strategies to allocate spot instances in the spot fleet
- lowest price: from the pool with the lowest prices (short workloads, cost optimization)
- diversified: distributed across all pools (great for availability and long workloads)
- capacityOptimized: pool with the optimal capacity for the number of instances
R instance type
applications that need a lot of RAM - in-memory caches, in-memory databases
C instance type
applications that need good CPU -apps that do a lot of computations such as big data
M instance type
medium, between RAM and CPU.
web app, general
I instance type
for good I/O (instance storage) - databases, a lot of disk operations
G instance type
come with GPU and are great for video rendering and machine learning
graphic processing unit
T2/T3 instance types
burstable instances can be amazing to handle unexpected traffic and getting the insurance that it will be handled correctly. But if your instance runs low on credit all the time, then you probably shouldn’t use T2/T3 (C or M, non-burstable)
overall, when the instance is running, you get good performance, the CPU has OK performance
And then sometimes maybe you need to process something unexpected (a spike in a load for example), your CPU skyrockets and goes to 100%. During these spikes the CPU can do something called a burst (a boost of power). The CPU is very good during that burst.
But if the machine bursts, it utilizes burst credits. If the credits are all gone, the CPU becomes bad. When your load is over, the CPU is stopped bursting, then you gain back credits over time
So if you over-abuse bursts - you will lose the burst and your capacity
you can see peaks in CloudWatch
T2/T3 unlimited have unlimited burst credit balance. You pay extra money if you go over your credit balance, but you don’t lose performance
if we want to create our own image (AMI)
it’s only for a specific region
and by default private
locked for your account/region
we can make them public and sell in Amazon Marketplace
Can be copied to another region
Advantages of custom AMIs
- pre-installed packages
- faster boot time, because no need for ec2 user data at boot time
- machine comes configured with monitoring/entrerprise software
- security concerns: we need to install some security software
- control of maintenance and update of VMs over time
- set up Active Directory Integration out of the box
- install apps ahead of time (for faster deploys during auto-scale)
- use someone else’s AMI optimized for running an app, DB, you can pay by the hour (rent)
AMIs can be found in
Amazon Marketplace (careful - some may contain malware)
AMI storage
when you create AMI, it takes space and they live in Amazon S3
but we won’t see them in S3 console
S3 is cheap, durable and resilient to
AMI pricing
you get charged for the actual space it takes in S3
how to create our own image (AMI)
you need to launch an instance, install what you need, then right click on intance and Image -> Create Image
All images can be found on the left panel images –> AMIs
If you then right click on AMI, select Launch - the VM will be launched based on this image
Cross account AMI copy
- you can share AMI with another AMI account
- It doesn’t change the ownership of AMI. But if they copy your AMI (in another region), they become the owner of that AMI
- you can prevent copying AMI -
- either not grant EBS snapshot access
- or S3 bucket access
But this does not prevent someone from launching a VM based on your AMI and making their own AMI from it
To copy an AMI that was shared with you from another account,
the owner must grant you read permissions for the storage that backs the AMI:
- either the associated EBS snapshot (for EBS-backed AMI)
- or an associated S3 bucket (for an instance store-backed AMI)
ecnrypted AMI copy
you can’t copy an encrypted AMI shared with you. If the underlying snapshot and the encryption key were shared with you, you can copy the snapshot while re-encrypting it with a key of your own. You can register this copy as new AMI, because the ownership is yours
copy AMI with associated billing product code
you can’t copy AMI with associated billing product code
if you get a Windows AMI or AMI from Amazon Marketplace, they will have their billing product
The only way to copy is to launch your own EC2 instance from it and then create a new AMI from this instance
To share an AMI
right click on AMI and Modify image permissions
you can make it public or private
if private - you can specify the IDs of AWS accounts you want to share with
if I tick the checkbox “Add create volume permissions…”, then the accounts I share with will be allowed to make a copy of AMI
If I don’t - they can still launch instance and make their own AMI, but they can’t copy using the copying utility
this is valid for Marketplace images, you can’t directly copy them
placement groups
control over EC2 instance placement strategy. The strategy is defined by the placement group
we don’t get direct interaction with AWS hardware, but we let AWS know how we would like instances to be placed compared to one another
under Network and security
when you create a placement group, you specify one of the following strategies:
- cluster
- spread
- partition
cluster placement group strategy
clusters instances into a low-latency group in a single AZ
the instances will be grouped together in a low-latency hardware setup within a single AZ
high performance and high risk
can be 10 Gbps bandwidth between instances
all instances on the same rack, i.e. same hardware, same AZ. If the rack fails, all the instances fail at the same time
examples: big data job to be completed very fast
app with requirement of extremely low latency and high network throughput and we are willing to take the risk of failure
spread placement group strategy
your instances are going to be spread across different hardware
restriction: you can only have 7 EC2 instance per placement group per AZ
for critical applications, instance failures must be isolated from one another
minimize failure risk, maximizes availability
partition placement group strategy
you want to spread your instances but across many different partitions, which rely on different sets of racks of hardware within an AZ
so if one partition fails, all of the instances on it fail, but a partition is isolated from another partition’s failure
AWS defines a number of partitions within the group and scatters the group instances across these partitions. Instance gets access to the partition info as metadata
The partitions share a common availability zone, but do not share hardware. In fact, each partition resides within a different rack in the AWS datacenter.
provide hardware redundancy, you can scale to hundreds of EC2 instances per group, is typically used by large distributed and replicated workloads, such as Hadoop, Cassandra, and Kafka, HDFS, HBase.
distributed big data applications
ENI
Elastic Network interfaces
a logical component in a VPC that represents a virtual network card. They give EC2 instances access to the network
accessible under Network and Security –> Network interfaces or on the instance
one instance can have multiple ENIs
ENI can have the following attributes
- primary private IPv4 (eth0)
- one or more secondary IPv4 (eth1)
- an elastic IPv4 per private IPv4
- one public IPv4
- one or more security groups attached to ENI
- MAC address
you can create ENIs independently
from your EC2 instances and attach them on-the-fly or move them from EC2 instances for failover
but bound to a specific AZ
what happens with data when we start or stop our instances
if we stop an instance, its data is left intact until the next start if that’s an EBS volume
if we terminate - any data on our EBS volumes that is root that is also set up to be destroyed alongside our instances is going to be lost
but if it’s an EBS volume attached as a secondary drive and it’s not meant to be destroyed when your instance is being terminated - you will keep the data
On instance start
first start: OS boots, EC2 User data script is run
following starts: OS boots up
then your application starts, caches get warmed up, this takes time
EC2 Hibernate
the in-memory (RAM) state is preserved, all data in RAM preserved
when you restart instance after hibernating it, the instance boot is going to be much faster because the OS has not been stopped, the OS will be up
the state of the RAM is dumped into a file onto the root EBS volume. This volume must be encrypted
Use cases for EC2 Hibernate
- keep long running processing
- you need to save RAM state
- you have services that take a lot of time to initialize
EC2 Hibernate current limitations
- does not support all instance families
- the instance RAM size cannot be bigger than 150 gigabytes
- no support for bare metal instances
- not all AMI
- root volume must be encrypted EBS, not instance store, large enough
- only on-demand and reserved instances
- cannot be hibernated more than 60 days
EC2 Hibernate how to create
- you create an instance as normally, but have to select the type supported by hibernate
- in step 3 in Shutdown behavior select Stop, then you’ll Stop hibernate behavior - tick checkbox
t2.micro
is free tier
You are getting started with AWS and your manager wants things to remain simple yet secure. He wants the management of engineers to be easy, and not re-invent the wheel every time someone joins your company. What will you do?
create multiple IAM users and groups, assign policies to groups
new users will be added to groups
You pay for an EC2 instance compute component only when
it is in running state
You plan on running an open-source MongoDB database year-round on EC2. Which instance launch mode should you choose?
reserved instances
You are launching an EC2 instance in us-east-1. It works well, so you decide to deploy your script in us-west-1 as well. There, the script does not work and fails with “ami not found” error. What’s the problem?
AMI is region locked and the same ID cannot be used across regions
You would like to deploy a database technology and the vendor license bills you based on the physical cores and underlying network socket visibility. Which EC2 launch modes allow you to get visibility into them?
dedicated hosts
You are running a critical workload of three hours per week, on Monday. As a solutions architect, which EC2 Instance Launch Type should you choose to maximize the cost savings while ensuring the application stability?
scheduled reserved instances
