ELB Load Balancing (High Availability and Scalability) Flashcards
scalability
your application can handle greater loads by adapting
two kinds of scalability
vertical
horizontal (=elasticity)
vertical scalability
increasing the size of the instance
if your app runs on t2.micro, scaling means running it on t2.large
vertical scalability use cases
non-distributed systems such as a database
RDS, ElastiCache
horizontal scalability
increase the number of instances / systems for your application
distributed system
common for modern web applications
high availability
usually goes hand in hand with horizontal scaling but not all the time
you run your app in at least 2 data centers (== AZ in AWS) in order to survive a data center loss
can be passive (RDS multi AZ) or active (horizontal scaling)
RDS
Amazon relational database service
is a web service that makes it easier to set up, operate, and scale a relational database in the AWS Cloud.
Multi-AZ
Amazon RDS provides high availability and failover support for DB instances using Multi-AZ deployments.
In a Multi-AZ deployment, Amazon RDS automatically provisions and maintains a synchronous standby replica in a different Availability Zone.
high availability for EC2
run instances for the same application across multi AZ
Autoscaling Group multi AZ, Load Balancer multi AZ
horizontal scalability for EC2
increase number of instances (scale in or scale out)
Autoscaling Group
Load Balancer
Amazon ElastiCache
Amazon ElastiCache is a web service that makes it easy to deploy, operate, and scale an in-memory data store or cache in the cloud. The service improves the performance of web applications by allowing you to retrieve information from fast, managed, in-memory data stores, instead of relying entirely on slower disk-based databases.
load balancers
servers that forward internet traffic to multiple servers (EC2 instances) downstream
available on the left panel Load balancing
why should we use load balancers
- we can spread load across instances and expose a single point of access to your app, DNS.
- we dont need to know about all the EC2 backend instances, we just need to know the single point of access, the host name of your balancer.
- LB will perform regular health checks to your instances so it knows when not to send traffic to your instances
- provide SSL termination (HTTPS) for your website
- enforces stickiness with cookies
- high availability across availability zones
- separate public traffic (user - to LB) from private traffic (LB to EC2 instances)
why AWS load balancer
- it’s a managed LB. AWS guarantees that it will be working, takes care of upgrades, maintenance, high availability
- although costs less to set up your own, a lot more effort on my end
- integrated with many AWS offerings and services
health checks
enables LB to know if the instances it sends traffic to are available to reply to requests in a good way
done on a port and a route (/health is common). If the response is not 200 OK, the instance is deemed unhealthy
you can configure how often
three types of load balancers
- classic LB (v1 old generation) - 2009
- application LB (v2 new generation) - 2016
- network LB (v2 new generation) - 2017
it is recommended to use the 2 last of the new generation, provide more features
classic load balancer
supports TCP, HTTP and HTPPS traffic
health checks are TCP or HTTP based
fixed hostname
application load balancer
- supports HTTP, HTPPS and WebSocket traffic
- only layer 7 LB
- fixed host name
-
network load balancer
supports TCP, TLS (secure TCP), UDP
internal (private) LB
private within your account, you can’t access it from the public web
external (public) LB
will allow your users to access for ex., your website
load balancer security groups
- create security group on LB with inbound rules allowing HTTP (port 80) traffic and HTTPS (port 443) from any users from any IPs
- create security group in EC2 instances that allows incoming traffic only from LB. In the security group we can reference the security group we created on LB in step1
load balancer and scaling
LB can scale but not instantaneously - contact AWS for a warm up
load balancer troubleshooting
- 4xx are client induced errors
- 5xx errors are app induced errors
- 503 means at capacity or no registered target
- if LB can’t connect to your app, check security groups
load balancer monitoring
- ELB access logs will log all requests
2. CloudWatch Metrics will give you aggregate statistics
TCP, HTTP´, HTTPS layers
TCP layer 4
others layer 7
load balancer stickiness
it is possible to implement stickiness so that the same client is always redirected to the same instance behind LB
works for classic and application LB
You are running a website with a load balancer and 10 EC2 instances. Your users are complaining about the fact that your website always asks them to re-authenticate when they switch pages. You are puzzled, because it’s working just fine on your machine and in the dev environment with 1 server.
load balancer stickiness use case
we want to make sure that user doesn’T lose his session data so keep on talking to the same EC2 instance
the cookie used for stickiness has an expiration date you control
enabling stickiness downside
may bring imbalance to the load over the backend EC2 instances
load balancer stickiness configuration
for classic LB it is configured in LB properties
for application LB - at the target group level (left panel –> Load Balancing –> Target Groups), checkbox
you can configure for how long
takes a little bit of time to kick in
cross-zone load balancing
each LB instance distributes evenly across all registered instances in all AZ
(one LB can be deployed in multiple AZ –> hence multiple LB instances)
if cross-zone load balancing disabled, each instance will distribute evenly only in its zone
cross-zone load balancing and classic LB
disabled by default but you will not be charged if you enable it
the setting is on tab Description
cross-zone load balancing and application LB
always on and can’t be disabled but you will not be charged for data crossing AZones
there is no setting to change it
cross-zone load balancing and network LB
disabled by default but you will be charged if you enable it
the setting is on tab Description
SSL certificate allows traffic
between your clients and your load balancer to be encrypted in-transit (in-flight encryption)
SSL
secure sockets layer
used to encrypt connections
TLS
Transport Layer Security - newer version of SSL
TLS certificates are mainly used nowadays, but people still say SSL
have expiration date and have to be renewed
public SSL certificates are issued by
Certificate Authorities
Load balancers and SSL certificates
- users connect to LB over HTTPS, so using SSL certificates, over public internet
- internally LB does SSL certificate termination
- in the backend it can talk to your EC2 instance using HTTP, not encrypted but the traffic goes over your VPC, which is private network and somewhat secure
- LB will load X509 certificate (SSL (TLS) server certificate)
you can manage certificates using
ACM
AWS ceritficate manager
you can upload your own certificates to ACM
when you set up HTTPS listener on LB
you must
- specifiy a default certificate
you can
- add a list of optional certs to support multiple domains
- clients can you use SNI (server name indication) to specify the host name they reach
- specify a security policy to support older versions of SSL / TLS (legacy clients)
SNI and LB
With SNI support we’re making it easy to use more than one certificate with the same ALB.
In order to handle different domains with the same load balancer.
It’s always been possible to use wildcard and subject-alternate-name (SAN) certificates with ALB, but these come with limitations.
Wildcard certificates only work for related subdomains that match a simple pattern and while SAN certificates can support many different domains, the same certificate authority has to authenticate each one. That means you have reauthenticate and reprovision your certificate everytime you add a new domain.
classic LB and SSL
supports only 1 SSL certificate
must use multiple CLB for multiple hostname with multiple SSL certificates
application LB and SSL
supports multiple listeners with multiple SSL certificates
uses server name indication SNI to make it work
network LB and SSL
supports multiple listeners with multiple SSL certificates
uses server name indication SNI to make it work
add HTTPS listener
select LB and then tab Listeners –> when adding a listener you can add certificate(s)
for classic LB - only one
for application and network LB you can create rules using SNIs - in which case which certificate can be used
application LB allows
- load balancing to multiple HTTP applications across machines (target groups)
- load balancing to multiple applications on the same EC2 instance (ex: using containers und ECS)
- routing to different target groups
routing to different target groups
- based on path in URL (example.com/users or example.com/posts)
- based on hostname in URL (one.example.com and other.example.com)
- based on QueryString and Headers
example. com/users?id=123 - based on source IP
in LB configuration you configure rules - to which target group to forward on condition
application LB are great
when you have micro services and container-based application (ex.: Docker and Amazon ECS)
have a port mapping feature which allows to redirect to a dynamic port on the ECS instance
application vs classical LB
if we wanted to have multiple applications behind LB,
with classic LB we would have to have multiple classical LB, one per application
but with app LB - one in front of many apps
Application LB target groups behind it
- EC2 instances (can be managed by an autoscaling group) - HTTP
- ECS tasks (managed by ECS itself) - HTTP
- lambda functions - HTTP request is translated into a JSON event
- private IP addresses
ALB can route to multiple target groups
Health checks will be done at the target group level
application LB communication client - LB- instance
application servers dont see the IP of the client directly
the true IP is inserted into the header X-Forwarded-For, port - X-Forwarded-Porto and protocol - X-Forwarded-Proto
When client talks to the LB, it performs connection termination. When LB talks to EC2 instance, it uses LoadBalancer IP (private IP)
so if EC2 instance needs to know the IP of the client, it needs to look into the extra headers of the HTTP request
SNI only works for
application and network LBs
ELB connection draining naming
classic LB - Connection draining
Target Group: deregistration delay (ALB, NLB)
connection draining
time to complete in-flight requests while the instance is de-registering or unhealthy
allows the instance to shut down anything it was doing before being de-registered
ELB will stop sending new requests to the instance while it’s being deregistered
connection draining workflow
EC2 instance is being terminated or unhealthy so it’s gonna go into draining mode
during this mode the existing connections will be waiting for the duration of the connection draining period to be completed
any new connection to ELB will be redirected to another EC2 instance
duration of the connection draining period
300 sec by default
you can change between 1 sec and 1 hour or disable completely
duration of the connection draining period strategy
if your app has very short requests, like a web app, for ex. 1-5 seconds, then you should set to 10-20 seconds max, because you don’t expect any request to last longer than 20 sec
if your EC2 instances are very slow to respond, have a lot of data processing to do, then you want to set it higher to give a chance to those request that are in-flight to be completed
if you disable completely: so in case a connection is dropped while your EC2 instance is being killed, users will receive error and will have to repeat the request, which will be redirected to another instance
SSL based communication
- client sends request to web server secured with SSL
- webserver sends its certificate to the client, so that the client can verify its identity. The certificate contains the server’s public key
- the client generates a random session key and encrypts it with the server’s public key. And sends to the server.
- Only the server can decrypt the session key. After that the communication is encrypted symmetrically
SNI
server name indication, extention of TLS
allows multiple secure (HTTPS) websites (or any other service over TLS) to be served by the same IP address without requiring all those sites to use the same certificate
why is SNI required
Since TLS operates at the transport layer, below HTTP, it doesn’t see the hostname requested by a client. And can’t choose certificate based on the hostname
how does SNI work
SNI works by having the client tell the server “This is the domain I expect to get a certificate for” when it first connects.
The server can then choose the correct certificate to respond to the client.
In order to provide any of the server names, clients MAY include an extension of type “server_name” in the (extended) client hello.
All modern web browsers and a large majority of other clients support SNI.
host name in SSL/TLS certificate
The Common Name (AKA CN) represents the server name protected by the SSL certificate. The certificate is valid only if the request hostname matches the certificate common name. Most web browsers display a warning message when connecting to an address that does not match the common name in the certificate.
In the case of a single-name certificate,
the common name consists of a single host name (e.g. example.com, www.example.com)
in the case of a wildcard certificate
the common name consists of a wildcard name (e.g. *.example.com).
The common name is technically represented by
the commonName field in the X.509 certificate specification.
what kind of IP or name does an LB provide
static DNS name we can use in our application
AWS wants your load balancer to be accessible using a static endpoint, even if the underlying infrastructure that AWS manages changes
use case for network load balancer
You are designing a high performance application that will require millions of connections to be handled, as well as low latency.
NLB provide the highest performance if your application needs it
You are running at desired capacity of 3 and the maximum capacity of 3. You have alarms set at 60% CPU to scale out your application. Your application is now running at 80% capacity. What will happen?
Nothing
The capacity of your ASG cannot go over the maximum capacity you have allocated during scale out events
ALB with elastic IP
is not technically feasible
You would like to expose a fixed static IP to your end-users for compliance purposes, so they can write firewall rules that will be stable and approved by regulators. Which Load Balancer should you use?
Network Load Balancers expose a public static IP, whereas an Application or Classic Load Balancer exposes a static DNS (URL)
Load Balancers provide what kind of address
static DNS name that we can use in our app
not static IP4, not static IP6
You are running a website with a load balancer and 10 EC2 instances. Your users are complaining about the fact that your website always asks them to re-authenticate when they switch pages. You are puzzled, because it’s working just fine on your machine and in the dev environment with 1 server. What could be the reason?
LB doesn’t having stickiness enabled
Your application is using an Application Load Balancer. It turns out your application only sees traffic coming from private IP which are in fact your load balancer’s. What should you do to find the true IP of the clients connected to your website?
Look into X-Forwarded-For header in the backend
You quickly created an ELB and it turns out your users are complaining about the fact that sometimes, the servers just don’t work. You realise that indeed, your servers do crash from time to time. How to protect your users from seeing these crashes?
Enable health checks
You are designing a high performance application that will require millions of connections to be handled, as well as low latency. The best Load Balancer for this is
Network LB
The application load balancer can redirect to different target groups based on all these except…
Hostname and request path but not client IP
You are running at desired capacity of 3 and the maximum capacity of 3. You have alarms set at 60% CPU to scale out your application. Your application is now running at 80% capacity. What will happen?
Nothing
I have an ASG and an ALB, and I setup my ASG to get health status of instances thanks to my ALB. One instance has just been reported unhealthy. What will happen?
ASG will terminate this instance
Your boss wants to scale your ASG based on the number of requests per minute your application makes to your database.
create CloudWatch custom metric and build an alarm on this to scale your ASG
You would like to expose a fixed static IP to your end-users for compliance purposes, so they can write firewall rules that will be stable and approved by regulators. Which Load Balancer should you use?
Network Load Balancers expose a public static IP, whereas an Application or Classic Load Balancer exposes a static DNS (URL)
Your application load balancer is hosting 3 target groups with hostnames being users.example.com, api.external.example.com, and checkout.example.com. You would like to expose HTTPS traffic for each of these hostnames. How do you configure your ALB SSL certificates to make this work?
SNI (Server Name Indication) is a feature allowing you to expose multiple SSL certs if the client supports it. Read more here: https://aws.amazon.com/blogs/aws/new-application-load-balancer-sni/
The Application Load Balancers target groups can be all of these EXCEPT…
Network LB
You are running an application in 3 AZ, with an Auto Scaling Group and a Classic Load Balancer. It seems that the traffic is not evenly distributed amongst all the backend EC2 instances, with some AZ being overloaded. Which feature should help distribute the traffic across all the available EC2 instances?
Cross Zone Load Balancing
Your Application Load Balancer (ALB) currently is routing to two target groups, each of them is routed to based on hostname rules. You have been tasked with enabling HTTPS traffic for each hostname and have loaded the certificates onto the ALB. Which ALB feature will help it choose the right certificate for your clients?
SNI
An application is deployed with an Application Load Balancer and an Auto Scaling Group. Currently, the scaling of the Auto Scaling Group is done manually and you would like to define a scaling policy that will ensure the average number of connections to your EC2 instances is averaging at around 1000. Which scaling policy should you use?
Target Tracking
