HA & Scaling

Question 1

What can you use in the absence of a LB?

Answer 1

You can use Route 53 or Multi-Value Routing which uses DNS.

DNS, where a single DNS instance might point to multiple backend servers.

This is not ideal however as failover is handled on the client side.

Question 2

Does a client point to the end server when using a LB?

Answer 2

No - Users do not connect to a server running an app, they instead point at/connect to the LB in the middle.

The LB will “listen” for traffic on specific ports/protocols - like HTTP and HTTPS i.e 80 and 443; if applied it becomes a “Listener” on those ports.

Question 3

LB Intro Summary

Answer 3

· Clients connect to the LB, not the application.. specifically, they connect to the Listener of the LB.

· LB connects on the client behalf to 1 or more targets/servers

· 1 connection between client/listener and another between the LB/backend instance - they are 2 separate connections

· LB’s job to abstract the client away from individual servers - this allows for HA/scaling/fault tolerance

Question 4

Can a LB run health checks?

Answer 4

Yes - LB’s will generally run health checks to all of the servers it’s connected to, so if a server fails, it will stop sending traffic to that server and the client/user will never know

Question 5

What is an Elastic LB (ELB)?

Answer 5

This is an umbrella term in AWS that refers to Classic LB, App LB, and/or Network LB.

The Classic LB is basically the original ELB that was succeeded by the ALB and then the NLB.

Question 6

What is an App LB?

Answer 6

Layer 7- LB

The ALB is capable of understanding the App Layer i.e L7 and can take action based on things within those L7 protocols.

Question 7

Are ALB’s public or private?

Answer 7

They can be either.

If PUBLIC - they can Listens on the OUTSIDE interface (public IP) and then sends to TARGETS or Target Groups.

If PRIVATE - has no public address and only LB’s traffic within the VPC that it’s in or between tiers of a multi-tier App.

Question 8

What is an LB Capacity Unit? (LCU)

Answer 8

An LCU is a new metric for determining how you pay for an Application Load Balancer. An LCU defines the maximum resource consumed in any one of the dimensions below:

○ constitutes for 25 new connections per second
○ 3k active connections per minute
○ 1GB per hour with EC2/containers as targets or 0.4 GB/hour for Lambda functions as targets

Question 9

What is Cross-Zone LB?

What is it the feature solving for?

Answer 9

A feature within the AWS LB that was introduced to enforce the ability for every LB Node, that an ALB DNS name points to, to distribute connections that they receive across all EC2 in all AZ’s that are registered to the LB.

Ex) The app has EC2’s in multiple AZ’s - the ALB can load balance traffic across all of them, where LB-B in AZ-B can send traffic directly to EC2’s in AZ-A.

Solving for if there are 10 EC2’s in AZ-A and 2 x EC2’s in AZ-B, where the traffic is split 50/50 by the LB. This would be very uneven in terms of load because AZ-B would split 50% of traffic between 2 instances and AZ-A would do the same across 10.

Question 10

ALB ARCH Terms:

Answer 10

○ Targets - services that an ALB can distribute services/connections to (like EC2, containers running ECS, Lambda, etc.)

○ Group - a grouping of targets that represent the same service; ALB’s distribute connections amongst Targets within the Group

○ Rules - the options are:

1. Host Rules - use different DNS names –> EX) sports.com/nfl is TG-A and sports.com/mlb is TG-B
2. Path Rules - interpret different paths within the HTTP address and direct incoming requests to different Target Groups –> EX) nfl.sports.com is TG-A and mlb.sports.com is TG-B

Question 11

ALB Summary:

→ has a listener config that controls exactly what ports/protocols it’s listening for

→ Has host-based or path-based rules which direct those incoming connections to different target group(s)

→ A target group is a collection of targets

→ A target can be an EC2 instance, container, or lambda function (a COMPUTE resource)

Answer 11

· Targets = a compute resource that connections are directed towards (EC2, containers, Lamda, etc)

· Target Groups = groups of targets which are addressed with Rules

· Rules = path-based or host-based

· Supports EC2, ECS, EKS, Lambda, HTTPS, etc.

· Classic LB (legacy) only exists for legacy reasons and are not recommended moving forward

** if you need to deploy multiple SSL Certificates to a single LB, you’d have to use an ALB (Classic LB does not support this) **

Question 12

What do Launch Configs and Launch Templates aim to achieve?

Answer 12

Both of these tools perform the same task - they allow you to define the config on an EC2 instance in advance.

Anything you define at the point where you launch an instance, you can define ahead of time with their of these 2 methods:

○ AMI, Instance Type, Storage and Key Repair
○ Networking/SEC group
○ User Data & IAM Role (for permissions)

Question 13

Can you edit Launch Configs or Launch Templates?

Answer 13

No - Neither of these are editable, they are defined ONCE.

With LT’s however you can create different versions.

Question 14

What is an Auto-Scaling Group?

Answer 14

Feature within AWS that allows the auto-scale and self-healing of EC2 instances i.e if an instance fails, it will kill the instance and auto-provision a new one based on a Launch Config or Launch Template that it’s linked with.

Question 15

What are the 3 important values/parameters with ASGs?

Answer 15

You must define these 3 things for what the group will provision from an EC2 instance perspective:

Minimum
Maximum
Desired

ASG will provision or terminate EC2 instances within the min/max threshold based on how many are provisioned and what the desired state is.

Question 16

What is a Scaling Policy within an ASG?

Answer 16

The policy that determines what the Desired State is for an ASG.

This can be manual or automatic.

Question 17

REVIEW: Types of Scaling Policies

Answer 17

○ Manual scaling - manually adjust the desired capacity

○ Schedule scaling - time-based adjustment; great for sales peak seasons or non-work hours

○ Dynamic scaling:
§ Simple - define a rule based on an metric; EX) if CPU goes above 50% then add another instance, if below, then remove one –> take ACTION based on a METRIC

§ Stepped - more detailed rules than Simple; EX) add another instance if CPU is above 50% or add 3 if above 80% during a spike –> react in more extreme way the more extreme a change in conditions

§ Target Tracking - define an ideal Aggregate Metric; the group will scale to try and stay at that defined amount as a collective group; EX) all instance CPU’s to be at 40% utilization

○ Cooldown Period - control over how long ASG will wait at the end of a scaling action before performing another
§ This can help with sporadic costs because there are minimum billing durations for EC2
If your environment is constantly changing and you’re always spinning up/down resources, you can get billed $$$$$

Question 18

Can ASG’s run health checks?

Answer 18

Yes - they can monitor the health of instances and can terminate a failed instances and then instruct EC2 to spin up another. This isolates issues to a single instance.

This is part of the Self-Healing ARCH concept.

Question 19

Can an ASG integrate with an LB?

Answer 19

Yes - this is actually best practice.

ASG can be configured to use the LB for EC2 health checks rather than the EC2 status checks which are not as granular and are also not app-aware.

Question 20

What is the cost of an Auto-Scaling Group?

Answer 20

The feature is FREE

You only pay for the resources that are provisioned.

Question 21

“The ASG defines ____ and ____ and the Launch Template defines ______”

Answer 21

ASG defines:

WHEN - certain time, or when a CPU hit’s a certain amount

and

WHERE you want to launch something - in what subnet and in what VPC the resources will be provisioned into

The Launch Template defines:

WHAT you are launching - the config instructions defining the parameters of the EC2 that will be launched such as IAM, what SEC group, what Networking parameters, etc.

Question 22

What is a Network Load Balancer (NLB)?

Answer 22

An LB that operates at L4.

Source/Desty IP address, TCP, and UDP. They cannot understand HTTP/HTTPs like a ALB does.

Anything running above L4 is just passed along.

Question 23

What is a major benefit of a NLB over an ALB?

Answer 23

Because they operate only up to L4, they are MUCH faster and have MUCH better performance i.e way less latency.

If HTTP/HTTPS is not required and you need very low latency, go with a NLB

Question 24

When would you use a NLB?

Answer 24

If you need to LB for any protocols other than HTTP/HTTPS.

Question 25

What are the 3 ways that a LB can handle Secure Connections?

Answer 25

Bridging - default; 2 separate encrypted connections between host and target

Passthrough - 1 secure connection, LB passes connection through

SSL Offload - LB accepts encrypted connection and sends it along in plain text

Question 26

What happens in LB with Bridging Mode?

Answer 26

Default with an ALB. Listener configured for HTTPS.

§ Connection is terminated/decrypted on the LB itself which means the LB needs to have an SSL certificate present which match the domain the app uses.

§ LB will then issue a new SSL connection from itself to the backend EC2 instance(s); this is a new encrypted session

§ Every EC2 instance on the backend therefore needs to be able to perform cryptographic operations (this can create a lot of overhead)

THIS CAUSES OVERHEAD.

Question 27

What happens in LB Passthrough Mode?

Answer 27

Listener configured for TCP.

Client connects to the LB but the LB just passed that connection along. No crypto functions happens on the LB.

§ Connection encryption is maintained between the client and the backend EC2 instance(s); ONE encrypted tunnel.

Question 28

What happens in LB SSL Off-Load Mode?

Answer 28

Listener configured for HTTPS.

Clients connect to the LB via HTTPS that terminates on the LB, but the LB connects to the backend EC2 using HTTP (not encrypted)

The data is in clear-text behind the LB to the EC2 instances.

LESS OVERHEAD than Bridging Mode because no certs or crypto functions are required on the backend EC2’s.

Question 29

What is Connection/Session Stickiness?

Answer 29

If enabled, the first time a user makes a request, the LB will generate a cookie (where you define “x” amount of time between 1 second to 7 days).

During that time frame, sessions that a user generates will always be sent to the SAME backend EC2 instance to continue the session.

Question 30

Where are scaling policies defined? As in, where do you define policies for how a group will horizontally scale?

Answer 30

Optionally in the ASG

Question 31

Can Launch Configuration and Launch Templates be used to directly launch an EC2 instance?

Answer 31

No. Only LT’s can do that.

Question 32

Do ALBs and NLBs have a static IP address?

Answer 32

No - only NLBs receive a static IP