Scaling, Caches, LBs

Question 1

Key characteristics of distributed systems

Answer 1

scalability
reliability
availability
efficiency
manageability

Question 2

Scalability

Answer 2

the capability of a system, process or network to grow and manage increased demand

Question 3

Horizontal Scaling

Answer 3

Scale up by adding more servers to your pool of resources

Question 4

Vertical Scaling

Answer 4

Scale up by adding more power (CPU, RAM, Storage) to an existing server

vertical scaling is usually limited to the capacity of a single server

Question 5

Reliability

Answer 5

Reliability refers to a system’s ability to perform it’s intend functions correctly and consistently

Reliability can be achieved through redundancy

Question 6

Redundancy

Answer 6

the intentional duplication of critical components or functions of a system with the goal of increasing reliability and eliminating singles points of failure

Question 7

Database Replication

Answer 7

Database replication is the process of copying and synchronizing data from one database to one or more additional databases.

This is common in distributed systems

ensures data availability, fault tolerance, and scalability.

Question 8

Synchronous Replication

Answer 8

with synchronous replication, changes made to the primary database are immediately replicated to the replica databases before the write operation is considered complete

this ensures strong consistency between the database and its replicas

Question 9

Asynchronous Replication

Answer 9

a type of database replication where changes made to the primary database are not immediately replicated to the replicas

Instead, changes are queued and replicated to the replicas at a later time.

PROS - Faster Performance, and writes won’t fail bc of issues with the replica

CONS - This delay can result in temporary inconsistencies between the primary and replica databases

Question 11

Availability

Answer 11

A measure of the percentage of the time that a system, service or machine remains operational under normal conditions

Question 12

Reliability vs. Availability

Answer 12

A system is RELIABLE if it’s able to function correctly

A system is AVAILABLE depending on the % of the time that it is operational under normal conditions

Good reliability translates to good availability
But availability does not indicate reliability
You can have high availability on an unreliable product by minimizing repair times and ensuring that replicas are always ready

Question 13

Efficiency

Answer 13

Efficiency of a system is measured through latency and throughput

Question 14

Latency

Answer 14

The time it takes for a certain operation to complete in a system

Question 15

Throughput

Answer 15

The number of operations that a system can handle properly per time unit. Commonly measured in RPS (requests per second)

Question 16

Serviceability

Answer 16

AKA manageability
describes how easy it is to maintain and repair a distributed system

key considerations:
- ease of diagnosing and understanding problems
- ease of making updates / modifications
- how simple the system is to operate

Question 17

Load Balancer

Answer 17

device that sits between the client and the server to accept incoming network traffic / distribute it across multiple backend servers using various algorithms

reduces individual server load
prevents single-point-of-failure
improves availability

HEALTH CHECKS:
important to note that load balancers should keep track of the status of available resources while distributing requests. If a server is not available or has an elevated error rate, LB will not send traffic there

Question 18

Where can we place load balancers?

Answer 18

Between the user and the web servers

Between web servers and an internal platform layer, like application servers or cache servers

Between internal platform layer and a database

Question 19

Benefits of Load Balancing

Answer 19

• Users experience uninterrupted service
• less downtime / higher throughput
• smart load balancers can use predictive analytics to predict bottlenecks
• fewer stressed / failing systems component; no one device is shouldering all the work

Question 20

Least Connection Method

Answer 20

Load Balancing Algorithm

directs traffic to the server with the fewest active connections. useful for evening out distribution between servers

Question 21

Least Bandwidth Method

Answer 21

Load Balancing Algorithm

directs traffic to the server with the least amount of traffic (Mbps)

Question 22

Least Response Time Method

Answer 22

Load Balancing Algorithm

directs traffic to the server with the lowest average response time

Question 23

Round Robin Method

Answer 23

Load Balancing Algorithm

cycles through a list of available servers and sends each new request to the next server. useful when the servers are of equal specification and there are not many persistent connections

Question 24

Weighted Round Robin Method

Answer 24

Load Balancing Algorithm

Round robin method, but servers are assigned a weight and servers with higher weights get more new connections

Question 25

IP Hash Method

Answer 25

Load Balancing Algorithm

A hash of the IP address of the client is calculated to redirect the request to a server

Useful for ensuring that requests from the same client go to the same server

Question 26

Cache

Answer 26

A cache is like short-term memory for a system. It stores recently or accessed data for fast retrieval.

It’s often used to store responses to network requests or the results of computationally long operations

Caches can quickly return data without taxing downstream systems

Question 27

Application Server Cache

Answer 27

cache placed directly on the request layer node

this enables the local storage of response data

each time a request is made to the service, the node will return local data if it exists, or fetch the data from disk

Question 28

Global Cache

Answer 28

With a global cache, all distributed nodes use the same cache.

it’s very easy to overwhelm as number of clients and requests increase

Question 29

Distributed Cache

Answer 29

The cache is divided up by using a consistent hashing function, and each of it’s nodes owns part of the cached data

Question 30

CDN

Answer 30

Content Delivery Network OR
Content Distribution Network

CDNs are a kind of cache that we can use for serving large amounts of static media.

A request first asks the CDN for the media, and if it has it locally available, it will serve it. But it if it is not available, it will query backend servers, serve + cache the result

Question 31

Cache Invalidation

Answer 31

Cache invalidation is maintenance that is required to keep the cache coherent with the source of truth (for example, a database)

If the data is modified in the database, it should be invalidated in the cache

Question 32

Write-Through Cache

Answer 32

Cache-Writing Policy

data is written into the cache and corresponding database simultaneously

PROS: complete consistency between cache and storage. minimizes risk of data loss

CONS: higher latency (takes longer)

Question 33

Write-Around Cache

Answer 33

Cache-Writing Policy

Data is written only to storage, bypassing cache

PROS: prevents cache from being flooded with write operations

CONS: a read request for recently-written data will be a cache miss (higher latency)

Question 34

Write-Back Cache

Answer 34

Cache-Writing Policy

Data is written ONLY to the cache, and writes to storage are done after specified intervals (or under other conditions

PROS: low latency, high throughput for write-intensive applications

CONS: risk of data loss since the only copy of the data is the cache

Question 35

Cache Invalidation Methods

Answer 35

Methods to invalidate a cache

PURGE - removes cached content for a specific object or URL, typically when there’s an update to that content

REFRESH - fetches requested content from origin server, even if available in cache, then updates the cache

BAN - invalidates cached content based on specific criteria such as URL pattern or header

Time to Live (TTL) - expires cached values after a certain amount of time

Stale While Revalidate - serves cached content immediately, then makes a request to update the cache with fresh content from the server

Question 36

Read Through Cache

Answer 36

Cache Reading Strategy

if data is not in the cache, the cache makes the read request to the source, updates itself, and returns the data

ideal for scenarios where cache misses are infrequent

Question 37

Read Around Cache

Answer 37

Cache Reading Strategy

if the data is not in the cache, the application makes the request to the source, then decides how / when to update the cache

Question 38

Cache Eviction Policies

Answer 38

FIFO - first in, first out
LIFO - last in, first out
LRU - least recently used
MRU - most recently used
LFU - least frequently used
RR - random replacement