Introduction Flashcards
What is a distributed system ?
A distributed system is a collection of autonomous computing elements that appears to its users as a single coherent system.
Definition of a Distributed System
Formal Definition:
A distributed system is a collection of autonomous computing elements that appears to its users as a single coherent system.
Attributes:
Comprises multiple computing nodes.
Nodes interact autonomously to achieve a common goal.
Users perceive the system as unified despite its distributed nature.
Computing Elements in a Distributed System
Definition:
In a distributed system, computing elements are autonomous entities capable of computation.
In context, these elements are physical machines (nodes) interacting to serve users or other systems.
Characteristics:
Nodes operate independently and can run distinct programs.
They communicate and coordinate to achieve system goals.
Coherent Operation in a Distributed System
User Perspective:
Users interacting with a distributed system experience a unified and coherent system.
The system’s operation appears seamless, and users’ expectations are met consistently.
Example:
Instagram functions as a distributed system, providing a smooth experience across its features despite running on multiple nodes.
Key Aspects of a Distributed System
Node Functionality:
Nodes interact to achieve shared objectives.
Each node performs autonomous tasks.
User Experience:
A distributed system presents a unified and consistent interface to users.
Users interact with the system without noticing its distributed nature.
Achieving Coherence
Objective:
Distributed systems aim to avoid inconsistencies in user experience.
Efforts focus on maintaining coherence across system functionalities.
Challenge:
Achieving coherence in a distributed system is complex due to its decentralized nature.
Most Obvious Distributed System
Example:
The internet serves as a widespread distributed system.
Networked nodes globally interact to facilitate data exchange and user interaction.
Characteristics:
Comprised of numerous interconnected nodes.
Seamlessly serves users’ requests across multiple servers.
Most Obvious Distributed System
Example:
The internet serves as a widespread distributed system.
Networked nodes globally interact to facilitate data exchange and user interaction.
Characteristics:
Comprised of numerous interconnected nodes.
Seamlessly serves users’ requests across multiple servers.
Simple Example: My Cool App
Scenario:
Development of a mobile app named “My Cool App” for user interaction.
Requires server interaction and data retrieval from a database.
Initial Setup:
Single node hosts both the server program and the database.
Potential risks include data loss on node failure.
Loneliness Mitigation
Strategy:
Separate the server and database programs onto distinct nodes.
Mitigates risks associated with single-node failure.
Enhancement:
Increases system resilience by distributing server and database functionalities across separate nodes.
Strengthening the System
Enhancement Approach:
Expand the fleet by adding more server and database nodes.
Load balancing ensures even distribution of user requests across multiple server nodes.
Replication:
Database nodes contain replicated data for redundancy and data consistency.
Transition to a Distributed System
Characteristics:
Transition from single-node to multi-node architecture.
System operates coherently and autonomously despite multiple nodes.
Goal:
Prevent system failure due to single-node issues.
Scale resources to handle increased user load.
Key Takeaways
Load Handling:
Increased resources required to handle higher system loads.
Multi-node architecture mitigates single-node failure risks.
Autonomy and Coherence:
Nodes operate independently yet coherently to serve user needs.
Ensures a seamless experience despite the distributed architecture.
Hardware Limitations and Failures
Single nodes can crash due to hardware failures.
Scaling with a single node has limitations in terms of performance and capacity.
No hardware is infallible; everything degrades over time.
Consistency and Redundancy
Running multiple copies of programs on different nodes ensures consistency and redundancy.
Redundancy prevents a total system failure if one or more nodes go down.
Scalability
As the user base grows, the system needs to scale to handle increased load.
Distributing server programs across multiple nodes helps manage the increased load efficiently.
System Reliability and Load Balancing
Distributing programs across nodes prevents overwhelming a single node with excessive requests.
Redundancy ensures that even if some nodes fail, the system still functions, albeit partially.
Distributed Systems: Precautions and Considerations
Evaluating the need for a distributed system is crucial.
It’s essential to weigh the benefits against the complexity and costs.
Business requirements and user needs should guide the decision to adopt a distributed system.
Decision-Making for Distributed Systems
Critical questions about scalability, downtime, and resource utilization need to be addressed.
Careful analysis helps determine if the complexity of a distributed system aligns with business needs.
Cautionary Approach
While distributed systems may seem attractive, careful evaluation is necessary before implementation.
Understanding the business requirements and use-case scenarios helps in making informed decisions.
What does fault tolerance mean in the context of distributed systems?
Fault tolerance in distributed systems refers to the system’s ability to continue operating and providing its services even in the presence of faults or failures in different components or nodes within the system. It involves designing the system to handle and recover from faults to maintain functionality.
Why we need fault tolerance?
Other Fault Tolerance
Examples of Faults in Distributed Systems
Highlight various scenarios like hardware failures, software bugs, network disruptions, and sudden traffic spikes that can cause faults in distributed systems.
Importance of Fault Tolerance
Why is fault tolerance essential in distributed systems? Discuss how it prevents system-wide failures and ensures continuous operation despite faults.
Strategies for Achieving Fault Tolerance
Explain different strategies like redundancy, replication, error handling, and graceful degradation used to achieve fault tolerance in distributed systems.
Challenges in Implementing Fault Tolerance
Explore the difficulties and complexities involved in building fault-tolerant distributed systems, such as maintaining consistency and performance while tolerating faults.
What is the definition of reliability in the context of distributed systems?
Reliability in distributed systems refers to the system’s capability to function correctly even when facing faults or failures in various components or nodes within the system. It involves ensuring that the system continues to serve users’ expectations, handles errors gracefully, remains performant, and endures malicious attacks or abuses.
Fault Tolerance:
Definition: Fault tolerance refers to a system’s ability to continue functioning, albeit in a degraded mode or reduced capacity, in the presence of faults or failures within its components. These faults could be hardware failures, software bugs, or network disruptions.
Focus: It primarily emphasizes the system’s ability to handle and recover from faults without causing complete system failure or severe disruption to its operations.
Strategies: Fault tolerance involves implementing mechanisms and strategies that enable a system to detect, isolate, and recover from faults. These may include redundancy, replication, error detection and correction codes, self-healing mechanisms, and graceful degradation.
Reliability:
Definition: Reliability in a distributed system refers to the system’s capability to consistently provide correct and expected functionality, delivering the intended service or output even in the face of faults or failures.
Focus: It emphasizes the overall dependability and consistency of the system in fulfilling its intended purpose or service, regardless of occasional faults that may occur within its components.
Scope: Reliability encompasses not only fault tolerance but also the system’s ability to maintain correctness, availability, and performance, meeting users’ expectations under normal and fault scenarios.
Fault Tolerance, Reliability Differences
Emphasis: Fault tolerance focuses on the system’s ability to handle faults and continue operating despite the presence of failures, whereas reliability focuses on consistently delivering correct and expected functionality regardless of faults.
Approach: Fault tolerance involves implementing specific mechanisms and strategies to address faults and mitigate their impact on the system. Reliability, on the other hand, is a broader concept encompassing the overall trustworthiness and dependability of the system.
Outcome: The goal of fault tolerance is to minimize the impact of faults by providing mechanisms for fault recovery and resilience. Reliability aims to ensure consistent and correct system behavior, maintaining service quality and user expectations.
Fault Tolerance:
Definition: System’s ability to continue operating despite component faults or failures.
Focus: Handling faults without causing complete system failure.
Strategies: Redundancy, replication, error detection, self-healing mechanisms.
Reliability:
Definition: System’s consistency in providing correct functionality even with faults.
Focus: Consistently delivering expected services despite occasional faults.
Scope: Encompasses correctness, availability, and meeting user expectations.
Handling Hardware Faults:
RAID: Redundant Array of Independent Disks to store data redundantly across multiple drives.
Purpose: Ensures data recovery in case of disk failures, reducing downtime.
Handling Software Faults:
Testing: Writing unit, integration, and end-to-end tests to ensure code correctness and interaction reliability.
Error Handling: Implementing robust error handling across the codebase to manage unexpected behaviors.
Monitoring: Continuous monitoring of critical system parts, adding logs, and using third-party services for immediate issue detection.
Summary:
Hardware Faults: Mitigated using RAID for redundancy, ensuring data retrieval and system recovery.
Software Faults: Managed through rigorous testing, error handling, and active system monitoring for fault detection and response.
Availability Definition:
Property: Ensures the system is ready to serve users when needed.
Calculation: Availability (A) = (U / (U + D)), where U is the time system was up, and D is the time system was down.
Motivation for Availability:
Business Impact: Downtime equals potential monetary loss and user dissatisfaction.
Use Cases: Critical for e-commerce, learning platforms, and any service dependent on user accessibility.
Availability Chart:
Percentage Allowed Downtime
90% : 36.5 days
95% : 18.25 days
99% : 3.65 days
99.5% : 1.83 days
99.9% : 8.76 hours
99.95% : 4.38 hours
99.99% : 52.6 minutes
99.999% : 5.26 minutes
Practices and Key Considerations:
System’s Expectation: Aim for higher availability percentages to reduce downtime.
Denoting Availability: Represented by “nines” (e.g., 99.99% is 4-nines).
SPOF
Definition: A component whose failure can bring down the entire system.
Example: Home router: Failure leads to loss of internet access.
Achieving Availability:
Redundancy: Add backup resources to handle system failures and avoid Single Points of Failure (SPoF).
Expectations: Set realistic availability expectations; even the best systems can face rare, simultaneous failures.
SLA, SLO, and SLI:
Service Level Agreement (SLA): Agreement between system and clients outlining requirements, e.g., uptime or response time.
Service Level Objective (SLO): Specific promise in an SLA, e.g., loading feeds within 200ms.
Service Level Indicator (SLI): Actual measured metrics compared to the SLA/SLO, e.g., promised uptime vs. actual uptime.
Measurement and Improvement:
Continuous Tuning: SLA, SLO, and SLI measurements tune expectations over time for better system improvements.
Focus on Improvements: These metrics guide system-level improvements and focus areas for better service delivery.
Scalability Definition:
Capability: It’s the system’s ability to handle increased usage or load efficiently.
Increased Load: Addressing system capability when user count or activity surges.
Scalability Questions:
System Impact: How does the user base increase affect the system’s performance?
Handling Increased Load: Can the system manage the amplified load without issues?
Resource Addition: Strategies for seamlessly adding computing resources to accommodate the load surge.
Vertical Scaling (Scaling Up):
Upgrade Node: Replace existing machines with more powerful ones.
Resource Enhancement: Add more RAM, CPU, storage, etc., to handle increased demands.
Limitations: Costly, hardware limitations, single point of failure.
Vertical Scaling Limitations:
Costly Approach: Continuous investment in larger machines.
Hardware Limitations: Eventually reaches a point of infeasibility with system growth.
Single Point of Failure: Relying on one machine’s resources.
Horizontal Scaling (Scaling Out):
Add More Nodes: Distribute load among multiple nodes to handle increased demands.
Cost Effectiveness: Utilize cheaper hardware, scalability, and redundancy.
Infinite Scaling Capacity: Potential for seemingly unlimited scalability.
Benefits of Horizontal Scaling:
Cost-Effectiveness: Running on cheaper hardware with added redundancy.
Infinite Scaling: If managed well, offers seemingly unlimited scaling capacity.
Avoid Single Points of Failure: Load distributed among multiple nodes, avoiding a single point of failure.
Scalability Conclusion:
Necessity in Growth: Crucial for systems as they evolve and face increased user demands.
Scaling Techniques: Vertical scaling becomes limited, while horizontal scaling is the industry standard for large systems.
What is a load balancer?
A load balancer (LB) is basically a node in a distributed system that tries to evenly distribute traffic among the actual server nodes.
Unhealthy nodes can be detected by the load balancer
It can be some /status endpoint that the LB can hit and then check the response code. Based on the response, the LB can declare a node as healthy or unhealthy, and take further actions to make sure the system remains highly available
Load Balancer Definition:
Purpose: Distributes incoming traffic across multiple server nodes.
Even Load Distribution: Ensures no server node is overwhelmed with requests.
Load Balancer Role:
Traffic Management: Sits between clients and server nodes, routing requests.
Distribution Algorithm: Routes requests based on predefined or combined algorithms.
Load Balancer Definition:
Purpose: Distributes incoming traffic across multiple server nodes.
Even Load Distribution: Ensures no server node is overwhelmed with requests.
Load Balancer Role:
Traffic Management: Sits between clients and server nodes, routing requests.
Distribution Algorithm: Routes requests based on predefined or combined algorithms.
Health Checks and High Availability:
Monitoring Nodes: Regularly checks server node health to avoid routing to unhealthy nodes.
Actions on Unhealthy Nodes: Stops routing requests, triggers alerts, initiates new node setup for high availability.
System Scaling:
Load-Triggered Scaling: Initiates node creation based on increased load to maintain system balance.
Regular Node Status Checks: Periodically verifies the status of seemingly faulty nodes for reintegration.
Load Balancer Single Point of Failure:
SPoF Risk: A single load balancer can lead to complete system downtime if it fails.
Redundancy Solution: Implementing multiple load balancers in a cluster with propagated IP addresses via DNS for failover.
Key Significance:
Critical Component: Vital for system resilience in distributed setups.
SPoF Mitigation: Conscious development and redundancy measures prevent load balancer-induced failures.
Application-layer Algorithms:
Hashing:
Utilizes predefined attributes (e.g., user_ID) to generate a hash value mapped to server nodes.
Endpoint Evaluation:
Routes requests based on different endpoint targets (e.g., /photos, /videos) to respective server sets.
Network-layer Algorithms:
Random Selection:
Routes requests randomly to any server node based on probability calculations.
Round Robin:
Distributes requests to servers in sequence (e.g., Server 1, Server 2, Server 3, repeat).
Least Connection:
Routes requests to the server with the fewest active connections.
IP Hashing:
Uses hashed source IP addresses to route requests to specific servers.
Least Pending Requests:
Routes incoming requests to the server with the fewest pending requests to optimize response times.
Choosing the Right Algorithm:
System-specific Selection:
Decide between network-layer or application-layer load balancing based on system needs.
Consider Request Patterns:
Short request-response patterns may suit round-robin or random algorithms.
System Response Times:
Systems with varied response times benefit from algorithms like least connection or least response time.
Client Behavior Impact:
Algorithms relying on certain attributes (e.g., IP hashing) may face imbalance due to client behavior, necessitating careful consideration.
End Point LB
Measuring load in distributed systems
Queries per Second (QPS):
Calculates the rate of requests a node handles per second.
Offers a simple metric, but may not accurately represent variations in load distribution across time periods.
Read-to-Write Ratio (r/w):
Determines if a system is read-heavy or write-heavy based on the proportion of read and write operations.
Read-heavy systems might need read replicas, while scaling write-heavy systems requires careful synchronization among nodes.
Additional Load Metrics:
Data size, response-request size, number of concurrent users, etc., are system-specific metrics that can also gauge load.
Measuring Performance
Response Time:
Represents the time taken for a client to receive a response after sending a request.
Average response time gives an overall view but might not reflect delays for specific users.
Percentiles:
Percentiles, like p50, p95, p99, indicate the response time for a certain percentage of requests.
Higher percentile values imply a slower response time for a smaller proportion of requests.
Deciding When to Scale:
Monitoring load metrics and performance indicators informs decisions on scaling requirements.
Higher QPS or response times beyond acceptable percentiles signal the need for scaling to meet increased demand.
Business Considerations:
Beyond a certain point, optimizing performance might incur high costs without significant benefit.
Ensuring resources align with the system’s needs is crucial for efficient operations.
QPS
This basically means how many queries or requests a node needs to handle each second. For example, say a server receives
10M requests per day.
Read-to-write ratio
An r/w value of means for each write, the database serves
10 read operations.
average response time
Maintainability of Distributed Systems
Maintainability Aspects:
Ease of Operation
Monitoring and Logging
Automation Support
Thorough Documentation
Self-Healing Mechanisms
Surprise Minimization
Maintainability Aspects:
Ease of Extension
Good Coding Practices
Refactoring Habits
Defined Deployment Procedures
Simple flow of data in a system
Data-Driven Modern Systems
Characteristics:
Data-Centric Systems
Intensified Data Handling
Data as a Business Driver
Data Handling in Modern Systems
Data Collection:
Personal Data (Sensitive)
User-Interaction Data (Logged Events)
Data Storage:
Various Storage Options
Choice Based on Data Volume & Usage
Data Processing in Modern Systems
Processing Stage:
Data Transformation for Usability
Challenge with Large Data Volume
Insights Generation
Insights from Data:
Useful Information for Business Growth
Predictive Analytics & Behavior Modeling
Key Takeaways
Data Intensity Impact:
Critical for Business Success
Requires Proficiency in Data Handling
Careful Collection, Processing, and Use of Data
Replication
Data Replication Essentials
Definition:
Storing Identical Data Copies Across Multiple Machines
Purpose:
Enhancing Read/Write Capacity
Increasing System Reliability
Importance of Network Connectivity
Network Connection:
Essential for Data Sync Across Machines
Communication for Data Changes
Network Reliability:
Critical for Data Replication
Challenges in Replication
Complexities:
Network Unreliability
Machine Failures
Storage Capacity Limitations
Dynamic Data Handling:
Challenges in Handling Changing Data
Replication Process
Scenarios:
Large Data Volumes
Multiple Machines Handling Data
Continuous Data Changes
Objective:
Synchronized Updates Across All Nodes
Consistent Data Availability
Replication Benefits:
Ensures Availability Despite Node Failures
Enhances System Scalability
Critical for Consistent Data Responses
Complexity with Volume:
Increased Data Volume Adds Complexity to Replication
