System Design & Optimization Flashcards

Question

What is gRPC?

Answer 1

gRPC is a high-performance, open-source universal RPC (Remote Procedure Call) framework initially developed by Google. It leverages HTTP/2 for transport, Protocol Buffers as the interface description language, and provides features such as authentication, load balancing, and more. gRPC is designed to enable efficient and robust communication between services in a microservices architecture, making it a popular choice for building distributed systems and APIs. Key Features of gRPC: 1. Protocol Buffers: By default, gRPC uses Protocol Buffers (proto files) as its interface definition language (IDL). This makes gRPC messages smaller and faster compared to JSON or XML. 2. HTTP/2 Based Transport: gRPC uses HTTP/2 for transport, which allows for many improvements over HTTP/1.x. 3. Multiple Language Support: gRPC supports a wide range of programming languages. 4. Bi-Directional Streaming: gRPC supports streaming requests and responses, allowing for the development of sophisticated real-time applications with bidirectional communication like chat services.

Answer 2

The diagram below shows top 6 data models. 🔹 Flat Model The flat data model is one of the simplest forms of database models. It organizes data into a single table where each row represents a record and each column represents an attribute. This model is similar to a spreadsheet and is straightforward to understand and implement. However, it lacks the ability to efficiently handle complex relationships between data entities. 🔹 Hierarchical Model The hierarchical data model organizes data into a tree-like structure, where each record has a single parent but can have multiple children. This model is efficient for scenarios with a clear "parent-child" relationship among data entities. However, it struggles with many-to-many relationships and can become complex and rigid. 🔹 Relational Model Introduced by E.F. Codd in 1970, the relational model represents data in tables (relations), consisting of rows (tuples) and columns (attributes). It supports data integrity and avoids redundancy through the use of keys and normalization. The relational model's strength lies in its flexibility and the simplicity of its query language, SQL (Structured Query Language), making it the most widely used data model for traditional database systems. It efficiently handles many-to-many relationships and supports complex queries and transactions. 🔹 Star Schema The star schema is a specialized data model used in data warehousing for OLAP (Online Analytical Processing) applications. It features a central fact table that contains measurable, quantitative data, surrounded by dimension tables that contain descriptive attributes related to the fact data. This model is optimized for query performance in analytical applications, offering simplicity and fast data retrieval by minimizing the number of joins needed for queries. 🔹 Snowflake Model The snowflake model is a variation of the star schema where the dimension tables are normalized into multiple related tables, reducing redundancy and improving data integrity. This results in a structure that resembles a snowflake. While the snowflake model can lead to more complex queries due to the increased number of joins, it offers benefits in terms of storage efficiency and can be advantageous in scenarios where dimension tables are large or frequently updated. 🔹 Network Model The network data model allows each record to have multiple parents and children, forming a graph structure that can represent complex relationships between data entities. This model overcomes some of the hierarchical model's limitations by efficiently handling many-to-many relationships.

Answer 3

The diagram below shows top 6 Heartbeat Detection Mechanisms. Heartbeat mechanisms are crucial in distributed systems for monitoring the health and status of various components. Here are several types of heartbeat detection mechanisms commonly used in distributed systems: 🔹 Push-Based Heartbeat The most basic form of heartbeat involves a periodic signal sent from one node to another or to a monitoring service. If the heartbeat signals stop arriving within a specified interval, the system assumes that the node has failed. This is simple to implement, but network congestion can lead to false positives. 🔹 Pull-Based Heartbeat Instead of nodes sending heartbeats actively, a central monitor might periodically "pull" status information from nodes. It reduces network traffic but might increase latency in failure detection. 🔹 Heartbeat with Health Check This includes diagnostic information about the node's health in the heartbeat signal. This information can include CPU usage, memory usage, or application-specific metrics. It Provides more detailed information about the node, allowing for more nuanced decision-making. However, it Increases complexity and potential for larger network overhead. 🔹 Heartbeat with Timestamps Heartbeats that include timestamps can help the receiving node or service determine not just if a node is alive, but also if there are network delays affecting communication. 🔹 Heartbeat with Acknowledgement The receiver of the heartbeat message must send back an acknowledgment in this model. This ensures that not only is the sender alive, but the network path between the sender and receiver is also functional. 🔹 Heartbeat with Quorum In some distributed systems, especially those involving consensus protocols like Paxos or Raft, the concept of a quorum (a majority of nodes) is used. Heartbeats might be used to establish or maintain a quorum, ensuring that a sufficient number of nodes are operational for the system to make decisions. This brings complexity in implementation and managing quorum changes as nodes join or leave the system.

Answer 4

Designing secure systems is important for a multitude of reasons, spanning from protecting sensitive information to ensuring the stability and reliability of the infrastructure. As developers, we should design and implement these security guidelines by default. The diagram below is a pragmatic cheat sheet with the use cases and key design points. 🔹 Authentication 🔹 Authorization 🔹 Encryption 🔹 Vulnerability 🔹 Audit & Compliance 🔹 Network Security 🔹 Terminal Security 🔹 Emergency Responses 🔹 Container Security 🔹 API Security 🔹 3rd-Party Vendor Management 🔹 Disaster Recovery

Answer 5

🔹 Cache Aside When an application needs to access data, it first checks the cache. If the data is not present (a cache miss), it fetches the data from the data store, stores it in the cache, and then returns the data to the user. This pattern is particularly useful for scenarios where data is read frequently but updated less often. 🔹 Materialized View A Materialized View is a database object that contains the results of a query. It is physically stored,meaning the data is actually computed and stored on disk, as opposed to being dynamically generated upon each request. This can significantly speed up query times for complex calculations or aggregations that would otherwise need to be computed on the fly. Materialized views are especially beneficial in data warehousing and business intelligence scenarios where query performance is critical. 🔹 CQRS CQRS is an architectural pattern that separates the models for reading and writing data. This means that the data structures used for querying data (reads) are separated from the structures used for updating data (writes). This separation allows for optimization of each operation independently, improving performance, scalability, and security. CQRS can be particularly useful in complex systems where the read and write operations have very different requirements. 🔹 Event Sourcing Event Sourcing is a pattern where changes to the application state are stored as a sequence of events. Instead of storing just the current state of data in a domain, Event Sourcing stores a log of all the changes (events) that have occurred over time. This allows the application to reconstruct past states and provides an audit trail of changes. Event Sourcing is beneficial in scenarios requiring complex business transactions, auditability, and the ability to rollback or replay events. 🔹 Index Table The Index Table pattern involves creating additional tables in a database that are optimized for specific query operations. These tables act as secondary indexes and are designed to speed up the retrieval of data without requiring a full scan of the primary data store. Index tables are particularly useful in scenarios with large datasets and where certain queries are performed frequently. 🔹 Sharding Sharding is a data partitioning pattern where data is divided into smaller, more manageable pieces, or "shards", each of which can be stored on different database servers. This pattern is used to distribute the data across multiple machines to improve scalability and performance. Sharding is particularly effective in high-volume applications, as it allows for horizontal scaling, spreading the load across multiple servers to handle more users and transactions.

Answer 6

1. Follow Code Specifications When we write code, it is important to follow the industry's well-established norms, like “PEP 8”, “Google Java Style”, adhering to a set of agreed-upon code specifications ensures that the quality of the code is consistent and readable. 2. Documentation and Comments Good code should be clearly documented and commented to explain complex logic and decisions, and comments should explain why a certain approach was taken (“Why”) rather than what exactly is being done (“What”). Documentation and comments should be clear, concise, and continuously updated. 3. Robustness Good code should be able to handle a variety of unexpected situations and inputs without crashing or producing unpredictable results. Most common approach is to catch and handle exceptions. 4. Follow the SOLID principle “Single Responsibility”, “Open/Closed”, “Liskov Substitution”, “Interface Segregation”, and “Dependency Inversion” - these five principles (SOLID for short) are the cornerstones of writing code that scales and is easy to maintain. 5. Make Testing Easy Testability of software is particularly important. Good code should be easy to test, both by trying to reduce the complexity of each component, and by supporting automated testing to ensure that it behaves as expected. 6. Abstraction Abstraction requires us to extract the core logic and hide the complexity, thus making the code more flexible and generic. Good code should have a moderate level of abstraction, neither over-designed nor neglecting long-term expandability and maintainability. 7. Utilize Design Patterns, but don't over-design Design patterns can help us solve some common problems. However, every pattern has its applicable scenarios. Overusing or misusing design patterns may make your code more complex and difficult to understand. 8. Reduce Global Dependencies We can get bogged down in dependencies and confusing state management if we use global variables and instances. Good code should rely on localized state and parameter passing. Functions should be side-effect free. 9. Continuous Refactoring Good code is maintainable and extensible. Continuous refactoring reduces technical debt by identifying and fixing problems as early as possible. 10. Security is a Top Priority Good code should avoid common security vulnerabilities. Over to you: which one do you prefer, and with which one do you disagree?

Answer 7

When we develop microservices, we need to follow the following best practices: 1. Use separate data storage for each microservice 2. Keep code at a similar level of maturity 3. Separate build for each microservice 4. Assign each microservice with a single responsibility 5. Deploy into containers 6. Design stateless services 7. Adopt domain-driven design 8. Design micro frontend 9. Orchestrating microservices

Answer 8

Explanation 1 OAuth 2.0 is an authorization framework that enables applications to access a user’s data on another service without sharing the user’s password. It’s essentially a digital handshake between the app, service, and user, with everyone agreeing on what is shared. The process generally follows 6 steps: 🔶 1) Request access 🔶 2) Redirect to service 🔶 3) Permission request 🔶 4) Authorization code 🔶 5) Exchange code for token 🔶 6) Use the token There are typically 4 components involved in the process: 🔷 Client (app wanting access) 🔷 Resource owner (user) 🔷 Authorization server 🔷 Resource server OAuth 2.0 provides multiple grant types to cater to different use cases. These grant types dictate how the application gets an access token. For most web applications, the Authorization Code Grant is the recommended and most secure method to obtain access tokens. Explanation 2 OAuth 2.0 is a powerful and secure framework that allows different applications to securely interact with each other on behalf of users without sharing sensitive credentials. The entities involved in OAuth are the User, the Server, and the Identity Provider (IDP). What Can an OAuth Token Do? When you use OAuth, you get an OAuth token that represents your identity and permissions. This token can do a few important things: Single Sign-On (SSO): With an OAuth token, you can log into multiple services or apps using just one login, making life easier and safer. Authorization Across Systems: The OAuth token allows you to share your authorization or access rights across various systems, so you don't have to log in separately everywhere. Accessing User Profile: Apps with an OAuth token can access certain parts of your user profile that you allow, but they won't see everything. Remember, OAuth 2.0 is all about keeping you and your data safe while making your online experiences seamless and hassle-free across different applications and services. Over to you: Imagine you have a magical power to grant one wish to OAuth 2.0. What would that be? Maybe your suggestions actually lead to OAuth 3.

Answer 9

As modern websites and applications are like busy beehives, we use a variety of tools to manage the buzz. Here we'll explore three superheroes: Reverse Proxy, API Gateway, and Load Balancer. 🔹Reverse Proxy: change identity - Fetching data secretly, keeping servers hidden. - Perfect for shielding sensitive websites from cyber-attacks and prying eyes. 🔹API Gateway: postman - Delivers requests to the right services. - Ideal for bustling applications with numerous intercommunicating services. 🔹Load Balancer: traffic cop - Directs traffic evenly across servers, preventing bottlenecks - Essential for popular websites with heavy traffic and high demand. In a nutshell, choose a Reverse Proxy for stealth, an API Gateway for organized communications, and a Load Balancer for traffic control. Sometimes, it's wise to have all three - they make a super team that keeps your digital kingdom safe and efficient.

Answer 10

🔹 What is Sensitive Data? Personal Identifiable Information (PII), health information, intellectual property, financial information, education and legal records are all sensitive data. Most countries have laws and regulations that require the protection of sensitive data. For example, the General Data Protection Regulation (GDPR) in the European Union sets stringent rules for data protection and privacy. Non-compliance with such regulations can result in hefty fines, legal actions, and sanctions against the violating entity. When we design systems, we need to design for data protection. 🔹 Encryption & Key Management The data transmission needs to be encrypted using SSL. Passwords shouldn’t be stored in plain text. For key storage, we design different roles including password applicant, password manager and auditor, all holding one piece of the key. We will need all three keys to open a lock. 🔹 Data Desensitization Data desensitization, also known as data anonymization or data sanitization, refers to the process of removing or modifying personal information from a dataset so that individuals cannot be readily identified. This practice is crucial in protecting individuals' privacy and ensuring compliance with data protection laws and regulations. Data desensitization is often used when sharing data externally, such as for research or statistical analysis, or even internally within an organization, to limit access to sensitive information. Algorithms like GCM store cipher data and keys separately so that hackers are not able to decipher the user data. 🔹 Minimal Data Permissions To protect sensitive data, we should grant minimal permissions to the users. Often we design Role-Based Access Control (RBAC) to restrict access to authorized users based on their roles within an organization. It is a widely used access control mechanism that simplifies the management of user permissions, ensuring that users have access to only the information and resources necessary for their roles. 🔹 Data Lifecycle Management When we develop data products like reports or data feeds, we need to design a process to maintain data quality. Data developers should be granted with necessary permissions during development. After the data is online, they should be revoked from the data access.

Answer 11

Efficient load balancing is vital for optimizing the performance and availability of your applications in the cloud. However, managing load balancers can be overwhelming, given the various types and configuration options available. In today's multi-cloud landscape, mastering load balancing is essential to ensure seamless user experiences and maximize resource utilization, especially when orchestrating applications across multiple cloud providers. Having the right knowledge is key to overcoming these challenges and achieving consistent, reliable application delivery. In selecting the appropriate load balancer type, it's essential to consider factors such as application traffic patterns, scalability requirements, and security considerations. By carefully evaluating your specific use case, you can make informed decisions that enhance your cloud infrastructure's efficiency and reliability. This Cloud Load Balancer cheat sheet would help you in simplifying the decision-making process and helping you implement the most effective load balancing strategy for your cloud-based applications. Over to you: What factors do you believe are most crucial in choosing the right load balancer type for your applications?

Answer 12

🔹 Atomicity The writes in a transaction are executed all at once and cannot be broken into smaller parts. If there.are faults when executing the transaction, the writes in the transaction are rolled back. So atomicity means “all or nothing”. 🔹 Consistency Unlike “consistency” in CAP theorem, which means every read receives the most recent write or an error, here consistency means preserving database invariants. Any data written by a transaction must be valid according to all defined rules and maintain the database in a good state. 🔹 Isolation When there are concurrent writes from two different transactions, the two transactions are isolated from each other. The most strict isolation is “serializability”, where each transaction acts like it is the only transaction running in the database. However, this is hard to implement in reality, so we often adopt loser isolation level. 🔹 Durability Data is persisted after a transaction is committed even in a system failure. In a distributed system, this means the data is replicated to some other nodes.

Answer 13

The diagram below explains the common acronyms in system designs. 🔹 CAP CAP theorem states that any distributed data store can only provide two of the following three guarantees: 1. Consistency - Every read receives the most recent write or an error. 2. Availability - Every request receives a response. 3. Partition tolerance - The system continues to operate in network faults. However, this theorem was criticized for being too narrow for distributed systems, and we shouldn’t use it to categorize the databases. Network faults are guaranteed to happen in distributed systems, and we must deal with this in any distributed systems. 🔹 BASE The ACID (Atomicity-Consistency-Isolation-Durability) model used in relational databases is too strict for NoSQL databases. The BASE principle offers more flexibility, choosing availability over consistency. It states that the states will eventually be consistent. 🔹 SOLID SOLID principle is quite famous in OOP. There are 5 components to it. 1. SRP (Single Responsibility Principle) Each unit of code should have one responsibility. 2. OCP (Open Close Principle) Units of code should be open for extension but closed for modification. 3. LSP (Liskov Substitution Principle) A subclass should be able to be substituted by its base class. 4. ISP (Interface Segregation Principle) Expose multiple interfaces with specific responsibilities. 5. DIP (Dependency Inversion Principle) Use abstractions to decouple dependencies in the system. 🔹 KISS "Keep it simple, stupid!" is a design principle first noted by the U.S. Navy in 1960. It states that most systems work best if they are kept simple.

Answer 14

1. High Availability This means we need to ensure a high agreed level of uptime. We often describe the design target as “3 nines” or “4 nines”. “4 nines”, 99.99% uptime, means the service can only be down 8.64 seconds per day. To achieve high availability, we need to design redundancy in the system. There are several ways to do this: - Hot-hot: two instances receive the same input and send the output to the downstream service. In case one side is down, the other side can immediately take over. Since both sides send output to the downstream, the downstream system needs to dedupe. - Hot-warm: two instances receive the same input and only the hot side sends the output to the downstream service. In case the hot side is down, the warm side takes over and starts to send output to the downstream service. - Single-leader cluster: one leader instance receives data from the upstream system and replicates to other replicas. - Leaderless cluster: there is no leader in this type of cluster. Any write will get replicated to other instances. As long as the number of write instances plus the number of read instances are larger than the total number of instances, we should get valid data. 2. High Throughput This means the service needs to handle a high number of requests given a period of time. Commonly used metrics are QPS (query per second) or TPS (transaction per second). To achieve high throughput, we often add caches to the architecture so that the request can return without hitting slower I/O devices like databases or disks. We can also increase the number of threads for computation-intensive tasks. However, adding too many threads can deteriorate the performance. We then need to identify the bottlenecks in the system and increase its throughput. Using asynchronous processing can often effectively isolate heavy-lifting components. 3. High Scalability This means a system can quickly and easily extend to accommodate more volume (horizontal scalability) or more functionalities (vertical scalability). Normally we watch the response time to decide if we need to scale the system.

Answer 15

HTTPS: Safeguards your data from eavesdroppers and breaches. Understand how encryption and digital certificates create an impregnable shield. SSL Handshake: Behind the Scenes — Witness the cryptographic protocols that establish a secure connection. Experience the intricate exchange of keys and negotiation. Secure Data Transmission: Navigating the Tunnel — Journey through the encrypted tunnel forged by HTTPS. Learn how your information travels while shielded from cyber threats. HTML's Role: Peek into HTML's role in structuring the web. Uncover how hyperlinks and content come together seamlessly. And why is it called HYPER TEXT. Over to you: In this ever-evolving digital landscape, what emerging technologies do you foresee shaping the future of cybersecurity or the web?

Answer 16

Testing system functionality is a crucial step in software development and engineering processes. It ensures that a system or software application performs as expected, meets user requirements, and operates reliably. Here we delve into the best ways: 1. Unit Testing: Ensures individual code components work correctly in isolation. 2. Integration Testing: Verifies that different system parts function seamlessly together. 3. System Testing: Assesses the entire system's compliance with user requirements and performance. 4. Load Testing: Tests a system's ability to handle high workloads and identifies performance issues. 5. Error Testing: Evaluates how the software handles invalid inputs and error conditions. 6. Test Automation: Automates test case execution for efficiency, repeatability, and error reduction

Answer 17

1. HTTP GET This retrieves a resource from the server. It is idempotent. Multiple identical requests return the same result. 2. HTTP PUT This updates or Creates a resource. It is idempotent. Multiple identical requests will update the same resource. 3. HTTP POST This is used to create new resources. It is not idempotent, making two identical POST will duplicate the resource creation. 4. HTTP DELETE This is used to delete a resource. It is idempotent. Multiple identical requests will delete the same resource. 5. HTTP PATCH The PATCH method applies partial modifications to a resource. 6. HTTP HEAD The HEAD method asks for a response identical to a GET request but without the response body. 7. HTTP CONNECT The CONNECT method establishes a tunnel to the server identified by the target resource. 8. HTTP OPTIONS This describes the communication options for the target resource. 9. HTTP TRACE This performs a message loop-back test along the path to the target resource.

Answer 18

We are dealing with massive amounts of data. Often we need to split data into smaller, more manageable pieces, or “shards”. Here are some of the top data sharding algorithms commonly used: 🔹 Range-Based Sharding This involves partitioning data based on a range of values. For example, customer data can be sharded based on alphabetical order of last names, or transaction data can be sharded based on date ranges. 🔹 Hash-Based Sharding In this method, a hash function is applied to a shard key chosen from the data (like a customer ID or transaction ID). This tends to distribute data more evenly across shards compared to range-based sharding. However, we need to choose a proper hash function to avoid hash collision. 🔹 Consistent Hashing This is an extension of hash-based sharding that reduces the impact of adding or removing shards. It distributes data more evenly and minimizes the amount of data that needs to be relocated when shards are added or removed. 🔹 Virtual Bucket Sharding Data is mapped into virtual buckets, and these buckets are then mapped to physical shards. This two-level mapping allows for more flexible shard management and rebalancing without significant data movement.

Answer 19

For high-scale user-facing systems, high latency is a big loss of revenue. Here are the top strategies to reduce latency: 1 - Database Indexing 2 - Caching 3 - Load Balancing 4 - Content Delivery Network 5 - Async Processing 6 - Data Compression

Answer 20

Load balancers are inherently dynamic and adaptable, designed to efficiently address multiple purposes and use cases in network traffic and server workload management. Let's explore some of the use cases: 1. Failure Handling: Automatically redirects traffic away from malfunctioning elements to maintain continuous service and reduce service interruptions. 2. Instance Health Checks: Continuously evaluates the functionality of instances, directing incoming requests exclusively to those that are fully operational and efficient. 3. Platform Specific Routing:Routes requests from different device types (like mobiles, desktops) to specialized backend systems, providing customized responses based on platform. 4. SSL Termination: Handles the encryption and decryption of SSL traffic, reducing the processing burden on backend infrastructure. 5. Cross Zone Load Balancing: Distributes incoming traffic across various geographic or network zones, increasing the system's resilience and capacity for handling large volumes of requests. 6. User Stickiness: Maintains user session integrity and tailored user interactions by consistently directing requests from specific users to designated backend servers.

Answer 21

The transition from Internet Protocol version 4 (IPv4) to Internet Protocol version 6 (IPv6) is primarily driven by the need for more internet addresses, alongside the desire to streamline certain aspects of network management. 🔹 Format and Length IPv4 uses a 32-bit address format, which is typically displayed as four decimal numbers separated by dots (e.g., 192.168.0. 12). The 32-bit format allows for approximately 4.3 billion unique addresses, a number that is rapidly proving insufficient due to the explosion of internet-connected devices. In contrast, IPv6 utilizes a 128-bit address format, represented by eight groups of four hexadecimal digits separated by colons (e.g., 50B3:F200:0211:AB00:0123:4321:6571:B000). This expansion allows for approximately much more addresses, ensuring the internet's growth can continue unabated. 🔹 Header The IPv4 header is more complex and includes fields such as the header length, service type, total length, identification, flags, fragment offset, time to live (TTL), protocol, header checksum, source and destination IP addresses, and options. IPv6 headers are designed to be simpler and more efficient. The fixed header size is 40 bytes and includes less frequently used fields in optional extension headers. The main fields include version, traffic class, flow label, payload length, next header, hop limit, and source and destination addresses. This simplification helps improve packet processing speeds. 🔹 Translation between IPv4 and IPv6 As the internet transitions from IPv4 to IPv6, mechanisms to allow these protocols to coexist have become essential: - Dual Stack: This technique involves running IPv4 and IPv6 simultaneously on the same network devices. It allows seamless communication in both protocols, depending on the destination address availability and compatibility. The dual stack is considered one of the best approaches for the smooth transition from IPv4 to IPv6.

Answer 22

A deadlock occurs when two or more transactions are waiting for each other to release locks on resources they need to continue processing. This results in a situation where neither transaction can proceed, and they end up waiting indefinitely. 🔹 Coffman Conditions The Coffman conditions, named after Edward G. Coffman, Jr., who first outlined them in 1971, describe four necessary conditions that must be present simultaneously for a deadlock to occur: - Mutual Exclusion - Hold and Wait - No Preemption - Circular Wait 🔹 Deadlock Prevention - Resource ordering: impose a total ordering of all resource types, and require that each process requests resources in a strictly increasing order. - Timeouts: A process that holds resources for too long can be rolled back. - Banker’s Algorithm: A deadlock avoidance algorithm that simulates the allocation of resources to processes and helps in deciding whether it is safe to grant a resource request based on the future availability of resources, thus avoiding unsafe states. 🔹 Deadlock Recovery - Selecting a victim: Most modern Database Management Systems (DBMS) and Operating Systems implement sophisticated algorithms for detecting deadlocks and selecting victims, often allowing customization of the victim selection criteria via configuration settings. The selection can be based on resource utilization, transaction priority, cost of rollback etc. - Rollback: The database may roll back the entire transaction or just enough of it to break the deadlock. Rolled-back transactions can be restarted automatically by the database management system.

Answer 23

Here’s a simple breakdown for both approaches: Session-Based Authentication In this approach, you store the session information in a database or session store and hand over a session ID to the user. Think of it like a passenger getting just the Ticket ID of their flight while all other details are stored in the airline’s database. Here’s how it works: 1 - The user makes a login request and the frontend app sends the request to the backend server. 2 - The backend creates a session using a secret key and stores the data in session storage. 3 - The server sends a cookie back to the client with the unique session ID. 4 - The user makes a new request and the browser sends the session ID along with the request. 5 - The server authenticates the user using the session ID. JWT-Based Authentication In the JWT-based approach, you don’t store the session information in the session store. The entire information is available within the token. Think of it like getting the flight ticket along with all the details available on the ticket but encoded. Here’s how it works: 1 - The user makes a login request and it goes to the backend server. 2 - The server verifies the credentials and issues a JWT. The JWT is signed using a private key and no session storage is involved. 3 - The JWT is passed to the client, either as a cookie or in the response body. Both approaches have their pros and cons but we’ve gone with the cookie approach. 4 - For every subsequent request, the browser sends the cookie with the JWT. 5 - The server verifies the JWT using the secret private key and extracts the user info.

Answer 24

APIs expose business logic and data to external systems, so designing them securely and efficiently is important. 🔹 API key generation We normally generate one unique app ID for each client and generate different pairs of public key (access key) and private key (secret key) to cater to different authorizations. For example, we can generate one pair of keys for read-only access and another pair for read-write access. 🔹 Signature generation Signatures are used to verify the authenticity and integrity of API requests. They are generated using the secret key and typically involve the following steps: - Collect parameters - Create a string to sign Hash the string: Use a cryptographic hash function, like HMAC (Hash-based Message Authentication Code) in combination with SHA-256, to hash the string using the secret key. 🔹 Send the requests When designing an API, deciding what should be included in HTTP request parameters is crucial. Include the following in the request parameters: - Authentication Credentials - Timestamp: To prevent replay attacks. - Request-specific Data: Necessary to process the request, such as user IDs, transaction details, or search queries. - Nonces: Randomly generated strings included in each request to ensure that each request is unique and to prevent replay attacks. 🔹 Security guidelines To safeguard APIs against common vulnerabilities and threats, adhere to these security guidelines.

Answer 25

1. HTTP GET This retrieves a resource from the server. It is idempotent. Multiple identical requests return the same result. 2. HTTP PUT This updates or Creates a resource. It is idempotent. Multiple identical requests will update the same resource. 3. HTTP POST This is used to create new resources. It is not idempotent, making two identical POST will duplicate the resource creation. 4. HTTP DELETE This is used to delete a resource. It is idempotent. Multiple identical requests will delete the same resource. 5. HTTP PATCH The PATCH method applies partial modifications to a resource. 6. HTTP HEAD The HEAD method asks for a response identical to a GET request but without the response body. 7. HTTP CONNECT The CONNECT method establishes a tunnel to the server identified by the target resource. 8. HTTP OPTIONS This describes the communication options for the target resource. 9. HTTP TRACE This performs a message loop-back test along the path to the target resource.

Answer 26

Load balancers are inherently dynamic and adaptable, designed to efficiently address multiple purposes and use cases in network traffic and server workload management. Let's explore some of the use cases: 1. Failure Handling: Automatically redirects traffic away from malfunctioning elements to maintain continuous service and reduce service interruptions. 2. Instance Health Checks: Continuously evaluates the functionality of instances, directing incoming requests exclusively to those that are fully operational and efficient. 3. Platform Specific Routing: Routes requests from different device types (like mobiles, desktops) to specialized backend systems, providing customized responses based on platform. 4. SSL Termination: Handles the encryption and decryption of SSL traffic, reducing the processing burden on backend infrastructure. 5. Cross Zone Load Balancing: Distributes incoming traffic across various geographic or network zones, increasing the system's resilience and capacity for handling large volumes of requests. 6. User Stickiness: Maintains user session integrity and tailored user interactions by consistently directing requests from specific users to designated backend servers.

Answer 27

XSS, a prevalent vulnerability, occurs when malicious scripts are injected into web pages, often through input fields. Check out the diagram below for a deeper dive into how this vulnerability emerges when user input is improperly handled and subsequently returned to the client, leaving systems vulnerable to exploitation. Understanding the distinction between Reflective and Stored XSS is crucial. Reflective XSS involves immediate execution of the injected script, while Stored XSS persists over time, posing long-term threats. Dive into the diagrams for a comprehensive comparison of these attack vectors. Imagine this scenario: A cunning hacker exploits XSS to clandestinely harvest user credentials, such as cookies, from their browser, potentially leading to unauthorized access and data breaches. It's a chilling reality. But fret not! Our flyer also delves into effective mitigation strategies, empowering you to fortify your systems against XSS attacks. From input validation and output encoding to implementing strict Content Security Policies (CSP), we've got you covered.

Answer 28

- The fundamental duo: RAM and ROM - DDR4 and DDR5 - Firmware and BIOS - SRAM and DRAM - HDD, SSD, USB Drive, SD Card

Answer 29

Everything is a compromise. There is no right or wrong design. The diagram below shows some of the most important trade-offs. 🔹 Cost vs. Performance 🔹 Reliability vs. Scalability 🔹 Performance vs. Consistency 🔹 Security vs. Flexibility 🔹 Development Speed vs. Quality

Answer 30

1 - It all starts with CI/CD pipelines that deploy code to the server instances. Tools like Jenkins and GitHub help over here. 2 - The user requests originate from the web browser. After DNS resolution, the requests reach the app servers. 3 - Load balancers and reverse proxies (such as Nginx & HAProxy) distribute user requests evenly across the web application servers. 4 - The requests can also be served by a Content Delivery Network (CDN). 5 - The web app communicates with backend services via APIs. 6 - The backend services interact with database servers or distributed caches to provide the data. 7 - Resource-intensive and long-running tasks are sent to job workers using a job queue. 8 - The full-text search service supports the search functionality. Tools like Elasticsearch and Apache Solr can help here. 9 - Monitoring tools (such as Sentry, Grafana, and Prometheus) store logs and help analyze data to ensure everything works fine. 10 - In case of issues, alerting services notify developers through platforms like Slack for quick resolution.

Answer 31

🔹 TCP/IP Developed by the IETF organization, the TCP/IP protocol is the foundation of the Internet and one of the best-known networking standards. 🔹 HTTP The IETF has also developed the HTTP protocol, which is essential for all web developers. 🔹 SQL Structured Query Language (SQL) is a domain-specific language used to manage data. 🔹 OAuth OAuth (Open Authorization) is an open standard for access delegation commonly used to grant websites or applications limited access to user information without exposing their passwords. 🔹 HTML/CSS With HTML, web pages are rendered uniformly across browsers, which reduces development effort spent on compatibility issues.HTML tags. CSS standards are often used in conjunction with HTML. 🔹 ECMAScript ECMAScript is a standardized scripting language specification that serves as the foundation for several programming languages, the most well-known being JavaScript. 🔹 ISO Date It is common for developers to have problems with inconsistent time formats on a daily basis. ISO 8601 is a date and time format standard developed by the ISO (International Organization for Standardization) to provide a common format for exchanging date and time data across borders, cultures, and industries. 🔹 OpenAPI OpenAPI, also known as the OpenAPI Specification (OAS), is a standardized format for describing and documenting RESTful APIs.

Answer 32

Imagine you have a special box called a JWT. Inside this box, there are three parts: a header, a payload, and a signature. The header is like the label on the outside of the box. It tells us what type of box it is and how it's secured. It's usually written in a format called JSON, which is just a way to organize information using curly braces { } and colons : . The payload is like the actual message or information you want to send. It could be your name, age, or any other data you want to share. It's also written in JSON format, so it's easy to understand and work with. Now, the signature is what makes the JWT secure. It's like a special seal that only the sender knows how to create. The signature is created using a secret code, kind of like a password. This signature ensures that nobody can tamper with the contents of the JWT without the sender knowing about it. When you want to send the JWT to a server, you put the header, payload, and signature inside the box. Then you send it over to the server. The server can easily read the header and payload to understand who you are and what you want to do.

Answer 33

1 - Dockerfile: It contains the instructions to build a Docker image by specifying the base image, dependencies, and run command. 2 - Docker Image: A lightweight, standalone package that includes everything (code, libraries, and dependencies) needed to run your application. Images are built from a Dockerfile and can be versioned. 3 - Docker Container: A running instance of a Docker image. Containers are isolated from each other and the host system, providing a secure and reproducible environment for running your apps. 4 - Docker Registry: A centralized repository for storing and distributing Docker images. For example, Docker Hub is the default public registry but you can also set up private registries. 5 - Docker Volumes: A way to persist data generated by containers. Volumes are outside the container’s file system and can be shared between multiple containers. 6 - Docker Compose: A tool for defining and running multi-container Docker applications, making it easy to manage the entire stack. 7 - Docker Networks: Used to enable communication between containers and the host system. Custom networks can isolate containers or enable selective communication. 8 - Docker CLI: The primary way to interact with Docker, providing commands for building images,running containers, managing volumes, and performing other operations.

Answer 34

🔹Load Balancer: This distributes incoming traffic across multiple backend services. 🔹CDN (Content Delivery Network): CDN is a group of geographically distributed servers that hold static content for faster delivery. The clients look for content in CDN first, then progress to backend services. 🔹API Gateway: This handles incoming requests and routes them to the relevant services. It talks to the identity provider and service discovery. 🔹Identity Provider: This handles authentication and authorization for users. 🔹Service Registry & Discovery: Microservice registration and discovery happen in this component, and the API gateway looks for relevant services in this component to talk to. 🔹Management: This component is responsible for monitoring the services. 🔹Microservices: Microservices are designed and deployed in different domains. Each domain has its database.

Answer 35

Basically, Single Sign-On (SSO) is an authentication scheme. It allows a user to log in to different systems using a single ID. Step 1: A user visits Gmail, or any email service. Gmail finds the user is not logged in and so redirects them to the SSO authentication server, which also finds the user is not logged in. As a result, the user is redirected to the SSO login page, where they enter their login credentials. Steps 2-3: The SSO authentication server validates the credentials, creates the global session for the user, and creates a token. Steps 4-7: Gmail validates the token in the SSO authentication server. The authentication server registers the Gmail system, and returns “valid.” Gmail returns the protected resource to the user. Step 8: From Gmail, the user navigates to another Google-owned website, for example, YouTube. Steps 9-10: YouTube finds the user is not logged in, and then requests authentication. The SSO authentication server finds the user is already logged in and returns the token. Step 11-14: YouTube validates the token in the SSO authentication server. The authentication server registers the YouTube system, and returns “valid.” YouTube returns the protected resource to the user. The process is complete and the user gets back access to their account.

Answer 36

The key features of HTTP2 play a big role in this: 1 - Binary Framing Layer HTTP2 encodes the messages into binary format. This allows the messages into smaller units called frames, which are then sent over the TCP connection, resulting in more efficient processing. 2 - Multiplexing The Binary Framing allows full request and response multiplexing.Clients and servers can interleave frames during transmissions and reassemble them on the other side. 3 - Stream Prioritization With stream prioritization, developers can customize the relative weight of requests or streams to make the server send more frames for higher-priority requests. 4 - Server Push Since HTTP2 allows multiple concurrent responses to a client’s request, a server can send additional resources along with the requested page to the client. 5 - HPACK Header Compression HTTP2 uses a special compression algorithm called HPACK to make the headers smaller for multiple requests, thereby saving bandwidth. Of course, despite these features, HTTP2 can also be slow depending on the exact technical scenario. Therefore, developers need to test and optimize things to maximize the benefits of HTTP2.

Answer 37

Top 6 log parsing commands. 1. GREP GREP searches any given input files, selecting lines that match one or more patterns. 2. CUT CUT cuts out selected portions of each line from each file and writes them to the standard output. 3. SED SED reads the specified files, modifying the input as specified by a list of commands. 4. AWK AWK scans each input file for lines that match any of a set of patterns. 5. SORT SORT sorts text and binary files by lines. 6. UNIQ UNIQ reads the specified input file comparing adjacent lines and writes a copy of each unique input line to the output file. These commands are often used in combination to quickly find useful information from the log files. For example, the below commands list the timestamps (column 2) when there is an exception happening for xxService. grep “xxService” service.log | grep “Exception” | cut -d” “ -f 2

Answer 38

The goal of Netflix is to keep you streaming for as long as possible. But a user’s typical attention span is just 90 seconds. They use EVCache (a distributed key-value store) to reduce latency so that the users don’t lose interest. However, EVCache has multiple use cases at Netflix. 1 - Lookaside Cache When the application needs some data, it first tries the EVCache client and if the data is not in the cache, it goes to the backend service and the Cassandra database to fetch the data. The service also keeps the cache updated for future requests. 2 - Transient Data Store Netflix uses EVCache to keep track of transient data such as playback session information. One application service might start the session while the other may update the session followed by a session closure at the very end. 3 - Primary Store Netflix runs large-scale pre-compute systems every night to compute a brand-new home page for every profile of every user based on watch history and recommendations. All of that data is written into the EVCache cluster from where the online services read the data and build the homepage. 4 - High Volume Data Netflix has data that has a high volume of access and also needs to be highly available. For example, UI strings and translations that are shown on the Netflix home page. A separate process asynchronously computes and publishes the UI string to EVCache from where the application can read it with low latency and high availability.

Answer 39

1 - Indexing: Check the query patterns of your application and create the right indexes. 2 - Materialized Views: Pre-compute complex query results and store them for faster access. 3 - Denormalization: Reduce complex joins to improve query performance. 4 - Vertical Scaling Boost your database server by adding more CPU, RAM, or storage. 5 - Caching Store frequently accessed data in a faster storage layer to reduce database load. 6 - Replication Create replicas of your primary database on different servers for scaling the reads. 7 - Sharding Split your database tables into smaller pieces and spread them across servers. Used for scaling the writes as well as the reads.

Answer 40

Idempotency is essential in various scenarios, particularly where operations might be retried or executed multiple times. Here are the top 6 use cases where idempotency is crucial: 1. RESTful API Requests We need to ensure that retrying an API request does not lead to multiple executions of the same operation. Implement idempotent methods (like PUT and DELETE) to maintain consistent resource states. 2. Payment Processing We need to ensure that customers are not charged multiple times due to retries or network issues. Payment gateways often need to retry transactions; idempotency ensures only one charge is made. 3. Order Management Systems We need to ensure that submitting an order multiple times results in only one order being placed. We design a safe mechanism to prevent duplicate inventory deductions or updates. 4. Database Operations We need to ensure that reapplying a transaction does not change the database state beyond the initial application. 5. User Account Management We need to ensure that retrying a registration request does not create multiple user accounts. Also, we need to ensure that multiple password reset requests result in a single reset action. 6. Distributed Systems and Messaging We need to ensure that reprocessing messages from a queue does not result in duplicate processing. We Implement handlers that can process the same message multiple times without side effects.

Answer 41

In distributed systems and networked applications, retry strategies are crucial for handling transient errors and network instability effectively. The diagram shows 4 common retry strategies. 🔹 Linear Backoff Linear backoff involves waiting for a progressively increasing fixed interval between retry attempts. Advantages: Simple to implement and understand. Disadvantages: May not be ideal under high load or in high-concurrency environments as it could lead to resource contention or "retry storms". 🔹 Linear Jitter Backoff Linear jitter backoff modifies the linear backoff strategy by introducing randomness to the retry intervals. This strategy still increases the delay linearly but adds a random "jitter" to each interval. Advantages: The randomness helps spread out the retry attempts over time, reducing the chance of synchronized retries across instances. Disadvantages: Although better than simple linear backoff, this strategy might still lead to potential issues with synchronized retries as the base interval increases only linearly. 🔹 Exponential Backoff Exponential backoff involves increasing the delay between retries exponentially. The interval might start at 1 second, then increase to 2 seconds, 4 seconds, 8 seconds, and so on, typically up to a maximum delay. This approach is more aggressive in spacing out retries than linear backoff. Advantages: Significantly reduces the load on the system and the likelihood of collision or overlap in retry attempts, making it suitable for high-load environments. Disadvantages: In situations where a quick retry might resolve the issue, this approach can unnecessarily delay the resolution. 🔹 Exponential Jitter Backoff Exponential jitter backoff combines exponential backoff with randomness. After each retry, the backoff interval is exponentially increased, and then a random jitter is applied. The jitter can be either additive (adding a random amount to the exponential delay) or multiplicative (multiplying the exponential delay by a random factor). Advantages: Offers all the benefits of exponential backoff, with the added advantage of reducing retry collisions even further due to the introduction of jitter. Disadvantages: The randomness can sometimes result in longer than necessary delays, especially if the jitter is significant.

Answer 42

These architecture patterns are among the most commonly used in app development, whether on iOS or Android platforms. Developers have introduced them to overcome the limitations of earlier patterns. So, how do they differ? - MVC, the oldest pattern, dates back almost 50 years - Every pattern has a "view" (V) responsible for displaying content and receiving user input - Most patterns include a "model" (M) to manage business data - "Controller," "presenter," and "view-model" are translators that mediate between the view and the model ("entity" in the VIPER pattern) - These translators can be quite complex to write, so various patterns have been proposed to make them more maintainable

Answer 43

In database management, locks are mechanisms that prevent concurrent access to data to ensure data integrity and consistency. Here are the common types of locks used in databases: 1. Shared Lock (S Lock) It allows multiple transactions to read a resource simultaneously but not modify it. Other transactions can also acquire a shared lock on the same resource. 2. Exclusive Lock (X Lock) It allows a transaction to both read and modify a resource. No other transaction can acquire any type of lock on the same resource while an exclusive lock is held. 3. Update Lock (U Lock) It is used to prevent a deadlock scenario when a transaction intends to update a resource. 4. Schema Lock It is used to protect the structure of database objects. 5. Bulk Update Lock (BU Lock) It is used during bulk insert operations to improve performance by reducing the number of locks required. 6. Key-Range Lock It is used in indexed data to prevent phantom reads (inserting new rows into a range that a transaction has already read). 7. Row-Level Lock It locks a specific row in a table, allowing other rows to be accessed concurrently. 8. Page-Level Lock It locks a specific page (a fixed-size block of data) in the database. 9. Table-Level Lock It locks an entire table. This is simple to implement but can reduce concurrency significantly.

Answer 44

We are dealing with massive amounts of data. Often we need to split data into smaller, more manageable pieces, or “shards”. Here are some of the top data sharding algorithms commonly used: 🔹 Range-Based Sharding This involves partitioning data based on a range of values. For example, customer data can be sharded based on alphabetical order of last names, or transaction data can be sharded based on date ranges. 🔹 Hash-Based Sharding In this method, a hash function is applied to a shard key chosen from the data (like a customer ID or transaction ID). This tends to distribute data more evenly across shards compared to range-based sharding. However, we need to choose a proper hash function to avoid hash collision. 🔹 Consistent Hashing This is an extension of hash-based sharding that reduces the impact of adding or removing shards. It distributes data more evenly and minimizes the amount of data that needs to be relocated when shards are added or removed. 🔹 Virtual Bucket Sharding Data is mapped into virtual buckets, and these buckets are then mapped to physical shards. This two-level mapping allows for more flexible shard management and rebalancing without significant data movement.

Answer 45

Pagination is crucial in API design to handle large datasets efficiently and improve performance. Six popular pagination techniques: 🔹 Offset-based Pagination: This technique uses an offset and a limit parameter to define the starting point and the number of records to return. - Example: GET /orders?offset=0&limit=3 - Pros: Simple to implement and understand. - Cons: Can become inefficient for large offsets, as it requires scanning and skipping rows. 🔹 Cursor-based Pagination: This technique uses a cursor (a unique identifier) to mark the position in the dataset. Typically, the cursor is an encoded string that points to a specific record. - Example: GET /orders?cursor=xxx - Pros: More efficient for large datasets, as it doesn't require scanning skipped records. - Cons: Slightly more complex to implement and understand. 🔹 Page-based Pagination: This technique specifies the page number and the size of each page. - Example: GET /items?page=2&size=3 - Pros: Easy to implement and use. - Cons: Similar performance issues as offset-based pagination for large page numbers. 🔹 Keyset-based Pagination: This technique uses a key to filter the dataset, often the primary key or another indexed column. - Example: GET /items?after_id=102&limit=3 - Pros: Efficient for large datasets and avoids performance issues with large offsets. - Cons: Requires a unique and indexed key, and can be complex to implement. 🔹 Time-based Pagination: This technique uses a timestamp or date to paginate through records. - Example: GET /items? start_time=xxx&end_time=yyy - Pros: Useful for datasets ordered by time, ensures no records are missed if new ones are added. - Cons: Requires a reliable and consistent timestamp. 🔹 Hybrid Pagination: This technique combines multiple pagination techniques to leverage their strengths. Example: Combining cursor and time-based pagination for efficient scrolling through time-ordered records. - Example: GET /items?cursor=abc&start_time=xxx&end_time=yyy - Pros: Can offer the best performance and flexibility for complex datasets. - Cons: More complex to implement and requires careful design.

Answer 46

1. Bob enters a URL into the browser and hits Enter. In this example, the URL is composed of 4 parts: 🔹 scheme - 𝒉𝒕𝒕𝒑://. This tells the browser to send a connection to the server using HTTP. 🔹 domain - 𝒆𝒙𝒂𝒎𝒑𝒍𝒆.𝒄𝒐𝒎. This is the domain name of the site. 🔹 path - 𝒑𝒓𝒐𝒅𝒖𝒄𝒕/𝒆𝒍𝒆𝒄𝒕𝒓𝒊𝒄. It is the path on the server to the requested resource: phone. 🔹 resource - 𝒑𝒉𝒐𝒏𝒆. It is the name of the resource Bob wants to visit. 2. The browser looks up the IP address for the domain with a domain name system (DNS) lookup. To make the lookup process fast, data is cached at different layers: browser cache, OS cache, local network cache, and ISP cache. 2.1 If the IP address cannot be found at any of the caches, the browser goes to DNS servers to do a recursive DNS lookup until the IP address is found (this will be covered in another post). 3. Now that we have the IP address of the server, the browser establishes a TCP connection with the server. 4. The browser sends an HTTP request to the server. The request looks like this: 𝘎𝘌𝘛 /𝘱𝘩𝘰𝘯𝘦 𝘏𝘛𝘛𝘗/1.1 𝘏𝘰𝘴𝘵: 𝘦𝘹𝘢𝘮𝘱𝘭𝘦.𝘤𝘰𝘮 5. The server processes the request and sends back the response. For a successful response (the status code is 200). The HTML response might look like this: 𝘏𝘛𝘛𝘗/1.1 200 𝘖𝘒 𝘋𝘢𝘵𝘦: 𝘚𝘶𝘯, 30 𝘑𝘢𝘯 2022 00:01:01 𝘎𝘔𝘛 𝘚𝘦𝘳𝘷𝘦𝘳: 𝘈𝘱𝘢𝘤𝘩𝘦 𝘊𝘰𝘯𝘵𝘦𝘯𝘵-𝘛𝘺𝘱𝘦: 𝘵𝘦𝘹𝘵/𝘩𝘵𝘮𝘭; 𝘤𝘩𝘢𝘳𝘴𝘦𝘵=𝘶𝘵𝘧-8 <𝘩𝘵𝘮𝘭 𝘭𝘢𝘯𝘨="𝘦𝘯"> 𝘏𝘦𝘭𝘭𝘰 𝘸𝘰𝘳𝘭𝘥 6. The browser renders the HTML content.

Answer 47

Do I keep the system available even though the data is incorrect? Or do I wait for the data to become consistent throughout the system, even if it means the system is unavailable in the meantime? This is a classic conundrum faced in distributed systems. It’s the core dilemma that CAP theorem explores CAP Theorem explained The CAP theorem is a fundamental principle in distributed computing that outlines the trade-offs a distributed system must make when dealing with three key properties—consistency, availability, and partition tolerance. CAP theorem asserts that a distributed system cannot simultaneously provide consistency, availability, and partition tolerance. Consistency ensures all nodes display the same data simultaneously, which is crucial for systems that need all clients to receive up-to-date and accurate information. Availability means that every request (read or write) receives a response, even if it’s not the most recent write. The system remains operational and responsive at all times. Partition tolerance refers to the system’s ability to continue operating despite message losses or failure within the system. Given that network partitions are inevitable, systems must choose between consistency and availability. It’s important to note that CAP theorem assumes ideal conditions of 100% availability and 100% consistency. In the real world, it’s not so black and white. The real world is complex, dynamic, and messy, with varying degrees of consistency and availability. While CAP theorem underscores a crucial aspect of system design—balancing trade-offs—the simplistic model can be misleading. It’s best to think of it as a guide or tool rather than a strict rule. Practical Implications and Trade-offs The CAP theorem highlights the need for trade-offs in distributed system design. Different systems must prioritize specific aspects based on their requirements. Consider an online retail store with multiple inventory databases across different locations. Consistency vs Availability Consistency Ensures all customers see the same inventory information. For example, if a customer in San Francisco sees that there are 5 units of a product available, a customer in Sydney will see the same. This prevents overselling but if one database becomes unreachable, the system may deny all sales transactions to maintain consistency, affecting availability. Availability Ensures customers can always place orders, even if the inventory databases are not perfectly synchronized. This means that if the San Francisco database is temporarily unreachable, customers can still place orders based on the Sydney database. This improves customer experience but risks inconsistencies, such as two customers purchasing the same product simultaneously, leading to overselling. The store must decide which aspect is more critical. If preventing overselling is paramount, consistency should be prioritized. If ensuring customers can always place orders is more important, availability should take precedence. Understanding these trade-offs helps us design a system that best meets their operational needs... Modern Interpretations and Applications The principles of CAP theorem remain highly applicable today, as cloud computing, big data, and microservices dominate the tech landscape. Given that modern workloads are highly dynamic, systems in these environments must continually reevaluate the balance between consistency and availability. Adopting adaptable models that offer the best balance between these components in real-time is generally advisable. The CAP theorem continues to serve as a guide for building resilient distributed systems capable of managing unanticipated issues. While it’s a good starting point, it doesn’t provide a complete picture of the trade-offs to consider when designing robust distributed systems. Distributed systems are complex, and consistency and availability are just two qualities to consider when designing a robust system. Final Thoughts The CAP theorem, while simple in its formulation, offers profound insights into the design and operation of distributed systems. It provides a framework that helps us understand the trade-offs involved in creating robust systems.

Answer 48

SQL injection is a type of attack where the attacker runs damaging SQL commands by inserting malicious SQL code into an application input field or URL. You can protect your system from SQL injections by doing the following: 1) Use prepared statements or parameterized queries User input cannot be executed because prepared statements and parameterized queries ensure a distinct separation between user input and SQL code. 2) Validate and clean inputs Use expected formats and constraints to validate user input, and clean inputs to get rid of characters that may be interpreted as SQL code. 3) Follow the least privilege principle Limit the permissions for database accounts used by applications and services to only what is required for their functionality. 4) Set Web Application Firewalls (WAF) By setting up WAFs, common threats and attacks from HTTP/S traffic like SQL injections can be identified and blocked before they ever reach your application.

Answer 49

How data is updated and cleared is a key component of the design of any caching strategy. Here are five of the most popular caching eviction approaches: 🔶 Least Recently Used (LRU) - This strategy deletes the oldest unused data to make room for new content. It operates on the premise that data accessed recently will likely be needed again soon. 🔶 Most Recently Used (MRU) - Contrary to LRU, MRU removes the most recently utilized data first and suits scenarios where once-used data isn’t needed again, like in streaming services. 🔶 Least Frequently Used (LFU) - LFU evicts the least frequently accessed data. It is more precise than LRU but adds complexity due to the need for tracking access frequency. 🔶 Time-To-Live (TTL) - TTL sets a predefined lifespan for stored data, ideal for information that becomes obsolete after a certain time, such as session data. 🔶 Two-Tiered Caching - This complex system divides data between 2 tiers, a high-speed, costly cache for frequently accessed data and a slower, cost-effective cache for less popular data. These strategies are also worth mentioning: 🔹 First in, First Out (FIFO): The oldest data is deleted first. 🔹 Random Replacement (RR): Randomly selects data to be deleted. 🔹 Adaptive Replacement Cache (ARC): Uses a self-tuning algorithm that tracks recency and frequency to determine which data to delete first.

Answer 50

API testing is key to building reliable, secure, and high-performing applications. Here are six essential types: 1) Workflow Testing – Verifies that multiple API calls work together to complete real-world processes, like checkout flows. 2)Performance Testing – Measures speed, responsiveness, and stability under load to uncover bottlenecks. 3) Security Testing – Detects vulnerabilities to prevent unauthorized access and data breaches (e.g., using OWASP guidelines). 4) Data-Driven Testing – Tests the API with various input/output combinations to ensure consistent behavior across scenarios. 5) Endpoint Testing – Ensures individual endpoints return correct data, status codes, and errors. 6) Contract Testing – Confirms that API interactions follow agreed-upon request/response formats and avoid breaking changes.

Answer 51

1 - Collaboration Tools Software development is a social activity. Learn to use collaboration tools like Jira, Confluence, Slack, MS Teams, Zoom, etc. 2 - Programming Languages Pick and master one or two programming languages. Choose from options like Java, Python, JavaScript, C#, Go, etc. 3 - API Development Learn the ins and outs of API Development approaches such as REST, GraphQL, and gRPC. 4 - Web Servers and Hosting Know about web servers as well as cloud platforms like AWS, Azure, GCP, and Kubernetes 5 - Authentication and Testing Learn how to secure your applications with authentication techniques such as JWTs, OAuth2, etc. Also, master testing techniques like TDD, E2E Testing, and Performance Testing 6 - Databases Learn to work with relational (Postgres, MySQL, and SQLite) and non-relational databases (MongoDB, Cassandra, and Redis) 7 - CI/CD Pick tools like GitHub Actions, Jenkins, or CircleCI to learn about continuous integration and continuous delivery. 8 - Data Structures and Algorithms Master the basics of DSA with topics like Big O Notation, Sorting, Trees, and Graphs. 9 - System Design Learn System Design concepts such as Networking, Caching, CDNs, Microservices, Messaging, Load Balancing, Replication, Distributed Systems, etc. 10 - Design patterns Master the application of design patterns such as dependency injection, factory, proxy, observers, and facade. 11 - AI Tools To future-proof your career, learn to leverage AI tools like GitHub Copilot, ChatGPT, Langchain, andPrompt Engineering.

Answer 52

A SQL query executes its statements in the following order: FROM / JOIN WHERE GROUP BY HAVING SELECT DISTINCT ORDER BY LIMIT / OFFSET The techniques you implement at each step help speed up the following steps. This is why it's important to know their execution order. To maximize efficiency, focus on optimizing the steps earlier in the query.

Answer 53

When formulating your branching strategy, take the most relevant features from the strategies below and apply your own set of tweaks. Every project and team has its own unique needs and boundaries, which should be reflected in their Git branching strategy. 🔶 Feature Branching: A popular method where each feature gets its own branch to ensure that changes are isolated and code reviews are simplified. 🔶 Gitflow: has two permanent branches — a production and a pre-production branch, often referred to as the “prod” and “dev” branches. Features, releases, and urgent bug fixes get temporary branches. It’s a great approach for scheduled releases and handling multiple production versions. 🔶 GitLab Flow: A blend of feature and environment-based branching. Changes merge into a main branch, then to branches aligned with the CI/CD environment stages. 🔶 GitHub Flow: Similar to feature branching but with a twist. The main branch is always production-ready, and changes to this branch set off the CI/CD process. 🔶 Trunk-based Development: Branches are short-lived. Changes merge into the main branch within a day or two, and feature flags are used for changes that require more time to complete. This is ideal for large teams with disciplined processes.

Answer 54

HTTP/1.1 (1997) brought key improvements like persistent connections, chunked transfers, and better caching—but suffered from sequential request blocking and reliance on multiple TCP connections. HTTP/2 (2015) introduced multiplexing (multiple requests over one connection), header compression (HPACK), and stream prioritization. However, it still struggled with TCP’s head-of-line (HoL) blocking. HTTP/3 (2022) is built on QUIC, using UDP instead of TCP. It solves HoL blocking by allowing independent streams, offers faster handshakes, requires TLS 1.3 encryption, and supports connection migration across networks. Bottom line: HTTP/2 enhanced TCP, but HTTP/3 moved away from it—built on QUIC over UDP, it delivers faster performance, stronger security, and greater resilience by default.

Answer 55

1) Define the problem Identify the problem’s symptoms, and compare expected versus actual outcomes. Determine its scope, assess its severity and impact, and note steps to reproduce it. This clarity streamlines the troubleshooting process. 2) Reproduce it Reproducing the bug is often the most effective way to pinpoint its cause. However, if this can't be done, try checking the environment where it occurred, search the error message online, assess the system's state at the time, note how often it happens, and identify any recurring patterns. These steps can offer vital clues. 3) Identify the cause Logs are a big help in the debugging process; if they're insufficient, add more logs and reproduce the issue. Some additional strategies are to use debugging tools for insights, test components in smaller chunks, and try commenting out code sections to pinpoint the problem area. 4) Provide a postmortem When a bug's cause is identified and resolved, thoroughly document the issue, the fix, and ways to prevent it in the future. Sharing this knowledge with the team is important to ensure everyone is informed and can benefit from the lessons learned, promoting a proactive approach to future challenges.

Answer 56

As the saying goes, if you fail to plan, you plan to fail. This adage holds a profound truth, especially when it comes to system design. System design provides the blueprint for building an application. Without it, applications often become an unwieldy, costly mess. The goal of system design is to create a dependable, efficient, secure, and user-friendly system that satisfies the requirements of the business and its users. The process involves various steps, which we will dive into below in the general sequence in which they occur. Let's dive in! The Core of System Design System design involves the complex process of planning and structuring software solutions to meet specific goals. It's the art and science of envisioning and defining software architectures, components, modules, interfaces, and data for a system to satisfy specified requirements. This process is iterative, requiring continuous refining and adjustment of the design based on evolving requirements and challenges. Diving Deeper into the System Design Process 1. Requirements analysis The process begins by defining system requirements. This involves understanding the goals, tasks, and constraints of the system. Talking with stakeholders, getting detailed requirements, and setting specific goals are key This phase is especially critical as it lays the foundation for the other stages. 2. High-Level Design Now we focus on the system's overall structure, creating the architectural overview of the system. In this phase, we describe the major components of the system and how they interact with each other. This step creates a basic map of the system, showing its components, the technologies it will use, and how it can grow and remain stable and easy to maintain. 3. Detailed Design With the overall design ready, we move on to the specifics of each part With the overall design of how the entire system works ready, we move on to the detailed specifications of each component. This includes setting up algorithms, data structures, and how each component works, making sure everything fits well together. 4. Interface Design Next up is interface design which involves planning the user interfaces (UIs) and the application programming interfaces (APIs) for smooth interaction between different parts of the system. 5. Database Design After interface design is typically database design. All phases are important, however, this is one of the more critical ones. This phase involves organizing data, designing tables, setting up relationships between them, deciding on indexes, and making sure the data is accurate, fast, and secure. As well as defining how data is going to be stored, accessed, and manipulated. 6. Security Design In this step, we look at one very important element — security. This is where we define how the system will protect data, ensure privacy, and handle potential threats. Things like encryption, authentication, authorization, and vulnerability assessments are all discussed and planned here. 7. Performance Design The main focus of this phase is on the performance criteria listed in the initial requirements analysis. The outcome is a design that meets those requirements. Here, we look at optimizing system responsiveness, throughput, and resource utilization, ensuring the system can handle the expected load and scale gracefully under peak conditions. 8. Error Handling and Logging Failure is going to occur, so it’s important to anticipate and plan for it. At this step, the emphasis is on analyzing potential areas of failure and determining how the system will respond. This includes defining robust error handling mechanisms and logging strategies to diagnose issues, monitor system health, and facilitate recovery from failures. 9. Testability Finally, designing for testability ensures that each component can be verified for correctness. This step involves identifying which components will be tested, how the tests will be carried out, and how the findings will be communicated and used. Wrapping Up System design is not a one-time task, it’s an iterative process. It involves going back and forth between the steps outlined above to refine the solution. Feedback loops and iterative cycles enhance the design’s robustness and adaptability, helping to ensure the system meets requirements.

Answer 57

Polling is a pull-based approach that operates on a 'check-in' system. Clients regularly initiate API calls to the server to inquire about any changes. This process involves the system routinely executing API requests at set intervals to ensure updates are consistently captured and communicated, even though they may not be instantaneous. Webhooks represent a push-based methodology, where notifications are sent from the server only when new data becomes available. This system relies on the server's initiative to send out notifications when there are updates. When this happens, the server dispatches information directly to a predefined webhook URL, with a payload containing details of the update. This mechanism allows for immediate data synchronization without the need for constant API requests. Webhooks provide a more efficient and real-time solution, enabling immediate data synchronization as opposed to the delayed response of polling. However, they do come with the trade-off of increased complexity in setup and maintenance compared with polling.