TM - Distributed Systems Flashcards
What is a Distributed System
Several Distributed components under a single system.
Features of a Distributed System
Can communicate with a message. Typically executed concurrently and autonomously. Coordinated through a computer network. Physically close or geographically distant. Memory can be shared or individual. Communication slow and unreliable.
Examples of a Distributed System
The WWW, Online Games, An Office Network.
Why build a Distributed System
Some problems are distributed by nature. Cost, Performance, Availability (Fault tolerant), Physical/Geographic Reasons.
Multithreading
Multiple threads of execution in a single app. Threads =/= cores.
Example of why use multithreading
UI apps can block while waiting for user input, this blocked work can execute on another thread.
Also video games use multithreading to render frames.
Fork-Join Model
Parallel Programming Paradigm. Threads can be executed separately but logic is not always independent and can cause interleavings.
Fork (Fork-Join)
Divides the task into smaller subtasks, creating parallel threads or processes to handle them
Join (Fork-Join)
Gathers results from parallel threads or processes and combines them for the final result.
Restricting possible interleavings
Race Condition. Atomic Actions. Critical Sections. Mutual Exclusion.
Race Condition
Occurs when the app behaviour depends on the timing of execution sequences in a multithreaded environment.
Atomic Action
An action to be executed as one unit
Critical Section
Group of instructions to be executed atomically. Executed by a single process at a time (Mutually exclusive).
Mutual Exclusion
One thread of execution never enters its critical section at the same time that another thread in execution is entering its critical section.
Lock
A synchronization tool used to control resource access from multiple threads of execution. Relies on using lock() and unlock().
Issues with Lock
Deadlock, Livelock, Starvation
Deadlock
No process can progress because each is waiting for a resource held by another.
Livelock
No process can progress because process is continuously changing state in response to changes in other processes without doing useful work.
Starvation
Some process cannot get its chance to progress (overlooked by scheduler due to bad luck or low priority).
Creating a Parallel Program (4 Steps)
- Decompose computation into tasks
- Assign tasks to processes
- Orchestrate data access, communicate, synchronize
- Map processes to processors
Speedup
improvement in the overall performance or execution time of a task when processing is distributed
Reasons against programs being able to execute in parallel
Performing Initialization
Communicating amongst processes
Resource Contention
Load Imbalance
Client-Server Pattern
Server running program waiting for client request.
Client process submitting requests and getting response from server.
Both can work on a single machine or a distributed system.
Two Main Classes of Servers
Iterative
Concurrent
Iterative Class of Server
Clients are blocked while a client is being served
Concurrent Class of Server
Time slicing applied by CPU, each processor given time to a client.
Client Server Challenges
Interoperability on development platforms.
Address space and referencing remote objects.
Java remote method invocation
Allows object to invoke a method from another JVM. Limited in cross platform combatibility as only works from JVM to JVM
COM/OLE
Component Object Model/Object Linking and Embedding.
Standard interface for Microsoft software components. COM is base of OLE.
Web Services
Use of existing infrastructure to enable development of client apps composed of web services (i.e services of remote objects) implemented on remote hosts.
Remote Procedure Call (RPC)
Procedure executed in a different address space than the one invoking it (used in client-server).
Execution order
Synchronous (Thread executed block until it’s complete)
Asynchronous (Thread executing current task proceeds with execution even if task is not complete). Completion Notified through call block.
Microservices architecture
Organize around business processes. Decentralized. Infrastructure support + Continuous Delivery. Operate despite failure, not necessarily innovative. Challenging to understand complex sequences across certain microservices.
Serialization
Transform state of objects into stream of bytes for consumption such that it can be rebuilt as an object with the same stream.
Formats of serialization
XML - human readable don’t need to know schema + more verbose than JSON
JSON - human readable don’t need to know schema + less verbose than XML
Protobuf, Thrift.. - Not human readable, smaller output, faster processing, schema known in advance.
Multiple clients working with same resource
Stale Data, Pessimistic Locking, Optimistic Locking
Stale Data
Data which has changed since being retrieved by the current process
Pessimistic Locking
Resource locked as soon as it is accessed and released as soon as changes committed. Prevents conflicts.
Optimistic Locking
Resources can be read and changed freely. Check for conflict when committing changed result and act according to specified conflict resolution protocol.
Avoids overhead of locking resource for a long period of time.
