622 Flashcards
Eight pleasant thoughts
-The network is reliable
-The network is secure
-The network is homogenous
-The topology does not change
-Latency is zero
-Bandwidth is infinite
-Transport cost is zero
-There is one administrator
What is a distributed system?[Tanenbaum]
A collection of independent computers that appears to its users as a single coherent system
Three key characteristics [Tanenbaum]
Multiple machines are autonomous
Software lets users see a single system
System easy to expand without user noticing
What is a distributed system?[Webopedia]
A type of computing in which different components and objects comprising an application can be located on different computers connected to a network.
Key requirement:set of standards that specify how objects communicate with one another(e.g. CORBA, DCOM, REST, …).
[Wikipedia] distributed computing:
decentralized and parallel computing, using two or more computers communicating over a network to accomplish a common objective or task.
Note:The types of hardware, programming languages, operating systems and other resources may vary drastically. It is similar to computer clustering with the main difference being a wide geographic dispersion of the resources.
Challenges of DS
-latency of communication
-coordination
-shared resources and mutual exclusion
-ordering, deadlock and live-lock
-timing
-adaptation to change
-failures, soft faults, and optimization
-service discovery and configuration
-heterogeneity and third-party software
-scalability and evolution
-security and privacy
-trust on machines, software, communications & other users
Advantages of DS
- processing capacity
- fault tolerant, evolving, scalable
-explicit control, preferences
Replicas
Often useful to have same task performed by multiple components so all have to fail for task to fail
What if data must be shared between components?
Often one component is “master” (aka “original” or “authoritative version”)
The other components are copies from the master
This may be apportioned, e.g., a component may be a master for just some portion like “names A-K”
Confusion in counting the number of “replicas”:
Some might not include the “master” in the count of replicas
Some might include the “master” (“replicas of each other”)
Make sure you know if the “master” is included
how to solve for number of replicas
If we assume independence and:
F = Probability that one replica fails in time period, F≠1
n = (natural) number of components (e.g., replicas including master). Thus F^n is the probability all n will fail simultaneously
G = Goal, permitted probability of total system failure where all n replicas fail (including original)
F^n ≤ G or
n ≥ (log G)/(log F)
Independence assumption
These calculations assume independent failures
Reasonable model for many hardware failures
Software failures often not independent
Knight & Leveson [1986] found via experiment that software faults are not independent
Thus “N-version programming” doesn’t lead to the reliability increase you might predict
It can be helpful, but less than you’d think
Caching: Special case of replication
Make cop(ies) of a resource (data)
Often happens on demand
Other replication approaches often planned & executed in advance
Challenges of DSexample: replication has downsides
buy more hardware
administration costs
software upgrades
load balancing
performance overhead
more complex software
consistency problems
sometimes tolerable
hiding access
hide differences in data representation and how a resource is accessed
(conversion of complex fortmats. latency vs fidelity of access)
hiding location
hide where a resource is located (trusted hosts, different performance, difference capabilities and network access)
migration
hide that a resource may move to another location (trusted hosts, different performance, difference capabilities and network access)
relocation
hide that a resource may be moved to another location while in use (trusted hosts, different performance, difference capabilities and network access)
replication
hide the fact that several copies of a resource exist (select server based on QoS)
concurrency
hide that a resource may be shared by several competitive users(cannot hide sharing of resources: they’re consumed , data is modified by others)
failure
hide the failure and recovery of a resource(unexplained behavior)
persistence
hide whether a (software) resource is in memory or on disk(someone needs to decide whether an object is persistent and commit it to disk)
awareness and adaptation
separate decisions from (controllable) mechanisms
Some measures (per Neumann) for system scaleability
Size
Users & resources
Geographical
May lie far apart
Administrative
May span many
independent
administrative
organizations
Decentralized algorithms
No machine has complete information about the system state.
Machines make decisions based only on local information.
Failure of one machine does not ruin the algorithm.
There is no implicit assumption that a global clock exists
Asynchronous communication
Hiding communication latencies important for geographical scaleability
Max speed is speed of light in vacuum (~3.00×108 m/s)
Information transfer through material normally less
Physical components have other performance latencies
Software takes time to execute once it receives data
Sending information and waiting for reply is synchronous communication
Alternative: Asynchronous – send information, don’t wait for reply
Moving location of execution
E.G., when checking form inputs, consider two options:
Send each input to server & wait for reply – maybe long delay
Could move checking code to client
Now check response can be immediate (once code is there)
Can be special-case (e.g., HTML5 form validators)
Can be general (e.g., a general execution engine like Javascript)
Beware of security ramifications
Often two sides (e.g., client & server) cross trust boundary
Security checks must often be redone on server in many cases
Server can’t trust client
Many checks are done on both client (for speed) and server (for security)
Client must often check the data it’s asked to execute (especially if it’s a full language like Javascript)
Client can’t trust server
Cloud computing
Clouds widely used, often misunderstood
Clouds often cheaper (where appropriate), many variations
Decisions to use cloud (and how) impact security
NIST Definition of Cloud Computing (NIST SP 800-145):
Cloud computing is “a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.”
Five essential characteristics: On-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service
Virtualization is common, but not required, to be a cloud
Book makes this (common) mistake
Cloud service models
Infrastructure as a Service (IaaS): “consumer [can] deploy and run arbitrary software [including] operating systems and applications….”
Platform as a Service (PaaS): “consumer [can deploy] consumer-created or acquired applications…” [on top of provided platform]
Software as a Service (SaaS): “consumer [can] use the provider’s applications running on a cloud infrastructure…”
OSI 7 layer model (Open Systems Interconnection)
physical, data link, network, transport, session, presentation, application
physical
specifies: pin layout, voltages, modulation
does: establish & terminate access to medium,flow control, contention resolution
at this level: hubs, repeaters,network adapters
data link
specifies: how to transfer data in a LAN
does: detect and correct errors
at this level: MAC addresses (flat, HW-based)
network
specifies: how to transfer data sequences across LANs (e.g., IP)
does: routing
at this level: hierarchical address scheme,routers, bridges & switches
IP Service Model
Connectionless (datagram/packet-based)
Best-effort delivery (unreliable service)
packets are lost
packets are delivered out of order
duplicate copies of a packet are delivered
packets can be delayed for a long time
Datagram format
Datagram forwarding
Strategy
every datagram contains destination’s address
if directly connected to destination network, then forward to host
if not directly connected to destination network, then forward to some router
forwarding table maps network number into next hop
each host has a default router
each router maintains a forwarding table
Forwarding Tables
Suppose there are n possible destinations, how many bits are needed to represent addresses in a routing table?
log2n
So, we need to store and search n * log2n bits in routing tables?
We’re smarter than that!
Global Addresses
Globally unique,
hierarchical: network+host
Dot Notation
10.3.2.4
128.96.33.81
192.12.69.77
Transport
specifies: reliable transference of data(e.g., TCP, UDP)
does: flow control, segmentation, error control,retransmission
UDP
(User Datagram Protocol)
connectionless - sends independent packets of data, called datagrams, from one computer to another with no guarantees about arrival
each time a datagram is sent, the local and receiving socket address need to be sent as well
TCP
(Transmission Control Protocol)
connection-oriented - provides a reliable flow of data between two computers: data sent from one end of the connection gets to the other end in the same order
in order to communicate using TCP protocol, a connection must first be established between the pair of sockets
once two sockets have been connected, they can be used to transmit data in both (or either one of the) directions
Overhead
UDP - every time a datagram is sent, the local and receiving socket address need to be sent along with it
TCP - a connection must be established before communications between the pair of sockets start (i.e. there is a connection setup time in TCP)
Packet size
UDP - there is a size limit of 64 kilobytes per datagram
TCP - there is no limit; the pair of sockets behaves like streams
reliability
UDP - there is no guarantee that the sent datagrams will be received in the same order by the receiving socket
TCP - it is guaranteed that the sent packets will be received in the order in which they were sent
which protocol to use?`
TCP - useful when indefinite amount of data need to be transferred ‘in order’ and reliably
UDP - useful when data transfer should not be slowed down by the extra overhead of the reliable connection
session
specifies: establishing long lived connections
does: checkpointing, adjournment, restart
presentation
specifies: data formats and transformation(e.g., MIME)
does: serialization, compression, encryption, encoding transformation (EBCDIC/ASCII)
application
specifies: application-specific protocols(e.g., http, smtp, ftp, telnet)
does: support app-specific functionality
goal of the OSI
separation of concerns enables good implementationat each level
each layer is independentof the ones on top
layer n depends on the spec of n-1, but not on its implementation/manufacturer
port
Generally, a computer has a single physical connection to the network
this connection is identified by the computer’s 32-bit IP address
all data destined for a particular computer arrives through this connection
TCP and UDP use ports to identify a particular process/application
port = abstract destination point at a particular host
each port is identified by a positive 16-bit number, in the range 0 - 65,535
port numbers 0 - 1023 are reserved for well-known services (HTTP - 80, telnet – 23)
socket
basic abstraction for network communication
“end-point of communication” uniquely identified with IP address and port
example: Socket MyClient = new Socket(“Machine name”, PortNumber);
gives a file-system like abstraction to the capabilities of the network
two end-points communicate by “writing” into and “reading” out of socket
there are two types of transport via sockets
reliable, byte-stream oriented unreliable datagram
socket programming with TCP
Server Side:
server runs on a specific computer and has a socket bound to a specific port number
server listens to the socket for a client to make a connection request
Client Side:
client tries to rendezvous with the server on the server’s machine and port
Server Side:
the server accepts the connection by creating a new socket bound to a different port
Client Side:
if the connection is accepted, the client uses the new socket to communicate with the server
Socket programming with UDP
All clients use the same socket to communicate with the server
Packets of data (datagrams) are exchanged
No new sockets need to be created
C- vs. Java- socket programming
Java keeps all the socket complexity “under the cover”
It does not expose the full range of socket possibilities
But, it enables sockets to be opened/used as easily as a file would be opened/used
By using the java.net.Socket class instead of relying on native code, Java programs can communicate over the network in a platform-independent fashion
Java socket programming
all classes related to sockets are in java.net package
Socket class - implements client sockets (also called just “sockets”)
ServerSocket class - implements server sockets
A server socket waits for requests to come in over the network. It performs some operation based on that request, and then possibly returns a result to the requester.
DatagramSocket class - socket for sending and receiving datagram packets
DatagramPacket class - represents a datagram packet
Datagram packets are used to implement a connectionless packet delivery service. Multiple packets sent from one machine to another might be routed differently, and might arrive in any order.
InetAddress class - represents an Internet Protocol (IP) address
MulticastSocket class - useful for sending and receiving IP multicast packets.
A MulticastSocket is a (UDP) DatagramSocket, with additional capabilities for joining “groups” of other multicast hosts on the internet. A multicast group is specified by a class D IP address.
what does middleware offer?
conceptual model for communication
different styles have different data sharing assumptions
RPC
(address space or memory)
RMI
object refs (middleware)
messages
data store -> files/objects persistent store
data stream -> and <- data store/source
read slide 49-60 in lecture 2
ok
RPC is implemented by
sending messages
where to send RPC messages?
hardwired for fixed deployment
some RPC environments support dynamic binding (more to come during the lecture on Service Discovery)
solution to object refs
increase granularity from bytes to objects
both local objects and references to remote objects are passed by value (serialization)
the result of the called method is also serialized and passed back to the caller
difference between RMI and RPC?
RMI:
doesn’t try to hide distribution in the language:remote objects are declared “remote”
marshalling is simplified
by passing by value only(object references can be used in nested RMIs)
(in Java) by having JVMs hide platform dependencies in data representation
serialization could be much heavier by having to pass the code for the objects with every call, but that can be avoided by passing URLs for downloading the code, rather than the code itself
reasons to escape the call return style
no result needs to be returned
a server may not be availableat the time of the request
make the client more responsiveto other events/user
allow any component to initiate communication
some middleware push the envelope
dealing with errors:idempotent, at-least-once, at-most-once…
the promised simplicity of procedure calling sometimes hinders more sophisticated solutions
when to use call return style
the server is ready to process each request
components and network are mostly reliable
not many concurrent events in the caller:it is fine to block the caller
one component (client) has the initiative,others (servers) wait for requests
what is needed for RMI
Java makes RMI (Remote Method Invocation) fairly easy, but there are some extra steps
To send a message to a remote “server object,”
The “client object” has to find the object
Do this by looking it up in a registry
The client object then has to marshal the parameters (prepare them for transmission)
Java requires Serializable parameters
The server object has to unmarshal its parameters, do its computation, and marshal its response
The client object has to unmarshal the response
remote object
an object on another computer
client object
object making the request
server object
object receiving the request((can easily trade roles with the client object)
rmiregistry
special server that looks up objects by name
rmic
special compiler for creating stub (client) and skeleton (server) classes
processes for RMI
The Client
The Server
The Object Registry, rmiregistry, which is like a DNS service for objects
You also need TCP/IP
interfaces
Interfaces define behavior
Classes define implementation
Therefore,
In order to use a remote object, the client must know its behavior (interface), but does not need to know its implementation (class)
In order to provide an object, the server must know both its interface (behavior) and its class (implementation)
In short,
The interface must be available to both client and server
The class should only be on the server
remote class
one whose instances can be accessed remotely On the computer where it is defined, instances of this class can be accessed just like any other object
On other computers, the remote object can be accessed via object handles
serializable class
one whose instances can be marshaled (turned into a linear sequence of bits)
Serializable objects can be transmitted from one computer to another
conditions for serializability
If an object is to be serialized:
The class must be declared as public
The class must implement Serializable
All fields of the class must be serializable: either primitive types or serializable objects
parts of remote class
The interface (used by both client and server):
Must be public
Must extend the interface java.rmi.Remote
Every method in the interface must declare that it throws java.rmi.RemoteException (other exceptions may also be thrown)
The class itself (used only by the server):
Must implement a Remote interface
Should extend java.rmi.server.UnicastRemoteObject
May have locally accessible methods that are not in its Remote interface
remote object
lives on another computer (like server)
You can send messages to a Remote object and get responses back from the object
All you need to know about the Remote object is its interface
Remote objects don’t pose much of a security issue
serializable object
You can transmit a copy of a Serializable object between computers
The receiving object needs to know how the object is implemented; it needs the class as well as the interface
There is a way to transmit the class definition
Accepting classes does pose a security issue
server class
The class that defines the server object should extend UnicastRemoteObject
This makes a connection with exactly one other computer
If you must extend some other class, you can use exportObject() instead
Sun does not provide a MulticastRemoteObject class
The server class needs to register its server object:
String url = “rmi://” + host + “:” + port + “/” + objectName;
The default port is 1099
Naming.rebind(url, object);
Every remotely available method must throw a RemoteException. Why?
rmic
The class that implements the remote object should be compiled as usual
Then, it should be compiled with rmic:
rmic Hello
This will generate files Hello_Stub.class and Hello_Skel.class
These classes do the actual communication
The “Stub” class must be copied to the client area
The “Skel” was needed in SDK 1.1 but is no longer necessary
overlay networks
network/transport support for multicast
hasn’t worked well in practice
overlay networks at application layer!
result is a logical network built on top of a (probabaly different) network
new network optimized for application
but may have performance issues due to underlying network
distributed solution
tree network approach
function to map topics to nodes
identifies unique root node for a given topic
follow() request follows tree
if node has not seen request
become a “forwarder”
make sender your “child” for that request
forward request to next node in tree
if node has seen request
make sender your “child” for that request
send() request follows topic tree
mesh network “epidemic” approach
node states: infected, susceptible, removed
goal is to become infected!
P picks a random Q
push: P updates pushed to Q // not so good…
pull: Q updates pulled to P
push-pull: both
gossip adaptation
less interest if receiver already has update
removing data
suppose an update is deleted
what happens when old copy is found?
distributed systems are different!
notion of a death certificate
“that update doesn’t matter anymore”
have to keep that update
how long?
persistence
once sent, messages endure in the system, regardless of the sender remaining activeand the recipient being available