Distributed Filesystems

Question 1

What does a filesystem keep track of?

Answer 1

Files: where the data is actually stored
Directories: Groups of files
Metadata: Information on the format and permissions related to a file

Question 2

What is the primary job of a filesystem?

Answer 2

To make sure data is always accessible and intact.

Question 3

What is the storage model of the Google Filesystem

Answer 3

• A single file can contain many objects
• Files are divided into fixed chunks with unique IDs
  > Disk seek time small compared to transfer time
  > A file can be larger than a node’s disk space
  > Fixed size makes allocation computations easy
• Files are replicated across chunk servers
• The master maintains all filesystem metadata
• Chunkservers store chunks on local disks as linux files
• Neither client nor chunkserver cache file data

Question 4

How do reads work on the Google Filesystem

Answer 4

• Client sends read request to GFS master
• Master replies with chunk handle and locations
• Client caches metadata
• Client sends a data request to one of the replicas
• The corresponding chunk server returns requested data

Question 5

How do writes work on the Google Filesystem?

Answer 5

• The client sends a request to the master to allocate the primary replica chunkserver
• The master sends the client the location of the chunkserver replicas and primary replica
• The client sends the write data to all the replicas’ chunk server buffer
• Once the replicas receive the data, the client tells the primary replica to begin the write
• The primary replica writes the data to the appropriate chunks
• The secondary replica completes the write function and reports back to the primary
• The primary sends the confirmation to the client

Question 6

Does the master keep a persistent record of chunk locations? If not, what does it do?

Answer 6

It queries the chunk servers at startup and is updated by periodic polling.

Question 7

What happens when a node fails in GFS?

Answer 7

• If it is a master, it has to start up somewhere else
• If it is a chunkserver, it just restarts

Question 8

How does the master know that a chunk he intends to use is up to date?

Answer 8

The master maintins a chunk version number to distinguish. Before an operation on a chunk, the master ensures version is advanced.

Question 9

How consistent is GFS?

Answer 9

It has a relaxed consistency model.