40. BACKUP AND RECOVERY METHODS

Question 1

Learning Objectives

By the end of this chapter, you should be able to:

Answer 1

• Identify and prioritize data that needs backup.
• ​Employ different kinds of backup methods, depending on the situation.
• Establish efficient backup and restore strategies.
• Use different backup utilities, such as cpio, tar, gzip, bzip2, xz, dd, and rsync.
• Describe the two most well-known backup programs, Amanda and Bacula.

Question 2

Why Backups?

Name a few reasons to back up data?

Answer 2

1. Data is valuable
2. Hardware fails
3. Software fails
4. People make mistakes
5. Malicious people can cause deliberate damage
6. Unexplained events happen
7. Rewinds can be useful

Question 3

What Needs Backup?

Some data is critical for backup, some less critical, and some never needs saving.

What data should be backed up?

Answer 3

Definitely yes

• The following data should always be backed up:
  
  • Business-related data
  • System configuration files
  • User files (usually under /home)

Maybe

• Spooling directories (for printing, mail, etc.)
• Logging files (found in /var/log, and elsewhere)

Probably not

• Software that can easily be re-installed; on a well-managed system, this should be almost everything
• The /tmp directory, because its contents are indeed supposed to be only temporary

Definitely not

• Pseudo-filesystems such as /proc, /dev and /sys
• Any swap partitions or files

Obviously, files essential to your organization require backup. Configuration files may change frequently, and along with individual user’s files, require backup as well.

Logging files can be important if you have to investigate your system’s history, which can be particularly important for detecting intrusions and other security violations.

You don’t have to back up anything that can easily be re-installed. Also, the swap partitions (or files) and /proc filesystems are generally not useful or necessary to backup, since the data in these areas is basically temporary (just like in the /tmp directory).

Question 4

Backup vs. Archive

All backup media have a finite lifetime before becoming unreadable. The conventional estimates are listed below:

• Magnetic Tapes: 10-30 years
• CDs and DVDs: 3-10 years
• Hard Disks: 2-5 years.
• Lifetime is very sensitive to:

Environmental conditions (temperature, humidity, etc.)

• Quality of media
• Having working software that can read data on current operating systems and hardware.
• Lifetime is sufficient for backup, but not for permanent digital archiving.

For lifetimes longer than the usual backup timescales, data can be preserved using multiple copies, plus copying over to newer media from time to time.

For very long times (i.e., many decades, centuries, etc.), standard methods do not work easily, as everything can go obsolete: hardware, software and document format, media.

None of the inexpensive digital formats can actually compete with paper and film for long periods of time (if they are properly stored and continuously cared for - like wine).

This is a problem serious people think about and there should be good solutions available before all is lost.

Question 5

Tape Drives

Tape drives are not as common as they used to be. They are relatively slow and permit only sequential access. On any modern setup, they are rarely used for primary backup. They are sometimes used for off-site storage for archival purposes, for long time reference. However, magnetic tape drives always have only a finite lifetime without physical degradation and loss of data.

Modern tape drives are usually of the LTO (Linear Tape Open) variety, whose first versions appeared in the late 1990s as an open standards alternative; early formats were mostly proprietary. Early versions held up to 100 GB; newer versions can hold 2.5 TB or more in a cartridge of the same size.

Day-to-day backups are usually done with some form of NAS (Network Attached Storage) or with cloud-based solutions, making new tape-based installations less and less attractive. However, they can still be found, and system administrators may be required to deal with them.

In what follows, we will try not to focus on particular physical forms for the backup media, and will speak more abstractly.

Question 6

Backup Methods

What are the different types of backs?

You should never have all backups residing in the same physical location as the systems being protected. Otherwise, fire or other physical damage could lead to a total loss. In the past, this usually meant physically transporting magnetic tapes to a secure location. Today, this is more likely to mean transferring backup files over the Internet to alternative physical locations. Obviously, this has to be done in a secure way, using encryption and other security precautions as is appropriate.

Answer 6

• Full
  
  • Backup for all files on the system.
• Incremental
  
  • Backup for all files that have changed since the last incremental or full backup.
• Differental
  
  • Backup for all files that have changed since the last full backup.
• Multiple level incremental
  
  • Backup for all files that have changed since the previous backup at the same or a previous level.
• User
  
  • Backup only for files in a specific user’s directory.

Question 7

Backup Methods

Explain the full backup method

Answer 7

Backup for all files on the system.

Question 8

Backup Methods

Explain the Incremental backup method

Answer 8

Backup for all files that have changed since the last incremental or full backup.

Question 9

Backup Methods

Explain the Differential backup method

Answer 9

Backup for all files that have changed since the last full backup.

Question 10

Backup Methods

Explain the Multiple level incremental backup method

Answer 10

Backup for all files that have changed since the previous backup at the same or a previous level.

Question 11

Backup Methods

Explain the User backup method

Answer 11

Backup only for files in a specific user’s directory.

Question 12

Backup Strategies

We should note that backup methods are useless without associated restore methods. You have to take into account the robustness, clarity and ease of both directions when selecting strategies.

The simplest backup scheme is to do a full backup of everything once, and then perform incremental backups of everything that subsequently changes. While full backups can take a lot of time, restoring from incremental backups can be more difficult and time consuming. Thus, you can use a mix of both to optimize time and effort.

An example of one useful strategy involving tapes (you can easily substitute other media in the description):

• Use tape 1 for a full backup on Friday.
• Use tapes 2-5 for incremental backups on Monday-Thursday.
• Use tape 6 for full backup on second Friday.
• Use tapes 2-5 for incremental backups on second Monday-Thursday.
• Do not overwrite tape 1 until completion of full backup on tape 6.
• After full backup to tape 6, move tape 1 to external location for disaster recovery.
• For next full backup (next Friday) get tape 1 and exchange for tape 6.

A good rule of thumb is to have at least two weeks of backups available.

Question 13

Some Backup Related Utilities

Name a few of the backup utilties

Answer 13

• cpio and tar
  
  • cpio and tar create and extract archives of files.
• gzip, bzip2, and xz
  
  • The archives are often compressed with gzip, bzip2, or xz. The archive file may be written to disk, magnetic tape, or any other device which can hold files. Archives are very useful for transferring files from one filesystem or machine to another.
• dd
  
  • This powerful utility is often used to transfer raw data between media. It can copy entire partitions or entire disks.
• rsync
  
  • This powerful utility can synchronize directory subtrees or entire filesystems across a network, or between different filesystem locations on a local machine.
• dump and restore
  
  • These ancient utilities were designed specifically for backups. They read from the filesystem directly (which is more efficient). However, they must be restored only on the same filesystem type that they came from. There are newer alternatives.
• mt
  
  • This utility is useful for querying and positioning tapes before performing backups and restores.

Question 14

Some Backup Related Utilities

What is the cpio and tar back up utilities used for?

Answer 14

cpio and tar create and extract archives of files.

Question 15

Some Backup Related Utilities

What is the gzip, bzip2, and xz back up utilities used for?

Answer 15

The archives are often compressed with gzip, bzip2, or xz. The archive file may be written to disk, magnetic tape, or any other device which can hold files. Archives are very useful for transferring files from one filesystem or machine to another.

Question 16

Some Backup Related Utilities

What is the dd back up utilities used for?

Answer 16

This powerful utility is often used to transfer raw data between media. It can copy entire partitions or entire disks.

Question 17

Some Backup Related Utilities

What is the rsync back up utilities used for?

Answer 17

This powerful utility can synchronize directory subtrees or entire filesystems across a network, or between different filesystem locations on a local machine.

Question 18

Some Backup Related Utilities

What is the dump and restore back up utilities used for?

Answer 18

These ancient utilities were designed specifically for backups. They read from the filesystem directly (which is more efficient). However, they must be restored only on the same filesystem type that they came from. There are newer alternatives.

Question 19

Some Backup Related Utilities

What is the mt back up utilities used for?

Answer 19

This utility is useful for querying and positioning tapes before performing backups and restores.

Question 20

Using tar for Backups

tar is easy to use:

When creating a tar archive, for each directory given as an argument, ___.

Answer 20

all files and subdirectories will be included in the archive

Question 21

Using tar for Backups

tar is easy to use:

When restoring, __.

Answer 21

it reconstitutes directories as necessary

Question 22

Using tar for Backups

tar is easy to use:

It even has a –newer option that lets you do ___.

Answer 22

• incremental backups

Question 23

Using tar for Backups

tar is easy to use:

The version of tar used in Linux can also handle backups that do not fit on one ___ or ___.

Answer 23

• tape
• whatever device you use

Question 24

Using tar for Backups

What command option with tar is used to create a archive?

Answer 24

• -c or just c

Examples:

• $ tar cvf /dev/st0 /root
• $ tar -cvf /dev/st0 /root

Question 25

Using tar for Backups

What is the command option with tar to create with multi-volume option?

You will be prompted to put the next tape when needed.

Answer 25

• -M

Examples:

• $ tar -cMf /dev/st0 /root

Question 26

Using tar for Backups

What command option with tar verifies files with the compare option?

After you make a backup, you can make sure that it is complete and correct using the above verification option.

Answer 26

• -d or –compare

Examples:

• $ tar –compare –verbose –file /dev/st0
  $ tar -dvf /dev/st0

Question 27

Using tar for Backups

What command option with tar allows you to specify a device or file?

Answer 27

• -f or –file option

Question 28

Using tar for Backups

By default, tar will ___ include all subdirectories in the archive.

Answer 28

• recursively

Question 29

Using tar for Backups

When you create an archive, tar prints a message about removing ___ from the absolute path name. While this allows you to restore the files anywhere, the default behavior can be modified.

Answer 29

• leading slashes

Question 30

Using tar for Backups

Most tar options can be given in ___ with one dash, or ___ with two: -c is completely equivalent to –create. Also note that you can ___ (when using the short notation), so that you don’t have to type every dash.

Answer 30

• short form
• long form
• combine options

Question 31

Using tar for Backups

Furthermore, single-dashed tar options can be used with or ___ dashes.

Answer 31

• without

Examples:

$ tar cvf file.tar dir1

has the same result as

$ tar -cvf file.tar dir1

Question 32

Using tar for Restoring Files

The ___ or ___ option extracts files from an archive, all by default. You can narrow the file extraction list by specifying ___. If a directory is specified, all included files and subdirectories are also extracted.

Answer 32

• -x
• –extract
• only particular files

Examples:

Extract from an archive:

$ tar -xpvf /dev/st0
$ tar xpvf /dev/st0

Specify only specific files to restore:

$ tar xvf /dev/st0 somefile

Question 33

Using tar for Restoring Files

The ___ or ___ options ensures files are restores with their original permissions.

Answer 33

• -p
• –same-permissions

Examples:

Extract from an archive:

$ tar –extract –same-permissions –verbose –file /dev/st0

Question 34

Using tar for Restoring Files

The ___ or ___ option lists, but does not extract, the files in the archive.

Answer 34

• -t
• –list

Examples:

List the contents of a tar backup:

• $ tar –list –file /dev/st0
• $ tar -tf /dev/st0

Question 35

Incremental Backups with tar

You can do an incremental backup with tar using the ___ (or the equivalent ___), or the –after-date options. Either option requires specifying either ___ or a ___.

Answer 35

• -N
• –newer
• a date
• qualified (reference) file name

Example:

$ tar –create –newer ‘2011-12-1’ -vzf backup1.tgz /var/tmp
$ tar –create –after-date‘2011-12-1’ -vzf backup1.tgz /var/tmp

Either form creates a backup archive of all files in /var/tmp which were modified after December 1, 2011.

!!When followed by an option line –newer you must use the dash in options like -vzf, or tar will get confused. This kind of option specification confusion sometimes occurs with old UNIX utilities like ps and tar with complicated histories involving different families of UNIX

Question 36

Incremental Backups with tar

Because tar only looks at a ___, it does not consider any other changes to the file, such as ___ or ___. To include files with these changes in the incremental backup, use ___ and create a list of files to be backed up.

Answer 36

• file’s date
• permissions
• file name
• find

Question 37

Archive Compression Methods

It is often desired to compress files to save disk space and/or network transmission time, especially since modern machines will often find the ___ cycle faster than just transmitting (or copying) an uncompressed file.

Answer 37

• compress -> transmit -> decompress

Question 38

Archive Compression Methods

What are the 3 common archive compression tools?

Answer 38

In order to increase compression efficiency (which comes at the cost of longer compression times):

• gzip
  
  • Uses Lempel-Ziv Coding (LZ77) and produces .gz files.
• bzip2
  
  • Uses Burrows-Wheeler block sorting text compression algorithm and Huffman coding, and produces .bz2 files.
• xz
  
  • Produces .xz files and also supports legacy .lzma format.

Question 39

Archive Compression Methods

What is the command option with tar to archive and compress using gzip compression tool?

Answer 39

• -z

Example:

$ tar zcvf source.tar.gz source

Question 40

Archive Compression Methods

What is the command option with tar to archive and compress using bzip2 compression tool?

Answer 40

• -j

Example:

$ tar jcvf source.tar.bz2 source

Question 41

Archive Compression Methods

What is the command option with tar to archive and compress using xz compression tool?

Answer 41

• -J

Example:

$ tar Jcvf source.tar.xz source

Question 42

Archive Compression Methods

When archiving a compressing at the same time with tar.

Example using xz compression:

$ tar Jcvf source.tar.xz source

What is the commands that do the same but do archive and compression in separate steps?

What do one over the other?

Answer 42

$ tar cvf source.tar source ; xz -v source.tar

It is after to do compression and archive at the same time because:

• There is no intermediate file storage.
• Archiving and compression happen simultaneously in the pipeline.

Question 43

Archive Compression Methods

What is the command option while using tar to decompression a compressed archive (.tar package)?

Example:

1. source.tar.gz source
2. source.tar.bz2 source
3. source.tar.xz source

Answer 43

1. $ tar xzvf source.tar.gz
2. $ tar xjvf source.tar.bz2
3. $ tar xJvf source.tar.xz

or even simpler:

1. $ tar xvf source.tar.gz
2. $ tar xvf source.tar.bz2
3. $ tar xvf source.tar.xz

as modern versions of tar can sense the method of compression and take care of it automatically.

Obviously, it is not worth using these methods on archives whose component files are already compressed, such as .jpg images, or .pdf files, etc.

Question 44

dd

dd is a common UNIX-based program whose primary purpose is? (2)

dd is used to copy a specified number of ___ or ___, performing on-the-fly byte order conversions, as well as being able to ___ data from one form to another.

dd can also be used to copy regions of raw device files, for example backing up the boot sector of a hard disk, or to read fixed amounts of data from special files like ___ or ___. The basic syntax is?

Answer 44

• the low-level copying
• conversion of raw data
• bytes
• blocks
• convert
• /dev/zero
• /dev/random
• $ dd if=input-file of=output-file options

Question 45

dd

The basic syntax of dd is?

Answer 45

$ dd if=input-file of=output-file options

Question 46

dd

What is the dd comand to create a file with 10 megabytes of null character data into a output file named outfile?

Answer 46

$ dd if=/dev/zero of=outfile bs=1M count=10

Question 47

dd

What is the dd command to back up an entire hard drive (with device node name sda) to another hard drive (with device node name sdb).

Answer 47

$ dd if=/dev/sda of=/dev/sdb

Question 48

dd

What is the dd command to create an image of a device node hard disk sda called sdadisk.img?

Answer 48

$ dd if=/dev/sda of=sdadisk.img

Question 49

dd

What is the dd command to create an backup partition of a device node hard disk sda1 called partition1.img?

Answer 49

$ dd if=/dev/sda1 of=partition1.img

Question 50

dd

What is the dd command to create an backup CD ROM named tgsservice.iso?

Answer 50

$ dd if=/dev/cdrom of=tgsservice.iso bs=2048

Question 51

Using rsync for Backups

rsync (remote synchronize) is used to transfer files across ___ or ___?

The basic syntax of resync command usage is?

Answer 51

• a network
• or between different locations on the same machine
• $ rsync [options] sourcefile destinationfile

Question 52

Using rsync for Backups

$ rsync [options] sourcefile destinationfile

The source and destination can take the form of target:path, where target can be in the form of ___. The ___ part is optional and used if the remote user is different from the local user. Thus, these are all possible rsync commands.

• $ rsync file.tar someone@backup.mydomain:/usr/local
• $ rsync -r –dry-run /usr/local /BACKUP/usr

Answer 52

• [user@]host
• user@

Question 53

Using rsync for Backups

You have to be very careful with rsync about exact location specifications (especially if you use the ___ option), so it is highly recommended to use the ___ option first, and then repeat if the projected action looks correct.

Answer 53

• –delete
• –dry-run

Question 54

Using rsync for Backups

rsync is very clever; it checks local files against remote files in small chunks, and it is very efficient in that when copying one directory to a similar directory, only the ___ are copied over the network. This synchronizes the second directory with the first directory. You may often use the ___ option, which causes rsync to recursively walk down the directory tree copying all files and directories below the one listed as the sourcefile. Thus, a very useful way to back up a project directory might be similar to:

$ rsync -r project-X archive-machine:archives/project-X

Answer 54

• differences
• -r

Question 55

Using rsync for Backups

A simple (and very effective and very fast) backup strategy is to simply ___ across a network with rsync commands and to do so ___.

Answer 55

• duplicate directories or partitions
• frequently

Question 56

Using cpio for Backups

cpio (copy in and out) is a general file archiver utility that has been around since the earliest days of UNIX and was originally designed for tape backups. Even though newer archiving programs (like tar, which is not exactly young) have been deployed to do many of the tasks that were once in the domain of cpio, it still survives.

For example, we have already seen the use of rpm2cpio to convert RPM packages into cpio archives and then extract them. Also, the Linux kernel uses a version of cpio internally to deal with initramfs and initrd initial ram filesystems and disks during boot.

What is 1 reason why cpio is still used today?

Answer 56

One reason cpio lives on is that it is lighter than tar and other successors, even if it is somewhat less robust.

Question 57

Using cpio for Backups

How do you use cpio?

Answer 57

You can specify the input (-I device) or output (-O) or use redirection on the command line.

The -o or –create option tells cpio to copy files out to an archive. cpio reads a list of file names (one per line) from standard input and writes the archive to standard output.

The -i or –extract option tells cpio to copy files from an archive, reading the archive from standard input. If you list file names as patterns (such as *.c) on the command line, only files in the archive that match the patterns are copied from the archive. If no patterns are given, all files are extracted.

The -t or –list option tells cpio to list the archive contents.

Adding the -v or –verbose option generates a long listing.

You can see some examples of using cpio below.

Create an archive, use -o or –create:

$ ls | cpio –create -O /dev/st0

Extract from an archive, use -i or –extract:

$ cpio -i somefile -I /dev/st0

List contents of an archive, use -t or –list:

$ cpio -t -I /dev/st0

Question 58

Backup Programs

There is no shortage of available backup program suites available for Linux, including proprietary applications or those supplied by storage vendors, as well as open-source applications.

Explain and describe 3 backup programs in linux.

Answer 58

Backup Programs

• Amanda
  
  • Amanda (Advanced Maryland Automatic Network Disk Archiver) uses native utilities (including tar and dump) but is far more robust and controllable. Amanda is generally available on Enterprise Linux systems through the usual repositories.
• Bacula
  
  • Bacula is designed for automatic backup on heterogenous networks. It can be rather complicated to use and is recommended (by its authors) only to experienced administrators. Bacula is generally available on Enterprise Linux systems through the usual repositories.
• Clonezilla
  
  • Clonezilla is a very robust disk cloning program, which can make images of disks and deploy them, either to restore a backup, or to be used for ghosting, to provide an image that can be used to install many machines.
  • ​The program comes in two versions: Clonezilla Live, which is good for single machine backup and recovery, and Clonezilla SE, server edition, which can clone to many computers at the same time. Clonezilla is not very hard to use and is extremely flexible, supporting many operating systems (not just Linux), filesystem types, and boot loaders.