The shell

Question 1

What is Bash?

Answer 1

“Bourne Again Shell”

other common shells include sh, csh, tcsh, will also be referenced in this tutorial, and they sometime differ from bash

Question 2

How can shell scripting be accomplished?

Answer 2

by directly executing shell commands at the shell prompt or by storing them in the order of execution, in a text file, called a shell script, and then executing the shell script. To execute, simply write the shell script file name, once the file has execute permission (chmod +x filename)

Question 3

What is “the shell”?

Answer 3

Simply put, the shell is a program that takes commands from the keyboard and gives them to the operating system to perform. In the old days, it was the only user interface available on a Unix-like system such as Linux. Nowadays, we have graphical user interfaces (GUIs) in addition to command line interfaces (CLIs) such as the shell.

Question 4

What is a “terminal”?

Answer 4

It’s a program called a terminal emulator. This is a program that opens a window and lets you interact with the shell. There are a bunch of different terminal emulators you can use. Most Linux distributions supply several, such as: gnome-terminal, konsole, xterm, rxvt, kvt, nxterm, and eterm.

Question 5

separate commands in one line

Answer 5

;

Question 6

Command history

Answer 6

Up/down arrows

Question 7

If you’re not logged in as root, what are you?

Answer 7

If the last character of your shell prompt is # rather than $, you are operating as the superuser. This means that you have administrative privileges. This can be potentially dangerous, since you are able to delete or overwrite any file on the system. Unless you absolutely need administrative privileges, do not operate as the superuser.

Question 8

Why do you need to learn the command line anyway?

Answer 8

Graphical user interfaces (GUIs) are helpful for many tasks, but they are not good for all tasks. I have long felt that most computers today are not powered by electricity. They instead seem to be powered by the “pumping” motion of the mouse! Computers were supposed to free us from manual labor, but how many times have you performed some task you felt sure the computer should be able to do but you ended up doing the work yourself by tediously working the mouse? Pointing and clicking, pointing and clicking.

Question 9

pwd

Answer 9

The directory you are standing in is called the working directory. To find the name of the working directory, use the pwd command.

Stands for “print working directory”

Question 10

ls

Answer 10

list the files in the working directory

Stands for “list”

Question 11

cd

Answer 11

Change the working directory (cd followed by the pathname)

Question 12

Absolute vs. relative path names

Answer 12

Where an absolute pathname starts from the root directory and leads to its destination, a relative pathname starts from the working directory. To do this, it uses a couple of special notations to represent relative positions in the file system tree. These special notations are “.” (dot) and “..” (dot dot).

The “.” notation refers to the working directory itself and the “..” notation refers to the working directory’s parent directory

Question 13

Root directory

Answer 13

/

Question 14

User directory (or home directory/where you start out normally)

Answer 14

~

Question 15

cd followed by nothing

Answer 15

Shortcut to change working directory to your home directory

Question 16

cd ~user_name

Answer 16

Change to home directory of specified user

Question 17

Hidden files

Answer 17

File names that begin with a period character are hidden. This only means that ls will not list them unless you say ls -a. When your account was created, several hidden files were placed in your home directory to configure things for your account. Later on we will take a closer look at some of these files to see how you can customize your environment. In addition, some applications will place their configuration and settings files in your home directory as hidden files.

Question 18

File names in Linux, like Unix, are…

Answer 18

case sensitive. The file names “File1” and “file1” refer to different files.

Question 19

Linux has no concept of…

Answer 19

a “file extension” like legacy operating systems. You may name files any way you like. However, while Linux itself does not care about file extensions, many application programs do.

Question 20

Punctuation in file names

Answer 20

Though Linux supports long file names which may contain embedded spaces and punctuation characters, limit the punctuation characters to period, dash, and underscore. Most importantly, do not embed spaces in file names. If you want to represent spaces between words in a file name, use underscore characters. You will thank yourself later.

Question 21

Most commonly used command

Answer 21

probably ls

Question 22

List files in a directory

Answer 22

ls /bin (/bin is just a directory for this example)

Question 23

List the files in the working directory in long format

Answer 23

ls -l

long-form shows a lot more info, like:

File Name
-The name of the file or directory.

Modification Time
-The last time the file was modified. If the last modification occurred more than six months in the past, the date and year are displayed. Otherwise, the time of day is shown.

Size
-The size of the file in bytes.

Group
-The name of the group that has file permissions in addition to the file’s owner.

Owner
-The name of the user who owns the file.

File Permissions
-A representation of the file’s access permissions. The first character is the type of file. A “-“ indicates a regular (ordinary) file. A “d” indicates a directory. The second set of three characters represent the read, write, and execution rights of the file’s owner. The next three represent the rights of the file’s group, and the final three represent the rights granted to everybody else. I’ll discuss this in more detail in a later lesson.

Question 24

List files in long format in multiple directories

Answer 24

ls -l dir_1 dir_2

Question 25

List all files (even ones with names beginning with a period character, which are normally hidden) in the parent of the working directory in long format

Answer 25

ls -la ..

Question 26

The format for most commands

Answer 26

command -options arguments

Question 27

less

Answer 27

a program that lets you view text files. This is very handy since many of the files used to control and configure Linux are human readable.

Question 28

What is “text”?

Answer 28

There are many ways to represent information on a computer. All methods involve defining a relationship between the information and some numbers that will be used to represent it. Computers, after all, only understand numbers and all data is converted to numeric representation.

Some of these representation systems are very complex (such as compressed multimedia files), while others are rather simple. One of the earliest and simplest is called ASCII text. ASCII (pronounced “As-Key”) is short for American Standard Code for Information Interchange. This is a simple encoding scheme that was first used on Teletype machines to map keyboard characters to numbers.

Text is a simple one-to-one mapping of characters to numbers. It is very compact. Fifty characters of text translates to fifty bytes of data. Throughout a Linux system, many files are stored in text format and there are many Linux tools that work with text files. Even the legacy operating systems recognize the importance of this format. The well-known NOTEPAD.EXE program is an editor for plain ASCII text files.

Question 29

How to display a text file?

Answer 29

less text_file

Once started, less will display the text file one page at a time. You may use the Page Up and Page Down keys to move through the text file. To exit less, type “q”.

Question 30

less commands

Answer 30

Page Up or b: Scroll back one page

Page Down or space: Scroll forward one page

G: Go to the end of the text file

1G: Go to the beginning of the text file

/characters: Search forward in the text file for an occurrence of the specified characters

n: Repeat the previous search
h: Display a complete list less commands and options
q: Quit

Question 31

file

Answer 31

examines a file and tell you what kind of file it is

While it may seem that most files cannot be viewed as text, you will be surprised how many can. This is especially true of the important configuration files. You will also notice during our adventure that many features of the operating system are controlled by shell scripts. In Linux, there are no secrets!

Question 32

File types recognizable by ‘file’

Answer 32

File Type / Description / Viewable as text?

ASCII text/The name says it all/yes

Bourne-Again shell script text/A bash script/yes

ELF 32-bit LSB core file/A core dump file (a program will create this when it crashes)/no

ELF 32-bit LSB executable/An executable binary program/no

ELF 32-bit LSB shared object/A shared library/no

GNU tar archive/A tape archive file. A common way of storing groups of files./no, use tar tvf to view listing.

gzip compressed data/An archive compressed with gzip/no

HTML document text/A web page/yes

JPEG image data/A compressed JPEG image/no

PostScript document text/A PostScript file/yes

RPM/A Red Hat Package Manager archive/no, use rpm -q to examine contents.

Zip archive data/An archive compressed with zip/no

Question 33

/

Answer 33

The root directory where the file system begins. In most cases the root directory only contains subdirectories.

Question 34

/boot

Answer 34

This is where the Linux kernel and boot loader files are kept. The kernel is a file called vmlinuz

Question 35

/etc

Answer 35

The /etc directory contains the configuration files for the system. All of the files in /etc should be text files. Points of interest:

/etc/passwd
The passwd file contains the essential information for each user. It is here that users are defined.

/etc/fstab
The fstab file contains a table of devices that get mounted when your system boots. This file defines your disk drives.

/etc/hosts
This file lists the network host names and IP addresses that are intrinsically known to the system.

/etc/init.d
This directory contains the scripts that start various system services typically at boot time.

Question 36

/bin, /usr/bin

Answer 36

These two directories contain most of the programs for the system. The /bin directory has the essential programs that the system requires to operate, while /usr/bin contains applications for the system’s users.

Question 37

/sbin, /usr/sbin

Answer 37

The sbin directories contain programs for system administration, mostly for use by the superuser.

Question 38

/usr

Answer 38

The /usr directory contains a variety of things that support user applications. Some highlights:

/usr/share/X11
Support files for the X Window system

/usr/share/dict
Dictionaries for the spelling checker. Bet you didn’t know that Linux had a spelling checker. See look and aspell.

/usr/share/doc
Various documentation files in a variety of formats.

/usr/share/man
The man pages are kept here.

/usr/src
Source code files. If you installed the kernel source code package, you will find the entire Linux kernel source code here.

Question 39

/usr/local

Answer 39

/usr/local and its subdirectories are used for the installation of software and other files for use on the local machine. What this really means is that software that is not part of the official distribution (which usually goes in /usr/bin) goes here.

When you find interesting programs to install on your system, they should be installed in one of the /usr/local directories. Most often, the directory of choice is /usr/local/bin.

Question 40

/var

Answer 40

The /var directory contains files that change as the system is running. This includes:

/var/log
Directory that contains log files. These are updated as the system runs. You should view the files in this directory from time to time, to monitor the health of your system.

/var/spool
This directory is used to hold files that are queued for some process, such as mail messages and print jobs. When a user’s mail first arrives on the local system (assuming you have local mail), the messages are first stored in /var/spool/mail

Question 41

/lib

Answer 41

The shared libraries (similar to DLLs in that other operating system) are kept here.

Question 42

/home

Answer 42

/home is where users keep their personal work. In general, this is the only place users are allowed to write files. This keeps things nice and clean :-)

Question 43

/root

Answer 43

This is the superuser’s home directory.

Question 44

/temp

Answer 44

/tmp is a directory in which programs can write their temporary files.

Question 45

/dev

Answer 45

The /dev directory is a special directory, since it does not really contain files in the usual sense. Rather, it contains devices that are available to the system. In Linux (like Unix), devices are treated like files. You can read and write devices as though they were files. For example /dev/fd0 is the first floppy disk drive, /dev/sda (/dev/hda on older systems) is the first hard drive. All the devices that the kernel understands are represented here.

Question 46

/proc

Answer 46

The /proc directory is also special. This directory does not contain files. In fact, this directory does not really exist at all. It is entirely virtual. The /proc directory contains little peep holes into the kernel itself. There are a group of numbered entries in this directory that correspond to all the processes running on the system. In addition, there are a number of named entries that permit access to the current configuration of the system. Many of these entries can be viewed. Try viewing /proc/cpuinfo. This entry will tell you what the kernel thinks of your CPU.

Question 47

/media,/mnt

Answer 47

Finally, we come to /media, a normal directory which is used in a special way. The /media directory is used for mount points. As we learned in the second lesson, the different physical storage devices (like hard disk drives) are attached to the file system tree in various places. This process of attaching a device to the tree is called mounting. For a device to be available, it must first be mounted.

When your system boots, it reads a list of mounting instructions in the file /etc/fstab, which describes which device is mounted at which mount point in the directory tree. This takes care of the hard drives, but you may also have devices that are considered temporary, such as CD-ROMs, thumb drives, and floppy disks. Since these are removable, they do not stay mounted all the time. The /media directory is used by the automatic device mounting mechanisms found in modern desktop oriented Linux distributions. On systems that require manual mounting of removable devices, the /mnt directory provides a convenient place for mounting these temporary devices. You will often see the directories /mnt/floppy and /mnt/cdrom. To see what devices and mount points are used, type mount.

Question 48

symbolic links

Answer 48

System.map, module-info, and vmlinuz for example

Symbolic links are a special type of file that points to another file. With symbolic links, it is possible for a single file to have multiple names. Here’s how it works: Whenever the system is given a file name that is a symbolic link, it transparently maps it to the file it is pointing to

Just what is this good for? This is a very handy feature. Let’s consider the directory listing above (which is the /boot directory of an old Red Hat 5.2 system). This system has had multiple versions of the Linux kernel installed. We can see this from the files vmlinuz-2.0.36-0.7 and vmlinuz-2.0.36-3. These file names suggest that both version 2.0.36-0.7 and 2.0.36-3 are installed. Because the file names contain the version it is easy to see the differences in the directory listing. However, this would be confusing to programs that rely on a fixed name for the kernel file. These programs might expect the kernel to simply be called “vmlinuz”. Here is where the beauty of the symbolic link comes in. By creating a symbolic link called vmlinuz that points to vmlinuz-2.0.36-3, we have solved the problem.

Example of a symbolic link in the output of a “ls -l” command:
lrwxrwxrwx 25 Jul 3 16:42 System.map -> /boot/System.map-2.0.36-3

Question 49

How to create a symbolic link

Answer 49

ln command

Question 50

Copy files and directories

Answer 50

cp

Question 51

Move or rename files and directories

Answer 51

mv

Question 52

Remove files and directories

Answer 52

rm

Question 53

What are the basic commands for manipulating files and directories?

Answer 53

cp, mv, rm, mkdir

These four commands are among the most frequently used Linux commands. They are the basic commands for manipulating both files and directories.

Now, to be frank, some of the tasks performed by these commands are more easily done with a graphical file manager. With a file manager, you can drag and drop a file from one directory to another, cut and paste files, delete files, etc. So why use these old command line programs?

The answer is power and flexibility. While it is easy to perform simple file manipulations with a graphical file manager, complicated tasks can be easier with the command line programs.

Question 54

Create directories

Answer 54

mkdir

Question 55

Match any characters

Answer 55

*

Question 56

Match any single character

Answer 56

?

Question 57

Match any character that is a member of a set of characters.

Answer 57

[characters]

The set of characters may also be expressed as a POSIX character class such as one of the following:

POSIX Character Classes
[:alnum:] Alphanumeric characters
[:alpha:] Alphabetic characters
[:digit:] Numerals
[:upper:] Uppercase alphabetic characters
[:lower:] Lowercase alphabetic characters

Question 58

Match any character that is NOT a member of a set of characters

Answer 58

[!characters]

Question 59

Using wildcards, it is possible to construct very sophisticated selection criteria for filenames. Here are some examples of patterns and what they match:

Answer 59

Pattern
Matches
————-

*
All filenames

g*
All filenames that begin with the character “g”

b*.txt
All filenames that begin with the character “b” and end with the characters “.txt”

Data???
Any filename that begins with the characters “Data” followed by exactly 3 more characters

[abc]*
Any filename that begins with “a” or “b” or “c” followed by any other characters

[[:upper:]]*
Any filename that begins with an uppercase letter. This is an example of a character class.

BACKUP.[[:digit:]][[:digit:]]
Another example of character classes. This pattern matches any filename that begins with the characters “BACKUP.” followed by exactly two numerals.

*[![:lower:]]
Any filename that does not end with a lowercase letter.

Question 60

You can use wildcards with…

Answer 60

any command that accepts filename arguments

Question 61

cp file… directory

Answer 61

A note on notation: … signifies that an item can be repeated one or more times.

Question 62

Where are you after the following command?

/a/./b/../../c

Answer 62

/c

Question 63

Examples of cp command

Answer 63

Command
Results

cp file1 file2
Copies the contents of file1 into file2. If file2 does not exist, it is created; otherwise, file2 is silently overwritten with the contents of file1.

cp -i file1 file2
Like above however, since the “-i” (interactive) option is specified, if file2 exists, the user is prompted before it is overwritten with the contents of file1.

cp file1 dir1
Copy the contents of file1 (into a file named file1) inside of directory dir1.

cp -R dir1 dir2
Copy the contents of the directory dir1. If directory dir2 does not exist, it is created. Otherwise, it creates a directory named dir1 within directory dir2.

Question 64

Examples of mv command

Answer 64

Command
Results
———–

mv file1 file2
If file2 does not exist, then file1 is renamed file2. If file2 exists, its contents are silently replaced with the contents of file1.

mv -i file1 file2
Like above however, since the “-i” (interactive) option is specified, if file2 exists, the user is prompted before it is overwritten with the contents of file1.

mv file1 file2 file3 dir1
The files file1, file2, file3 are moved to directory dir1. If dir1 does not exist, mv will exit with an error.

mv dir1 dir2
If dir2 does not exist, then dir1 is renamed dir2. If dir2 exists, the directory dir1 is moved within directory dir2.

Question 65

Examples of rm command

Answer 65

Command
Results
———-

rm file1 file2
Delete file1 and file2.

rm -i file1 file2
Like above however, since the “-i” (interactive) option is specified, the user is prompted before each file is deleted.

rm -r dir1 dir2
Directories dir1 and dir2 are deleted along with all of their contents.

Question 66

What must we do when using rm?

Answer 66

Be careful with rm!

Linux does not have an undelete command. Once you delete something with rm, it’s gone. You can inflict terrific damage on your system with rm if you are not careful, particularly with wildcards.

Before you use rm with wildcards, try this helpful trick: construct your command using ls instead. By doing this, you can see the effect of your wildcards before you delete files. After you have tested your command with ls, recall the command with the up-arrow key and then substitute rm for ls in the command.

Question 67

Examples of file manipulation commands using wildcards

Answer 67

Command
Results
——–

cp *.txt text_files
Copy all files in the current working directory with names ending with the characters “.txt” to an existing directory named text_files.

mv my_dir ../*.bak my_new_dir
Move the subdirectory my_dir and all the files ending in “.bak” in the current working directory’s parent directory to an existing directory named my_new_dir.

rm *~
Delete all files in the current working directory that end with the character “~”. Some applications create backup files using this naming scheme. Using this command will clean them out of a directory.

Question 68

Display information about a command type

Answer 68

type

The type command is a shell builtin that displays the kind of command the shell will execute, given a particular command name.

It works like this:

type 

where “command” is the name of the command you want to examine.

For example: “type type” will return “type is a shell builtin”; “type cp” will return “cp is /bin/cp”; “type ls” returns “ls is aliased to `ls –color=tty’”

Notice that the one for ls (taken from a Fedora system) and how the ls command is actually an alias for the ls command with the “– color=tty” option added. Now we know why the output from ls is displayed in color! In Mac, it returns “ls -GFh”

Question 69

Locate a command

Answer 69

which

Sometimes there is more than one version of an executable program installed on a system. While this is not very common on desktop systems, it’s not unusual on large servers. To determine the exact location of a given executable, the which command is used:

$ which ls
/bin/ls

which only works for executable programs, not builtins nor aliases that are substitutes for actual executable programs.

Question 70

Display reference page for shell bulletin

Answer 70

help

bash has a built-in help facility available for each of the shell builtins. To use it, type “help” followed by the name of the shell builtin. Optionally, you may add the -m option (seems to be -p in Mac) to change the format of the output.

Question 71

Display an online command reference

Answer 71

man

Most executable programs intended for command line use provide a formal piece of documentation called a manual or man page. A special paging program called man is used to view them. It is used like this:

man program

where “program” is the name of the command to view. Man pages vary somewhat in format but generally contain a title, a synopsis of the command’s syntax, a description of the command’s purpose, and a listing and description of each of the command’s options. Man pages, however, do not usually include examples, and are intended as a reference, not a tutorial

On most Linux systems, man uses less to display the manual page, so all of the familiar less commands work while displaying the page

Question 72

What are commands?

Answer 72

Commands can be one of 4 different kinds:

An executable program like all those files we saw in /usr/bin. Within this category, programs can be compiled binaries such as programs written in C and C++, or programs written in scripting languages such as the shell, Perl, Python, Ruby, etc.

A command built into the shell itself. bash provides a number of commands internally called shell builtins. The cd command, for example, is a shell builtin.

A shell function. These are miniature shell scripts incorporated into the environment. We will cover configuring the environment and writing shell functions in later lessons, but for now, just be aware that they exist.

An alias. Commands that you can define yourselves, built from other commands. This will be covered in a later lesson.

Question 73

Square brackets and vertical bars in notation

Answer 73

A note on notation: When square brackets appear in the description of a command’s syntax, they indicate optional items. A vertical bar character indicates mutually exclusive items

So, “cd [-L|-P] [dir]” means:
the command cd may be followed optionally by either a “-L” or a “-P” and further, optionally followed by the argument “dir”.

Question 74

–help

Answer 74

Many executable programs support a “–help” option that displays a description of the command’s supported syntax and options.

Question 75

README and other documentation

Answer 75

Many software packages installed on your system have documentation files residing in the /usr/share/doc directory. Most of these are stored in plain text format and can be viewed with less. Some of the files are in HTML format and can be viewed with your web browser. You may encounter some files ending with a “.gz” extension. This indicates that they have been compressed with the gzip compression program. The gzip package includes a special version of less called zless that will display the contents of gzip-compressed text files.

Question 76

Input/output redirection

Answer 76

As we have seen, many commands such as ls print their output on the display. This does not have to be the case, however. By using some special notations we can redirect the output of many commands to files, devices, and even to the input of other commands.

Most command line programs that display their results do so by sending their results to a facility called standard output. By default, standard output directs its contents to the display.

Question 77

How to redirect standard output to a file

Answer 77

>

Example:
ls > file_list.txt

In this example, the ls command is executed and the results are written in a file named file_list.txt. Since the output of ls was redirected to the file, no results appear on the display.

Each time the command above is repeated, file_list.txt is overwritten from the beginning with the output of the command ls.

Question 78

How do you redirect and APPEND standard output to a file?

Answer 78

> >

If you want the new results to be appended to the file instead, use “»” like this:
ls&raquo_space; file_list.txt

When the results are appended, the new results are added to the end of the file, thus making the file longer each time the command is repeated. If the file does not exist when you attempt to append the redirected output, the file will be created.

Question 79

Standard input

Answer 79

Many commands can accept input from a facility called standard input. By default, standard input gets its contents from the keyboard, but like standard output, it can be redirected.

Question 80

To redirect standard input from a file instead of the keyboard

Answer 80

<

Example:
sort < file_list.txt

we used the sort command to process the contents of file_list.txt. The results are output on the display since the standard output was not redirected. We could redirect standard output to another file like this:

sort < file_list.txt > sorted_file_list.txt

As you can see, a command can have both its input and output redirected. Be aware that the order of the redirection does not matter. The only requirement is that the redirection operators (the “”) must appear after the other options and arguments in the command.

Question 81

Pipelines

Answer 81

The most useful and powerful thing you can do with I/O redirection is to connect multiple commands together with what are called pipelines. With pipelines, the standard output of one command is fed into the standard input of another.

Question 82

less

Answer 82

By using this “| less” trick, you can make any command have scrolling output. I use this technique all the time.

Question 83

Common command to feed output of ls command into less

Answer 83

ls -l | less

Question 84

Pipelining examples

Answer 84

Command:
What it does
——-

ls -lt | head
Displays the 10 newest files in the current directory.

du | sort -nr
Displays a list of directories and how much space they consume, sorted from the largest to the smallest.

find . -type f -print | wc -l
Displays the total number of files in the current working directory and all of its subdirectories.

Question 85

Filters

Answer 85

One kind of program frequently used in pipelines is called filters. Filters take standard input and perform an operation upon it and send the results to standard output. In this way, they can be combined to process information in powerful ways.

Question 86

Common filter commands

Answer 86

Program:
What it does
———-

sort
Sorts standard input then outputs the sorted result on standard output.

uniq
Given a sorted stream of data from standard input, it removes duplicate lines of data (i.e., it makes sure that every line is unique).

grep
Examines each line of data it receives from standard input and outputs every line that contains a specified pattern of characters.

fmt
Reads text from standard input, then outputs formatted text on standard output.

pr
Takes text input from standard input and splits the data into pages with page breaks, headers and footers in preparation for printing.

head
Outputs the first few lines of its input. Useful for getting the header of a file.

tail
Outputs the last few lines of its input. Useful for things like getting the most recent entries from a log file.

tr
Translates characters. Can be used to perform tasks such as upper/lowercase conversions or changing line termination characters from one type to another (for example, converting DOS text files into Unix style text files).

sed
Stream editor. Can perform more sophisticated text translations than tr.

awk
An entire programming language designed for constructing filters. Extremely powerful.

Question 87

Printing from the command line

Answer 87

Linux provides a program called lpr that accepts standard input and sends it to the printer. It is often used with pipes and filters. Here are a couple of examples:

cat poorly_formatted_report.txt | fmt | pr | lpr

cat unsorted_list_with_dupes.txt | sort | uniq | pr | lpr

In the first example, we use cat to read the file and output it to standard output, which is piped into the standard input of fmt. fmt formats the text into neat paragraphs and outputs it to standard output, which is piped into the standard input of pr. pr splits the text neatly into pages and outputs it to standard output, which is piped into the standard input of lpr. lpr takes its standard input and sends it to the printer.

The second example starts with an unsorted list of data with duplicate entries. First, cat sends the list into sort which sorts it and feeds it into uniq which removes any duplicates. Next pr and lpr are used to paginate and print the list.

Question 88

Viewing the contents of tar files

Answer 88

Often you will see software distributed as a gzipped tar file. This is a traditional Unix style tape archive file (created with tar) that has been compressed with gzip. You can recognize these files by their traditional file extensions, “.tar.gz” or “.tgz”. You can use the following command to view the directory of such a file on a Linux system:

tar tzvf name_of_file.tar.gz | less

Question 89

Expansion

Answer 89

Each time you type a command line and press the enter key, bash performs several processes upon the text before it carries out your command. We have seen a couple of cases of how a simple character sequence, for example “*”, can have a lot of meaning to the shell. The process that makes this happen is called expansion. With expansion, you type something and it is expanded into something else before the shell acts upon it.

Question 90

echo *

Answer 90

Result:
Desktop Documents ls-output.txt Music Pictures Public Templates Videos

So what just happened? Why didn’t echo print “”? As you recall from our work with wildcards, the “” character means match any characters in a filename, but what we didn’t see in our original discussion was how the shell does that. The simple answer is that the shell expands the “” into something else (in this instance, the names of the files in the current working directory) before the echo command is executed. When the enter key is pressed, the shell automatically expands any qualifying characters on the command line before the command is carried out, so the echo command never saw the “”, only its expanded result. Knowing this, we can see that echo behaved as expected.

Question 91

Pathname expansion

Answer 91

The mechanism by which wildcards work is called pathname expansion. If we try some of the techniques that we employed in our earlier lessons, we will see that they are really expansions.

Ex:
echo [[:upper:]]*
Result: Desktop Documents Music Pictures Public Templates Videos

Another Ex:
echo /usr/*/share
Result: /usr/kerberos/share /usr/local/share

Question 92

Tilde expansion

Answer 92

As you may recall from our introduction to the cd command, the tilde character (“~”) has a special meaning. When used at the beginning of a word, it expands into the name of the home directory of the named user, or if no user is named, the home directory of the current user

echo ~
/home/me

echo ~foo
/home/foo

Question 93

Arithmetic Expansion

Answer 93

echo $((2 + 2))

Arithmetic expansion only supports integers (whole numbers, no decimals), but can perform quite a number of different operations.

Question 94

Arithmetic expression

Answer 94

$((expression))

where expression is an arithmetic expression consisting of values and arithmetic operators.

Spaces are not significant in arithmetic expressions and expressions may be nested. For example, to multiply five squared by three:
echo $(($((5**2)) * 3))

Question 95

Integer division

Answer 95

echo Five divided by two equals $((5/2))
Result: Five divided by two equals 2

(Notice the effect of integer division)

Question 96

Brace expansion

Answer 96

Perhaps the strangest expansion is called brace expansion. With it, you can create multiple text strings from a pattern containing braces. Here’s an example:

echo Front-{A,B,C}-Back
Result: Front-A-Back Front-B-Back Front-C-Back

Previous | Contents | Next

Expansion

Each time you type a command line and press the enter key, bash performs several processes upon the text before it carries out your command. We have seen a couple of cases of how a simple character sequence, for example “*”, can have a lot of meaning to the shell. The process that makes this happen is called expansion. With expansion, you type something and it is expanded into something else before the shell acts upon it. To demonstrate what we mean by this, let’s take a look at the echo command. echo is a shell builtin that performs a very simple task. It prints out its text arguments on standard output:

[me@linuxbox me]$ echo this is a test
this is a test

That’s pretty straightforward. Any argument passed to echo gets displayed. Let’s try another example:

[me@linuxbox me]$ echo *
Desktop Documents ls-output.txt Music Pictures Public Templates Videos

So what just happened? Why didn’t echo print “”? As you recall from our work with wildcards, the “” character means match any characters in a filename, but what we didn’t see in our original discussion was how the shell does that. The simple answer is that the shell expands the “” into something else (in this instance, the names of the files in the current working directory) before the echo command is executed. When the enter key is pressed, the shell automatically expands any qualifying characters on the command line before the command is carried out, so the echo command never saw the “”, only its expanded result. Knowing this, we can see that echo behaved as expected.

Pathname Expansion

The mechanism by which wildcards work is called pathname expansion. If we try some of the techniques that we employed in our earlier lessons, we will see that they are really expansions. Given a home directory that looks like this:

[me@linuxbox me]$ ls

Desktop
ls-output.txt
Documents Music
Pictures
Public
Templates
Videos
we could carry out the following expansions:

[me@linuxbox me]$ echo D*
Desktop Documents

and:

[me@linuxbox me]$ echo *s
Documents Pictures Templates Videos

or even:

[me@linuxbox me]$ echo [[:upper:]]*
Desktop Documents Music Pictures Public Templates Videos

and looking beyond our home directory:

[me@linuxbox me]$ echo /usr/*/share
/usr/kerberos/share /usr/local/share

Tilde Expansion

As you may recall from our introduction to the cd command, the tilde character (“~”) has a special meaning. When used at the beginning of a word, it expands into the name of the home directory of the named user, or if no user is named, the home directory of the current user:

[me@linuxbox me]$ echo ~
/home/me

If user “foo” has an account, then:

[me@linuxbox me]$ echo ~foo
/home/foo

Arithmetic Expansion

The shell allows arithmetic to be performed by expansion. This allow us to use the shell prompt as a calculator:

[me@linuxbox me]$ echo $((2 + 2))
4

Arithmetic expansion uses the form:

$((expression)) where expression is an arithmetic expression consisting of values and arithmetic operators.

Arithmetic expansion only supports integers (whole numbers, no decimals), but can perform quite a number of different operations.

Spaces are not significant in arithmetic expressions and expressions may be nested. For example, to multiply five squared by three:

[me@linuxbox me]$ echo $(($((5**2)) * 3))
75

Single parentheses may be used to group multiple subexpressions. With this technique, we can rewrite the example above and get the same result using a single expansion instead of two:

[me@linuxbox me]$ echo $(((5**2) * 3))
75

Here is an example using the division and remainder operators. Notice the effect of integer division:

[me@linuxbox me]$ echo Five divided by two equals $((5/2))
Five divided by two equals 2
[me@linuxbox me]$ echo with $((5%2)) left over.
with 1 left over.

Brace Expansion

Perhaps the strangest expansion is called brace expansion. With it, you can create multiple text strings from a pattern containing braces. Here’s an example:

[me@linuxbox me]$ echo Front-{A,B,C}-Back
Front-A-Back Front-B-Back Front-C-Back

Patterns to be brace expanded may contain a leading portion called a preamble and a trailing portion called a postscript. The brace expression itself may contain either a comma-separated list of strings, or a range of integers or single characters. The pattern may not contain embedded whitespace.

Another example, using integer range:

echo Number_{1..5}
Result: Number_1 Number_2 Number_3 Number_4 Number_5

echo {Z..A}
Result: Z Y X W V U T S R Q P O N M L K J I H G F E D C B A

Nested brace expansions:
echo a{A{1,2},B{3,4}}b
Result: aA1b aA2b aB3b aB4b

So what is this good for? The most common application is to make lists of files or directories to be created. For example, if you were a photographer and had a large collection of images you wanted to organize into years and months, the first thing you might do is create a series of directories named in numeric “Year-Month” format. This way, the directory names will sort in chronological order. You could type out a complete list of directories, but that’s a lot of work and it’s error-prone too. Instead, you could do this:

[me@linuxbox me]$ mkdir Photos
[me@linuxbox me]$ cd Photos
[me@linuxbox Photos]$ mkdir {2007..2009}-0{1..9} {2007..2009}-{10..12}
[me@linuxbox Photos]$ ls

2007-01 2007-07 2008-01 2008-07 2009-01 2009-07
2007-02 2007-08 2008-02 2008-08 2009-02 2009-08
2007-03 2007-09 2008-03 2008-09 2009-03 2009-09
2007-04 2007-10 2008-04 2008-10 2009-04 2009-10
2007-05 2007-11 2008-05 2008-11 2009-05 2009-11
2007-06 2007-12 2008-06 2008-12 2009-06 2009-12

Question 97

ps command

Answer 97

On its own the ps command shows the running processes by the user running it within a terminal window.

To invoke ps simply type the following:

ps

The output will show rows of data containing the following information:

PID
TTY
Time
Command
The PID is the process ID which identifies the running process. The TTY is the terminal type. 

On its own the ps command is quite limited. You probably want to see all the running processes.

Question 98

Parameter expansion

Answer 98

We’re only going to touch briefly on parameter expansion in this lesson, but we’ll be covering it more later. It’s a feature that is more useful in shell scripts than directly on the command line. Many of its capabilities have to do with the system’s ability to store small chunks of data and to give each chunk a name. Many such chunks, more properly called variables, are available for your examination. For example, the variable named “USER” contains your user name. To invoke parameter expansion and reveal the contents of USER you would do this:

echo $USER
me

To see a list of available variables, try this:
printenv | less

Question 99

Command substitution

Answer 99

Command substitution allows us to use the output of a command as an expansion:

echo $(ls)
Desktop Documents ls-output.txt Music Pictures Public Templates Videos

Some older shell programs used back-ticks instead of $()

Question 100

ls -l $(which cp)

Answer 100

Great example of command substitution

Here we passed the results of which cp as an argument to the ls command, thereby getting the listing of of the cp program without having to know its full pathname. We are not limited to just simple commands.

You can even use entire pipelines in command substitutions.

Question 101

Quoting

Answer 101

echo The total is $100.00
Result: The total is 00.00

this is because $1 is interpreted as an undefined variable, thus producing an empty string.

We can use quotes to prevent this from happening.
If you place text inside double quotes, all the special characters used by the shell lose their special meaning and are treated as ordinary characters. The exceptions are “$”, “\” (backslash), and “`” (back- quote). This means that word-splitting, pathname expansion, tilde expansion, and brace expansion are suppressed, but parameter expansion, arithmetic expansion, and command substitution are still carried out.

Example with space in file name without quotes:
ls -l two words.txt
ls: cannot access two: No such file or directory
ls: cannot access words.txt: No such file or directory

Example with space in file name WITH quotes:
ls -l “two words.txt”
-rw-rw-r– 1 me me 18 2008-02-20 13:03 two words.txt

Would be smart to do this:
mv “two words.txt” two_words.txt

Question 102

echo $(cal) vs echo “$(cal)”

Answer 102

Once the double quotes are added, our command line contains a command followed by a single argument. The fact that newlines are considered delimiters by the word-splitting mechanism causes an interesting, albeit subtle, effect on command substitution.

In the first instance, the unquoted command substitution resulted in a command line containing thirty-eight arguments. In the second, a command line with one argument that includes the embedded spaces and newlines.

Question 103

Single quotes vs double quotes

Answer 103

Single quotes suppress all expansions.

Parameter expansion, arithmetic expansion, and command substitution still take place within double quotes.

As you can see, with each succeeding level of quoting, more and more of the expansions are suppressed:

[me@linuxbox me]$ echo text ~/*.txt {a,b} $(echo foo) $((2+2)) $USER
text /home/me/ls-output.txt a b foo 4 me

[me@linuxbox me]$ echo “text ~/.txt {a,b} $(echo foo) $((2+2)) $USER”
text ~/.txt {a,b} foo 4 me

[me@linuxbox me]$ echo ‘text ~/.txt {a,b} $(echo foo) $((2+2)) $USER’
text ~/.txt {a,b} $(echo foo) $((2+2)) $USER

Question 104

Escaping characters

Answer 104

Sometimes you only want to quote a single character. To do this, you can precede a character with a backslash, which in this context is called the escape character. Often this is done inside double quotes to selectively prevent an expansion:

[me@linuxbox me]$ echo “The balance for user $USER is: $5.00”
The balance for user me is: $5.00

It is also common to use escaping to eliminate the special meaning of a character in a filename. For example, it is possible to use characters in filenames that normally have special meaning to the shell. These would include “$”, “!”, “&”, “ “, and others. To include a special character in a filename you can to this:

[me@linuxbox me]$ mv bad\&filename good_filename

Question 105

How to get the shell to ignore a newline character in your command?

Answer 105

ls -l \

• -reverse \
• -human-readable \
• -full-time

Using the backslash in this way allows us to embed newlines in our command. Note that for this trick to work, the newline must be typed immediately after the backslash. If you put a space after the backslash, the space will be ignored, not the newline.

Question 106

Backslashes are also used to insert special characters into our text. These are called backslash escape characters. Here are the common ones:

Answer 106

Escape Character/Name/Possible Uses

\n
newline
Adding blank lines to text

\t
tab
Inserting horizontal tabs to text

\a
alert
Makes your terminal beep

backslash
Inserts a backslash

\f
formfeed
Sending this to your printer ejects the page

Question 107

Permissions

Answer 107

The Unix-like operating systems, such as Linux differ from other computing systems in that they are not only multitasking but also multi-user.

What exactly does this mean? It means that more than one user can be operating the computer at the same time. While your computer only has one keyboard and monitor, it can still be used by more than one user. For example, if your computer is attached to a network, or the Internet, remote users can log in via ssh (secure shell) and operate the computer. In fact, remote users can execute graphical applications and have the output displayed on a remote computer. The X Window system supports this.

The multi-user capability of Unix-like systems is a feature that is deeply ingrained into the design of the operating system. If you remember the environment in which Unix was created, this makes perfect sense. Years ago before computers were “personal,” they were large, expensive, and centralized. A typical university computer system consisted of a large mainframe computer located in some building on campus and terminals were located throughout the campus, each connected to the large central computer. The computer would support many users at the same time.

In order to make this practical, a method had to be devised to protect the users from each other. After all, you could not allow the actions of one user to crash the computer, nor could you allow one user to interfere with the files belonging to another user.

Question 108

Modify file access rights

Answer 108

chmod

The chmod command is used to change the permissions of a file or directory. To use it, you specify the desired permission settings and the file or files that you wish to modify. There are two ways to specify the permissions. In this lesson we will focus on one of these, called the octal notation method.

It is easy to think of the permission settings as a series of bits (which is how the computer thinks about them). Here’s how it works:

rwx rwx rwx = 111 111 111
rw- rw- rw- = 110 110 110
rwx — — = 111 000 000

and so on…

rwx = 111 in binary = 7
rw- = 110 in binary = 6
r-x = 101 in binary = 5
r-- = 100 in binary = 4

Question 109

temporarily become superuser

Answer 109

su or sudo

difference: Comparing the both, sudo lets one use the user account password to run system command. On the other hand, su forces one to share the root passwords to other users. Also, sudo doesn’t activate the root shell and runs a single command.

Question 110

Change a file’s group ownership

Answer 110

chgrp

In the example below, we changed the group ownership of some_file from its previous group to “new_group”. You must be the owner of the file or directory to perform a chgrp.
chgrp new_group some_file

Question 111

File permissions

Answer 111

On a Linux system, each file and directory is assigned access rights for the owner of the file, the members of a group of related users, and everybody else. Rights can be assigned to read a file, to write a file, and to execute a file (i.e., run the file as a program).

To see the permission settings for a file, we can use the ls command. As an example, we will look at the bash program which is located in the /bin directory:

[me@linuxbox me]$ ls -l /bin/bash
-rwxr-xr-x 1 root root 316848 Feb 27 2000 /bin/bash

Here we can see:

• The file “/bin/bash” is owned by user “root”
• The superuser has the right to read, write, and execute this file
• The file is owned by the group “root”
• Members of the group “root” can also read and execute this file
• Everybody else can read and execute this file

Question 112

-rwxrwxrwx

Answer 112

• or d = file or directory

first rwx: read/write/execute permissions for file owner
second rwx: read/write/execute permissions for the group owner of the file
third rwx: read/write/execute permissions for all other users

Question 113

Common permission settings

Answer 113

The ones beginning with “7” are used with programs (since they enable execution) and the rest are for other kinds of files.

Value:
Meaning
———-

777
(rwxrwxrwx) No restrictions on permissions. Anybody may do anything. Generally not a desirable setting.

755
(rwxr-xr-x) The file’s owner may read, write, and execute the file. All others may read and execute the file. This setting is common for programs that are used by all users.

700
(rwx——) The file’s owner may read, write, and execute the file. Nobody else has any rights. This setting is useful for programs that only the owner may use and must be kept private from others.

666
(rw-rw-rw-) All users may read and write the file.

644
(rw-r–r–) The owner may read and write a file, while all others may only read the file. A common setting for data files that everybody may read, but only the owner may change.

600
(rw——-) The owner may read and write a file. All others have no rights. A common setting for data files that the owner wants to keep private.

Question 114

chmod on directories

Answer 114

The chmod command can also be used to control the access permissions for directories. Again, we can use the octal notation to set permissions, but the meaning of the r, w, and x attributes is different:

r - Allows the contents of the directory to be listed if the x attribute is also set.
w - Allows files within the directory to be created, deleted, or renamed if the x attribute is also set.
x - Allows a directory to be entered (i.e. cd dir).

Here are some useful settings for directories:

Value:
Meaning
———–

777
(rwxrwxrwx) No restrictions on permissions. Anybody may list files, create new files in the directory and delete files in the directory. Generally not a good setting.

755
(rwxr-xr-x) The directory owner has full access. All others may list the directory, but cannot create files nor delete them. This setting is common for directories that you wish to share with other users.

700
(rwx——) The directory owner has full access. Nobody else has any rights. This setting is useful for directories that only the owner may use and must be kept private from others.

Question 115

sudo

Answer 115

With sudo, one or more users are granted superuser privileges on an as needed basis. To execute a command as the superuser, the desired command is simply preceded with the sudo command. After the command is entered, the user is prompted for the user’s password rather than the superuser’s

Question 116

Change file ownership

Answer 116

Notice that in order to change the owner of a file, you must be the superuser. To do this, our example employed the su command, then we executed chown, and finally we typed exit to return to our previous session.

chown works the same way on directories as it does on files.

Question 117

Job control

Answer 117

In the previous lesson, we looked at some of the implications of Linux being a multi-user operating system. In this lesson, we will examine the multitasking nature of Linux, and how this is manipulated with the command line interface.

As with any multitasking operating system, Linux executes multiple, simultaneous processes. Well, they appear simultaneous, anyway. Actually, a single processor computer can only execute one process at a time but the Linux kernel manages to give each process its turn at the processor and each appears to be running at the same time.

There are several commands that can be used to control processes. They are:

ps - list the processes running on the system
kill - send a signal to one or more processes (usually to “kill” a process)
jobs - an alternate way of listing your own processes
bg - put a process in the background
fg - put a process in the foreground

Question 118

List the processes running on the system

Answer 118

ps

Question 119

send a signal to one or more processes (usually to “kill” a process)

Answer 119

kill

While the kill command is used to “kill” processes, its real purpose is to send signals to processes. Most of the time the signal is intended to tell the process to go away, but there is more to it than that. Programs (if they are properly written) listen for signals from the operating system and respond to them, most often to allow some graceful method of terminating. For example, a text editor might listen for any signal that indicates that the user is logging off, or that the computer is shutting down. When it receives this signal, it saves the work in progress before it exits. The kill command can send a variety of signals to processes.

Question 120

an alternate way of listing your own processes

Answer 120

jobs

Question 121

put a process in the background

Answer 121

bg

Question 122

put a process in the foreground

Answer 122

fg

Question 123

jobs vs ps

Answer 123

Suppose that you have a program that becomes unresponsive; how do you get rid of it? You use the kill command, of course. Let’s try this out on xload. First, you need to identify the process you want to kill. You can use either jobs or ps, to do this. If you use jobs you will get back a job number. With ps, you are given a process id (PID).

Question 124

kill -l

Answer 124

gives you a list of the signals it supports. Most are rather obscure, but several are useful to know:

Signal # / Name / Description

1
SIGHUP
Hang up signal. Programs can listen for this signal and act upon it. This signal is sent to processes running in a terminal when you close the terminal.

2
SIGINT
Interrupt signal. This signal is given to processes to interrupt them. Programs can process this signal and act upon it. You can also issue this signal directly by typing Ctrl-c in the terminal window where the program is running.

15
SIGTERM
Termination signal. This signal is given to processes to terminate them. Again, programs can process this signal and act upon it. This is the default signal sent by the kill command if no signal is specified.

9
SIGKILL
Kill signal. This signal causes the immediate termination of the process by the Linux kernel. Programs cannot listen for this signal.

Question 125

What to do if you have a program that is hopelessly hung up and you want to get rid of it?

Answer 125

Use the ps command to get the process id (PID) of the process you want to terminate.
Issue a kill command for that PID.
If the process refuses to terminate (i.e., it is ignoring the signal), send increasingly harsh signals until it does terminate.

[me@linuxbox me]$ ps x | grep bad_program
PID TTY STAT TIME COMMAND
2931 pts/5 SN 0:00 bad_program
[me@linuxbox me]$ kill -SIGTERM 2931
[me@linuxbox me]$ kill -SIGKILL 2931

In the example above I used the ps command with the x option to list all of my processes (even those not launched from the current terminal). In addition, I piped the output of the ps command into grep to list only list the program I was interested in. Next, I used kill to issue a SIGTERM signal to the troublesome program. In actual practice, it is more common to do it in the following way since the default signal sent by kill is SIGTERM and kill can also use the signal number instead of the signal name:

[me@linuxbox me]$ kill 2931

Then, if the process does not terminate, force it with the SIGKILL signal:

[me@linuxbox me]$ kill -9 2931

Question 126

What are shell scripts?

Answer 126

With the thousands of commands available for the command line user, how can you remember them all? The answer is, you don’t. The real power of the computer is its ability to do the work for you. To get it to do that, we use the power of the shell to automate things. We write shell scripts.

In the simplest terms, a shell script is a file containing a series of commands. The shell reads this file and carries out the commands as though they have been entered directly on the command line.

The shell is somewhat unique, in that it is both a powerful command line interface to the system and a scripting language interpreter. As we will see, most of the things that can be done on the command line can be done in scripts, and most of the things that can be done in scripts can be done on the command line.

We have covered many shell features, but we have focused on those features most often used directly on the command line. The shell also provides a set of features usually (but not always) used when writing programs.