Lect11 / 12 - Basic Analysis

Question 1

Explain the following ls commands:

• # ls -l
• # ls -R
• # ls -lt
• # ls -Rlth

Answer 1

• # ls -l => long list (size and dates)
• # ls -R => recursive list
• # ls -lt => sort by modification date
• # ls -Rlth => show human readable size

Question 2

How does this command looks like:

“find, starting in /mnt/analysis, by name, file with .jpg extension”

Answer 2

find /mnt/analysis -name *jpg

Other Parameters:

• -type f : files
• -type d : directories
• -name : by name
• -iname : by name (case insensitive)
• -many others : read the man page

Question 3

How can you execute a command based on the find result (e.g. md5hash)?

Answer 3

# find /mnt/analysis -type f -exec md5sum {} \;

or

# find /mnt/analysis -name *.jpg -exec md5sum {} \;

Question 4

How can you determine the filetype by showing the first line in hex?

Answer 4

# xxd cat_Warmer.jpg | head -n 1

Question 5

Explain the following grep options:

1. # grep -i
2. # grep -v
3. # grep ^string
4. # grep string$
5. # grep [char]
6. # grep ^$

Answer 5

1. :case insensitive
2. :reverse grep (exclude)
3. :string at beginning of a line
4. :string at the end of a line
5. :group of characters
6. :blank line

Question 6

How can you use a keyword list with grep?

Answer 6

# grep -abif analysis/searchlist.txt fat_fs.raw > analysis/hits.txt

Parameters are:

• -f <filename> : Use filename as keyword list</filename>
• -i : Case insensitive
• -a : Search binary as text
• -b : Return bit offset of keyword hits

Question 7

How can you replace control characters with new line characters using grep?

Answer 7

# tr ’[:cntrl:]’ ’\n’ < fat_fs.raw | grep -abif analysis/searchlist.txt

Question 8

Viewing of different file types. How can you:

1. stream the contents of a file to STDOUT
2. paging viewer for documents (and command output)
3. view MS Office docs from the command line (install first)
4. view MS Office xml format files (.docx, etc.)
5. for PDF files
6. to view graphics from the command line

Answer 8

1. cat
2. less
3. catdoc
4. catdocx
5. xpdf (or evince)
6. xv or display

Question 9

How to seek into the file to a specified number of bytes?

Answer 9

# xxd -s 75441 fat_fs.raw | head

Question 10

Name commands to parse structured data:

1. display contents of a file to STDOUT
2. same, but reverse the contents
3. search for patterns and strings in an object
4. sort the contents [reverse, unique, etc]
5. counts the number of words, lines and bytes in output
6. “an output processing tool”.
7. Replace or “translate” characters and sets
8. “Stream editor” - processes text in a stream

Answer 10

1. cat
2. tac
3. grep
4. sort
5. wc
6. awk
7. tr
8. sed

Question 11

How can you sort in alphabetical order and removes duplicates?

Answer 11

# cat names.txt | sort -u

Question 12

How can you change the field delimiter in awk?

Answer 12

# cat file.txt | awk -F “,” ‘{print $1 $2 $3”\t”$NF}’

or

# awk -F “,” ‘{print $1 $2 $3”\t”$NF}’ <filename></filename>

$NF is normally the last field of a line as it stands for “number of fields”.​

Question 13

awk examples:

How to add tabulator in the output?

Answer 13

cat text.txt | awk ‘{print $1 “\t” $2}’

Question 14

awk examples?

How to omit the header record?

Answer 14

# awk ‘NR!=1{print $1}’ file1

Question 15

awk examples:

Print entire file content?

Answer 15

# awk ‘{print $0}’ file1

or

# awk ‘1’ file1

Question 16

awk examples:

Use special field separator

Answer 16

# awk ‘{print $1,$3}’ FS=”,” file1

Question 17

awk examples:

Use comma to separate output

Answer 17

awk -F “,” ‘NR!=1{print $1,$3}’ OFS=”,” file1

Question 18

What is sed?

Answer 18

sed is a stream editor. A stream editor is used to perform basic text transformations on an input stream (a file or input from a pipeline). While in some ways similar to an editor which permits scripted edits (such as ed ), sed works by making only one pass over the input(s), and is consequently more efficient.

Question 19

sed examples:

Add “fruit” to the beginning of every line

Answer 19

$ sed ‘s/^/Fruit: /’ sample1.txt

Question 20

sed examples:

Add something to the end of the line

Answer 20

$ sed ‘s/$/ Fruit/’ sample1.txt

Question 21

sed examples:

To replace a particular char (a => A)

Answer 21

$ sed ‘s/a/A/’ sample1.txt

Question 22

sed examples:

Replace all occurrences (a => A)

Answer 22

$ sed ‘s/a/A/g’ sample1.txt

Question 23

sed examples:

Replace the 2nd occurrence (a => A)

Answer 23

$ sed ‘s/a/A/2’ sample1.txt

Question 24

sed examples:

Replace all occurrences from 2nd occurrence onwards

Answer 24

$ sed ‘s/a/A/2g’ sample1.txt

Question 25

sed examples:

Replace ‘a’ only in a specific line say 3rd line

Answer 25

$ sed ‘3s/a/A/g’ sample1.txt

Question 26

sed examples:

Replace or substitute ‘a’ on a range of lines

Answer 26

$ sed ‘1,3s/a/A/g’ sample1.txt

Question 27

sed examples:

To replace the entire line with something. For example, to replace ‘apple’ with ‘apple is a Fruit’.

Answer 27

$ sed ‘s/.*/& is a Fruit/’ sample1.txt

Question 28

sed examples:

Using sed, we can also do multiple substitutions. For example, say to replace all ‘a’ to ‘A’, and ‘p’ to ‘P’:

Answer 28

$ sed ‘s/a/A/g; s/p/P/g’ sample1.txt

or

$ sed -e ‘s/a/A/g’ -e ‘s/p/P/g’ sample1.txt

Question 29

sed examples:

Add header into first line

Answer 29

$ sed -i ‘1i Employee, EmpId’ empFile

Question 30

sed examples:

Add a line ‘——-‘ after the header line or the 1st line

Answer 30

$ sed -i ‘1a —————’ empFile

Question 31

sed examples:

Add a trailer line to this file

Answer 31

$ sed -i ‘$a —————’ empFile

Question 32

sed examples:

Add a record after a particular record

Answer 32

$ sed -i ‘/Hilesh/a Bharti, 1002’ empFile

Question 33

Logfile analysis:

Show only one filed of the log file

Answer 33

$ cat cisco.log | awk ‘{print $5}’

Output:

%DHCPD-4-PING_CONFLICT:

%DHCPD-4-PING_CONFLICT:

%SYS-5-CONFIG_I:

%SYS-5-CONFIG_I:

Question 34

Logfile analysis:

Group them together, count them, then sort them from the greatest to least number of occurrences

Answer 34

cat cisco.log | awk ‘{print $5}’| sort | uniq -c | sort -rn

Output:

8 %LINEPROTO-5-UPDOWN:

3 %SYS-5-CONFIG_I:

2 %DHCPD-4-PING_CONFLICT:

1 by

1 Software

1 Cisco

1 %SYS-5-RESTART:

Question 35

Logfile analysis:

Exclude garbage from output

Answer 35

cat cisco.log | grep %[a-zA-Z]*-[0-9]-[a-zA-Z]* | awk ‘{print $5}’ | sort | uniq -c | sort -rn

Output:

8 %LINEPROTO-5-UPDOWN:

3 %SYS-5-CONFIG_I:

2 %DHCPD-4-PING_CONFLICT:

1 %SYS-5-RESTART:

1 %SSH-5-ENABLED: