Midterm Review Flashcards
What is the “stat” structure?
A structure that can be obtained at runtime,
it contains statistics and information about a specific file.
Often used to check the access mode bits on a file before attempting to use it.
Shell Command Equivalents:
ln
From unistd.h library:
link( old_filepath, new_filepath);
Equivalent to:
%ln old new
Libraries:
Static Linking
vs
Dynamic Linking
- Static Linking
- Every routine needed from the library is included in the executable file
- Routines are PART OF the program
- Dynamic Linking
- References to library routines are left to resolve during execution
- Library objects not loaded until they are needed at runtime
How to list inodes on the shell
ls -i
Shell Command Equivalents:
mkdir
From unistd.h library:
mkdir( “newpath”, 0777);
Note: Not sure what the 0777 flag does
Equivalent to:
%mkdir newpath
How to get the
Status of a file
(in code)
Need to use the “stat” struct
- located in the “sys/stat.h” library
Use one of the functions:
- stat( “pathname”, stat s)
- fstat()
- lstat()
To return a stat structure, placed in s
Then check s->st_mode to determine the file type and if the program has the correct permissions to access
Physical Disk Organization:
Key Points
- Disk is arranged into blocks
- some for data
- some for management
- Each inode corresponds to one file and the data blocks associated with that file
- Physical layout is not consecutive, file blocks can be all over the place
How do inodes handle larger file sizes?
- The inode only has 13 pointers, so an indirect approach is used
- File blocks can each contain many pointers
- Pointers are classified into levels L0, L1, L2, …
- This allows a hierarchy of blocks to be created to handle arbitrary file sizes
- Level L0 is a pointer that points directly to an actual datablock
- L1 points to a block of L0 pointers, L2 to a block of L1 pointers, and so on.
2 “Families” of Common Errors
- Exogenous (Outward error)
- Hardware Issues
- Computer Crash/error
- Software Issues
- Data Issues:
- incorrect data
- too much data
- too fast/slow
- Hardware Issues
- Endogenous (Problems with development process/team)
- Analysis
- Wrong Problem
- Wrong Goals
- Wrong User
- Specification
- incorrect
- incomplete
- inconsistent
- Logic
- forgetfulness
- stupidity
- laziness
- Analysis
unistd.h
Methods that are equivalent to
common shell commands (5)
- link()
- equivalent to “ln”
- unlink()
- equivalent to “rm”
- chdir()
- equivalent to “chdir”
- mkdir()
- equivalent to “mkdir”
- rmdir()
- equivalent to “rmdir”
Useful Names and Functions:
Input/Output (I/O)
Library: stdio.h
- Named Values:
- NULL = 0
- EOF = -1
- Standard File Descriptors:
- stdin
- stdout
- stderr
- Functions:
- getchar() - gets next char from file or EOF
- putchar(c) - write one char to file
- printf(…) - high level output formatting to file
- fgets() - read one line from file
Shell Utility Toolkit:
Miscellaneous Tools (5)
- echo - echo arguments on stdout
- cat - copy a file, or files, to stdout
- tr - translate characters
- date - echo current date and time
- man - see manual page of a command
The Shell:
Redirection of
Input
By Default, input is normally stdin
Use “
This allows a file to be used as input.
Example:
% grep -e “abc” < myFile.txt
Layers of the Unix Architecture
- Kernel
- Essential Basic Services
- Only one kernel
- General functionality
- Libraries
- Useful additional services
- Many libraries
- Specific functionality
- Programs
- Executable files
- Lots of programs
- Specific functionality
- Shell
- Command Line environment
- Few
- Powerful
- Users
- Humans using the computer
Difference between
real uid
and
effective uid
Real uid
- the uid of the actual user who is logged in
Effective uid
- uid temporarily used for the lifetime of a process
Most of the time, the effective uid and real uid are the same,
but Administrative programs are able to operate with a different uid than the invoker.
Library for Working with Files
fcntl.h
Opening a File:
Basic Flags
Used in the open(path, flag, mode_permissions)
Defined in the fcntl libary
- O_CREAT
- Create File
- O_RDONLY
- Open for reading
- O_TRUNC
- Set size to 0
- O_WRONLY
- Open for writing
- O_APPEND
- Set offset to EOF before each write
- O_RDWR
- Open for both reading and writing
Unix Innovations:
5 Areas
- File System
- Organization
- Content
- Processes
- SImplicity
- Uniformity
- Programming
- C language
- OS Interface
- Shell
- Pipes
- Tools
- Malleability
- Portability
- Open Source
Which library contains the “fork()” function?
unistd.h
What is a
Process?
- An instance of an executing program
- A fundamental concept in Computer Science and a powerful coding technique
- Performs independent computations
- Processes interact via messages
- Identified using a process id (pid)
File Structure
- Regular Files are just arrays of bytes, starting at index 0
- Uses a byte offset to track the “position” while reading and writing to the file/data
- This increases flexibility:
- Can reposition offset to overwrite or reread
- Can move offset to end+1 to append without overwriting
Unix File System Model:
3 Types of Files
- Regular Files
- Array of bytes
- Simple sequence with arbitrary file size
- Directory Files:
- Single hierarchy of files
- Very deep nesting
- Special Files
- Things that LOOK like files, but are not actually on the disk
- terminals,
- network connections,
- memory,
- pipes,
- etc
- Things that LOOK like files, but are not actually on the disk
Two Building Blocks
of
Unix Data
- Files
- Actual storage areas on disk
- Store data
- Processes
- Manipulate data
- Running programs in memory
Program Memory:
Two Methods to get an area from the Heap
- malloc(size)
- memory allocation
- allocates a single area of the heap of the specified size, then is casted to a pointer
- calloc(n, size)
- allocate multiple n chunks of memory, for use by arrays
Shell Utility Toolkit:
Basic
Line Oriented Processing Tools (8)
- grep
- find patterns
- sed
- edit lines
- cut
- extract columns
- sort
- sort lines into specified order
- uniq
- display unique lines in a sorted file
- tail
- display last lines of a file
- head
- display fist lines of a file
- wc
- count words, lines or characters
Directory Structure:
How Move (mv)
operations work
- Physical data is not moved around on the disk, and will keep the same inode
- The inode number is copied from the table in the source directory
- Then added to the table in the destination directory with a new filename
- The old reference is then deleted from the table in the source directory
- Example:
mv dir_1/derp.txt dir_2/samederp.txt
3 Major Categories of
Process Operations
- Process Creation
- Process Communication
- Process Control
File Information
in the File Descriptor array
The File Descriptor Array manages information about the state of open files
- File location on disk
- Mode(read/write) the file is open in
- Current offset
Libaries:
What are
“Shared Libraries”?
Libraries that are prepared for
Dynamic Linking
These are also called
Dynamic Linking Libraries (DLLs)
Structure of a
Directory
- The directory is a specific type of file
- Contains a table of inodes paired with filenames
- inodes are identified with inode numbers
- The table contains a reference to the directory itself (.)
- and to it’s parent directory( .. )
dup:
Two Forms
and their use
Form 1:
new_fd = dup( old_fd );
When using the first form, the kernel picks a new file descriptor number and it is returned. It will be the smallest index that is not yet used for a file descriptor.
Form 2:
dup2( old_fd, new_fd);
The second form lets the programmer specify the new file descriptor number.
Either way, both old and new file descriptors now index the same value(file).
Unix Directory Structure:
Important Directories(7)
/ - Root Directory
/bin - commonly used commands
/dev - Device Files
/etc - System Maintenance Files
/lib - System Libraries
/tmp - Temporary Files
/usr - User Files
What is a Pipe?
- A kernel buffer with a “read” end and a “write” end
- The kernel presents each end as a file descriptor
- Processes can use the pipe with normal read/write routines,
- One process writes
- Another process reads
- The writer waits if the buffer is full
- The reader waits if the buffer is empty
- A parent process creates file descriptors with the
- pipe() routine
- These descriptors are preserved during fork() and exec(), and can be used by a child process.
Unix Innovations:
Programming
- C Language
- As efficient and flexible as assembly
- But more readable
- OS Interface
- Convenient subroutine interface to all system services
Unix Innovations:
Shell
- Pipes
- Simple model for connecting together programs
- Tools
- Rich set of utilities for common text manipulation tasks
Dynamic Linking:
Disadvantages
Disadvantages:
- Libary version control
- Can the executable find the right version of the library routine?
- “Thrashing” of library routines in memory due to a constrained environment
Writing to a File
Use the write() method from the unistd.h
actual = write(fd, buffer, actual)
- writing begins at the current offset
- bytes in buffer are written to the file
- if “fd” was opened with the O_APPEND flag, the offset is set to EOF + 1, so each write appends
- After the write, offset is incremented by “actual”, or the number of bytes written
The Shell:
Redirection of
Output
By default, output of programs is to stdout.
Use “>” to redirect the output to a different file descriptor.
This allows output of a program to be written to file.
Alternatively, use “>>” to append the file instead of writing over it.
Example:
% grep -e “abc” > my_output_file.txt
% grep -e “abc” >> my_output_file.txt
How to change the
Effective uid
of a program?
- Must use the “set-uid” bit on executables
- The Super User must turn on the set-uid bit for trusted programs
- The effective uid becomes that of the executable’s owner for the duration of the process
Turning on the set-uid bit:
Use chmod, as super user. It is the 4th of the permission bits:
-rws –x –x
Shell Command Equivalents:
mv
From stdio.h library:
rename( old_filepath, new_filepath);
Equivalent to:
%mv old new
Unix Innovations:
Malleability
- Portability
- both OS and tools are written in C
- both can be easily ported to other machines
- Open Source
- source can be licensed and modified for experiments
The Shell:
Redirection of
Input
By Default, input is normally stdin
Use “
This allows a file to be used as input.
Example:
% grep -e “abc” < myFile.txt
Design Consequences of Inodes
- Directories reference files only through inode numbers
- moving files between directories is very fast
- and, the operation is atomic (as are link and remove)
- Two directories can include the same inode number
- a file can be shared, appearing in several places with same or different name
- also, no special cases for “shortcuts” etc.
- Space is freed only when inode link & open counts are both 0
- removing a file may not create free space
- files can’t disappear while a process is working on them
- also, no special cases for “sharing violation” etc.
- An inode number is unique only within a file system
- files can’t span partitions or disks, or be moved to new disks
- and, files can’t be bigger than a single physical disk
- Files don’t have “type” (beyond regular vs. directory)
- no optimizations for databases, no self-identifying objects, no new types
- but, easy to code general programs such as “cp”, “mv”
- Plus, as we will see, file security information is in the inode
- data is protected even when files are shared (details later)