Managing Storage Flashcards
Pseudo (Virtual) Filesystems
Main locations of pseudo FS
Only exists in RAM
Two primary locations: /proc, /sys
What /proc and /sys dirs contain
- /proc contains info about processes running on the system.
- /sys contains info about system’s hardware and kernel modules
Main Filesystem locations
/ -root dir
/var -log files and dynamic content such as web sites, email files
/home -user’s home dir
/boot -often on a separate partition, and where Linux kernel and supporting files are stored, which help a system to boot up
/opt - optional software, used by third-party vendors
Swap space types
- swap partition - separate partition
2. swap file- similar to page file, and much slower than dedicated partiton
Recommended sizing of swap
- Older rules: 1.5-2 times of RAM
2. Not less than 50% of RAM
Mount point
Taking a partition and mounting it to a dir
List all partitions and mount points that are in use on the system
mount Disk labeling: xvda- virtual disk on zen hyperviser sda- sata or scsi nvm- ssd
List block devices
The same and show FS type for partititons
lsblk
lsblk -f
List information about partitions including type, size etc
fdisk -l /dev/sda or fdisk -l /dev/xvda
Show summary of devices and their partitions
partprobe -s
Show summary of swap usage, its size, type(partition or file)
swapon –summary
Legacy MBR partitions.
Utility to create partitions and its usage
1. fdisk /dev/sda m- help n -new p- partition table type: primary or secondary partition number: 1 w -write 2. parted help mklabel msdos mkpart: choose primary partition File system type: ext2 or ext3 Start: 1 End: 900 and enter quit
Partition ID-s
83- standard Linux FS
82- Standard swap
8e- Linux LVM volumes
GPT partitions
gdisk /dev/sdb
parted (here for mklabel instead of msdos use gpt)
Create swap partition
gdisk /dev/sda
here use Hex code=8200 (swap)
instead of default 8300
Format a partition to be used as swap space
mkswap -L SWAP /dev/sda2
Enable swap partition
swapon -a (reread /etc/fstab, old entries will be skipped)
specified:
swapon -U (UUID)
swapon -L SWAP
free -m to check if available free space is increased
Make swap permanent
vim /etc/fstab
add entry:
LABEL=SWAP [or /dev/sda2 or UUID=…] swap [mount point] swap [type] defaults 0 [dumping or backup] 0 [fs checking]
Disable swap
swapoff -L SWAP
Types of Linux FS
Non-journaling: ext2-legacy, released in 1993 Journaling: ext3-released in 2001 ext4-released in 2006 xfs (very fast and works great with small files) -released in 1991, ported in Linux in 2001. Default in RHEL, Centos7 \_\_\_\_\_\_\_\_\_\_\_\_\_\_ Btrfs uses Copy on write COW, uses subvolumes similar to partitions, and can be accessed like a dir Uses snapshots \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ FAT (Linux uses VFAT)
Display FS’s UUID
blkid
Create a file system on a new partition
mkfs -t ext4 -L SERV /dev/sdb1
mkfs.xfs -L OPTIONAL /dev/sdb2
under the hood mkfs runs mke2fs for ext2-3 FS types
/etc/mke2fs.conf contains deafult settings of blocksize,inode-size etc to apply when creating new fs for a partition
Mount/unmount dir
mount /dev/sda1 /opt
umount /opt
Show mounting points of specified types
mount -t ext4
Source file for mount command
/etc/mtab
Scan /etc/fstab to mount new fs listed in the file
mount -a
Meaning of defaults option in /etc/fstab
rw - read/write
suid- set user id bits to take effect
dev- treat that particular filesystem as a device that can be used to write data to
exec- permit execution of binary
auto- can be mounted with -a option
nouser -only root or sudo users can mount
async- all io should be done asynchronously
____________
other no default options:
user -allow user to mount fs
ro- mount fs read-only
remount- try to remount fs which was already been mounted
noexec-no binaries will be run from that fs
Unmount fs
umount /dev/sda1 or umount -L SRV (to do fsck further on)
umount /dir
Mount fs having a label
mkfs.xfs -L OPTIONAL /dev/sdb2
mount -L OPTIONAL -t xfs -o ro, nouser,noexec /opt
ls /opt
Mount cdrom to media dir
Mount iso file to media dir
mount /dev/sr0 /media
mount /root/install.iso -o ro,loop /media
after that umount /media
Multipath storage
DM-Multipath -kernel module that makes routing decisions
multipath- command responsible for listing devices
Multipathd- daemon that monitors paths, reactivates them
kpartx- command to create mapper entries
HBA -host bus adaptor (to connect a server to the storage)
Common utility to check FS
Get only report
fsck /dev/sda1
fsck -r LABEL=SRV (unmount fs first)
Other utilities to check FS
- ext2-4
- xfs
1.
1.1 e2fsck (for ext2-4 fs types) checks and repairs
e2fsck -f /dev/sda1 (show steps of checking)
e2fsck -p /dev/sda1- repair
2.
2.1. xfs_repair
2.2. xfs_fsr- similar to defrag on windows
2.3 xfs_db- debug xfs FS
unmount /opt
xfs_db /dev/vdb1 (run freesp to show free space, run frag to show frragmentation)
Adjust FS parameters for ext2-4 FS types
tune2fs -l /dev/sdb1 (list parameters applied)
tune2fs -i 3w /dev/sdb1 (perform next check after 3 weeks)
File space usage
du -h du -h /home/user/ --max-depth=2 du --inodes /etc du -sh- get only summary du -sh /home
Show available disk space on FS
free space of inodes: df -i
free space in HR format: df -h
df -h /dev/sda1
df –total -h (including pseudo fs, total will include swap and RAM)
Show RAM+swap
free -m
List all physical volumes
pvscan
Initialize PV for use by LVM
pvcreate /dev/sdb /dev/sdc
Create a new volume group on block devices
vgcreate my_vg /dev/sdb /dev/sdc
Scan for volume groups
vgscan
Create a logical volume
lvcreate -n(name) my_lv -L (size) 10G my_vg
Display information about logical volumes
lvdisplay my_vg/my_lv
Create file system on VG and logical volumes
mkfs.ext4 /dev/my_vg/my_lv
Create dir and mount to LVM
mk dir /mnt/volume
mount /dev/my_vg/my_lv /mnt/volume
Report CPU and i/o stat for devices and partitions
iostat
Test disk latency on a system
ioping -c 12 (count) . (this current drive)
List all limits on the system
ulimit -a
Soft and hard limits
Soft-limit that user or process can change without su privileges. Hard- up to which it can be changed without invoking su privileges.
I/O schedulers
I/O schedulers attempt to improve throughput, but it may lead to some I/O requests waiting for too long, causing latency issues. I/O schedulers attempt to balance the need for high throughput while trying to fairly share I/O requests amongst processes.
____________
cat /sys/block/sda/queue/sheduler
bfq (Budget Fair Queuing) Multiqueue -for slower I/O devices.
kyber (Multiqueue)-
Designed for fast multi-queue devices and is relatively simple (sync and async)
none (Multiqueue)
The multi-queue no-op I/O scheduler. Does no reordering of requests, minimal overhead. Ideal for fast random I/O devices such as NVME.
Not multiqueue:
deadline -FIFO (read queue and write queue)
cfq (Completely Fair Queueing)-roundrobin, each process gets a clice of time
noop (No-operation)
Performs merging of I/O requests but no sorting. Good for random access devices (flash, ramdisk, etc) and for devices that sort I/O requests such as advanced storage controllers.
Display disk limits (quotas)
quota -u [user] user
display quota on local fs only(no NFS): quota -l
display quota on all NFS FS: quota -A
