Lec 3: Virtualization & Containerization Flashcards

1
Q

What are the relevant OS concepts?

A

Process scheduling
Virtual memory

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is a process?

A

A process is a program that is executed. It is a basic unit of execution in an OS.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is process scheduling (time-sharing)?

A

-Each process is scheduled and executed within their time share.
-The number of processes can be much larger than that of the CPUs.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What are the different queues in process scheduling?

A

Job Queue
Ready Queue
I/O Waiting Queue > I/O

(then CPU > Exit)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Kernel mode: Ring number

A

Ring 0, most privileged

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

User mode: Ring number

A

Ring 3

Other: Ring 1 & 2

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

How many protection levels (rings) or CPU privilege levels (CPLs) does x86 has?

A

four: Ring 0 (most privileged) to 3

Modern OS (Linux, Windows, MacOS) only use ring 0 and 3

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Kernel mode vs user mode: executing processes

A

When CPU is in the user mode, it cannot execute sensitive/privileged instructions (e.g., HLT = halt / stop the processor’s operations)

User Mode = User Processes
Kernel Mode = Process Scheduling

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is involved in a virtual memory implementation?

A

Process A
Virtual memory
Memory map
Physical memory
Disk

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is on-demand paging?

A

Only when there is a need to access code / data that the OS will allocate a physical page, otherwise the code / data will just be stored in the disk

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is swapping in / out?

A

The least recently used pages are swapped out from physical memory to the disk

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What are the three steps to execute a process for the first time?

A
  1. CPU allocates a physical page based on the memory map
  2. CPU loads the corresponding data / code from disk to the allocated page
  3. CPU executes the code by fetching code from the allocated page
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is a memory map?

A

A memory map maintains the mapping between virtual and physical memory

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is a shared memory?

A

Two processes from virtual memory uses the same physical page, allowing exchanging of data

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Memory map implementation

A

Page table
In a modern OS, the OS uses a set of page tables to map virtual memory within a process to their corresponding physical memory in main memory.
Used to map virtual memory within a process to a corresponding physical memory in main memory
Used to implement the memory map
Established by the memory management unit MMU (a hardware component)
All mappings are maintained as page tables

Virtual address VA
pointing to a page-aligned memory location in a virtual page.

Physical address PA
pointing to a page-aligned memory location in a physical page

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is MMU?

A

Memory management unit
Page tables reside in MMU

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What is a page-aligned address?

A

In a page-aligned address, VA is mapped to PA via MMU.

Additional note: VA points to the base address of the virtual page
If VA = 0x60001000, then PA = 0x70001000
If VA1 = 0x60001800, then PA1 = 0x70001800 (same last 3 digits)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Memory map implementation: x86-32

A

Page table:
In a modern OS, the OS uses a set of page tables to map virtual memory within a process to their corresponding physical memory in main memory.
Has 2-level page tables: page directory + page table

32-bit version of x86, known as IA-32 (Intel Architecture 32-bit), where 32-bit virtual/linear addresses are used.

Division of the 32 bits:
31-22 (10 bits): index page directory; 2^10
21-12 (10 bits): index page table; 2^10
11-0 (12 bits): index physical page; will be the same last 12 bit of the physical page; the physical page is mapped to the virtual page where the linear address resides

size of entries: 2^12 / 2^10 = 2^2 = 4 bytes for a single entry

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Memory map implementation: x86-64

A

63-48: sign-extended, same bit as bit47; allowed entries are 2^9
47-39: page map level 4 offset
38-30: page-directory pointer offset
29-21: page-directory offset
20-12: page-table offset
11-0: physical-page offset

size of entries: 2^12 / 2^9 = 2^3 = 8 bytes for a single entry

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What is CR3?

A

Control register 3 stores the physical address of the base of the Page Directory (in x86-32) or the base of the Page-Map Level 4 (PML4) table (in x86-64).
This is essential because the memory management unit (MMU) starts its translation of a virtual address by referring to this base address.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What is virtualization?

A

Virtualization is the ability to run multiple operating systems on a single physical system and share the underlying hardware resources

Allows one computer software (called Virtual Machine Monitor or Hypervisor) to provide the appearance of many computers (called virtual machines).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What are the goals of virtualization?

A

Provide flexibility for users
Amortize hardware costs
Isolate completely separate users

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What are the characteristics of virtualizable computer architectures

A

First, the VMM is in complete control of all hardware resources, such as CPUs, memory, disk, etc.

Second, the VMM provides a virtualized hardware environment for a guest OS. The environment looks identical to its physical counterpart.

Last, the OS running in this environment shows minor decreases in performance compared to its execution in a bare-metal environment.

24
Q

What are the VMM types?

A

Type 1 Hypervisor: Bare-metal/Native architecture
Hardware > Hypervisor > Guest OS + Guest OS
VMM or Hypervisor is installed directly on hardware as the most privileged software.
Acknowledged as preferred architecture for mainstream public clouds

Type 2 Hypervisor: Hosted architecture
Hardware > Host OS > Hypervisor > Guest OS + Guest OS
Install as an application on an existing host OS (Win, Linux, MacOS)

25
Q

What are examples of Type 1 Hypervisor?

A

KVM
Xen
Hyper-V

26
Q

What are examples of Type 2 Hypervisor?

A

VirtualBox
UTM
VMware

27
Q

VMM Implementation on x86-64

A

Hardware support: Intel and AMD assist virtualization
-Intel VT-x and AMD SVM: CPU and memory virtualization
VT-x: Virtualization Technology for Execution
SVM: Secure Virtual Machine

-Intel VT-d and AMD Vi: Device virtualization
VT-d: Virtualization Technology for directed I/O
Vi: Virtualization for I/O

28
Q

Describe what virtualization is. Describe three types of virtualization (Language, Operating System and Hardware) and their respective attribute.

A

Virtualization is a technology that allows multiple virtual instances of resources (e.g., operating systems) to run on a single physical machine.

-Language virtualization: provides a virtual runtime environment (e.g., JVM) that allows code written in a specific programming language (e.g., Java) to execute.
—-Cross-OS: code can be executed in a runtime environment across OSes without modification.

-Operating system virtualization: allows multiple user-space instances (e.g., docker containers) within a single operating-system kernel (e.g., Linux kernel).
—-Limited Isolation: instances share the host OS kernel, which means a vulnerability in the kernel could affect all instances.

-Hardware virtualization: creates multiple virtual machines sharing a single physical server.
—-VM Snapshots: allows saving the current state of a VM
—-Live VM Migration: enables moving runtime VMs between physical hosts seamlessly.

29
Q

CPU virtualization: Intel VT-x & AMD SVM
What are the two VT-x operating modes?

A

-Less-privileged mode (guest or VMX non-root) for guest OSes (Ring 0 > 3)
-More-privileged mode (host or VMX root) for VMM (Ring 0 > 3)

30
Q

What is the structure of the VMX non-root?

A

VMX non-root Guest OS:
Ring 0: Kernel
Ring 3: Apps

31
Q

What is the structure of the VMX root?

A

VMX root VMM:
Ring 0: Hypervisor core
Ring 3: VM Management Utilities

32
Q

What is the difference between VMX root vs non-root?

A

VMX root: VMM runs directly on hardware
VMX non-root: Guest OS runs directly on hardware with a few exceptions (e.g., privileged/sensitive instructions)

33
Q

CPU virtualization: Intel VT-x & AMD SVM
What are the two transitions?

A

VM entry to non-root operation
VM exit to root operation

34
Q

Memory virtualization: Intel EPT & AMD NPT

A

Host Physical Memory (Host PA) > extended page table >
Guest Physical Memory (Guest PA) > guest page table >
Guest Virtual Memory (Guest VA)

VM1 covers Process 1 > GPA > guest page table > GVA

47-39: points to the address of EPT PDPT inside EPL PML4
38-30: points to the address of EPT PD inside EPT PDPT
29-21: points to the address of EPT PT inside EPT PD
20-12: points to the address of Page inside EPT PT (page table)
11-0: The last 12 bits of GPA refers to the last 12 bits (11-0) of PA

EPT Pointer points to EPL PML4

35
Q

What is a container?

A

A container is a sandboxed process running on a host OS that is isolated from all other processes running on that host OS

36
Q

What is a container image?

A

A container image is a stand-alone and executable software package (including software dependencies, binaries, etc) that contains everything needed to run a user application.

A container is a runnable instance of the image.

37
Q

Difference between virtual machines and containers in terms of structure/architecture

A

VM:
Infrastructure > Hypervisor > (Guest OS > Bins/Libs > App1)
(VM)

Containers:
Infrastructure > OS > Container Engine > (Bins/Libs > App1)
(Container)

38
Q

Containerization vs. Virtualization

A

Granularity
Containers are an abstraction of the process layer and VMs are a simulation of the hardware layer.

Overhead
-Required resources: containers are created to run one application and VMs support a whole OS.
-Efficiency: containers are launched to run an application. VMs need to boot up an entire OS.

Security/Isolation
Containers are isolated from each other at the process level. VMs are isolated at the OS level.

39
Q

Backbones of containers (3)

A

namespaces
cgroups
copy-on-write filesystem

40
Q

What are namespaces?

A

-Namespaces limits what a container can see
-Provide containers with their own view of system resources (e.g., network)

41
Q

What are cgroups?

A

-cgroups: limits how much a container can use
-Control Groups: manage resource usage and limits for processes within a container.

42
Q

What is copy-on-write filesystem?

A

copy-on-write filesystem: lightweight container
-Allow containers to share read-only underlying OS files. If a container write-access a file, a copy is created and updated accordingly.
-Similar to the copy-on-write fork

43
Q

What is a Docker container?

A

-Docker provides an interface on top of the techniques
-Popularized containers (a standard unit of software)

44
Q

Docker commands

A

docker run – Runs a command in a new container.
docker start – Starts one or more stopped containers
docker stop – Stops one or more running containers
docker build – Builds an image form a Docker file
docker pull – Pulls an image or a repository from a registry
docker push – Pushes an image or a repository to a registry
docker export – Exports a container’s filesystem as a tar archive
docker exec – Runs a command in a run-time container
docker search – Searches the Docker Hub for images
docker attach – Attaches to a running container
docker commit – Creates a new image from a container’s changes

45
Q

Describe what containers are with reference to Docker and discuss their similarities and differences from operating system virtualisation perspective as provided by VirtualBox.

A

A container is a sandboxed process running on a host OS that is isolated from all other processes running on that host OS

Granularity
Containers are an abstraction of the process layer and VMs are a simulation of the hardware layer.
Similarity: Both allow multiple applications to run on the same hardware without interference

Overhead
-Required resources: containers are created to run one application and VMs support a whole OS.

-Efficiency: containers are launched to run an application. VMs need to boot up an entire OS.

Security/Isolation
Containers are isolated from each other at the process level. VMs are isolated at the OS level.
Similarity: Both provide isolation in terms of running apps to avoid conflicts in software dependencies

Similarity: Both are ways of providing consistency across environments

46
Q

docker command: check what is running

A

docker ps -a

47
Q

docker command: stop and remove a container

A

docker stop my-app
docker rm my-app

48
Q

docker command: build docker image

A

docker build -t my-apache2 .

if with permission errors, run:
sudo usermod -a -G docker <username></username>

49
Q

docker command: run docker image

A

docker run -p 80:80 -dit –name my-app my-apache2

Open a browser and access address: http://localhost or http://127.0.0.1.

50
Q

[1] Create a security group

A

aws ec2 create-security-group –group-name <student>-sg --description "security group for development environment"</student>

51
Q

[2] Authorise inbound traffic for ssh

A

aws ec2 authorize-security-group-ingress –group-name <student>-sg --protocol tcp --port 22 --cidr 0.0.0.0/0</student>

52
Q

[3] Create a key pair

A

aws ec2 create-key-pair –key-name <student>-key --query 'KeyMaterial' --output text > <student>-key.pem</student></student>

To use this key on Linux, copy the file to a directory ~/.ssh and change the permissions to:
chmod 400 <student>-key.pem</student>

53
Q

[4] Create instance

A

aws ec2 run-instances –image-id <ami> --security-group-ids <student>-sg --count 1 --instance-type t2.micro --key-name <student>-key --query 'Instances[0].InstanceId'</student></student></ami>

54
Q

[5] Add tag to instance

A

aws ec2 create-tags –resources <Instance> --tags Key=Name,Value=<student></student></Instance>

55
Q

[6] Get public IP address

A

aws ec2 describe-instances –instance-ids <Instance> --query 'Reservations[0].Instances[0].PublicIpAddress'</Instance>

56
Q

[7] Connect to the instance via ssh

[8] List the created instance using the AWS console - no code

A

ssh -i <student>-key.pem ubuntu@<IP></IP></student>

57
Q

Install, start, enable, check docker version

A

Install: sudo apt install docker.io -y
Start: sudo systemctl start docker
Enable: sudo systemctl enable docker
Check version: docker –version