K8s Flashcards

1
Q

Service Account Tokens

A
  • ServiceAccount: A resource that provides an identity for processes running in pods
  • Every Pod has a service account. Created automatically
  • Allows apps to authenticate with the K8s API and access cluster resources
  • Service accounts are associated with access tokens that the pods can use to authenticate themselves to the k8s API server, enabling them to perform actions like reading or modifying resources within the cluster
  • When a pods starts, K8s mounts a Service Account token at a predefined location, eg /var/run/secrets/k8s.io/serviceaccount/token
  • This token is signed by the Kubernetes Certificate Authority. This token is signed
  • When decoded the ServiceAccount token contains things like namespace, pod name and uuid, service account name
  • Once the K8s API receives ServiceAccount token from the pod, it validates existence of the service account and checks if the system has any RBAC rules attached that allow/deny the current action
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
1
Q

OIDC

A

Used the OIDC authentication layer to validate and authenticate our external users managed on our corporate identity provider, JumpCloud

We used OIDC integration to access the K8s cluster

K8s uses OIDC validation mechanism to confirm it’s the correct user

Use the OIDC group and/or claims which get returned for looking up the RoleBinding

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Roles, Bindings, Subjects

A

Role: Grant permissions within a specific namespace. Contains rules that represent a set of permissions. There are no DENY rules, always additive

ClusterRole: Same idea but for the whole cluster and granting permissions for non-namespaced resources (like nodes)

RoleBinding: Grants the permissions defined in a Role to a user or set of users. Applies only within a specific namespace. Has a list of Subjects

ClusterRoleBinding: Same idea. Use this to grant permissions across the whole cluster

Subjects: Users, Groups, or ServiceAccounts

  • These Bindings get bound to subjects and they can be authenticated users returned from the OIDC integration or ServiceAccounts with a corresponding namespace
  • Within the Role you would define the rules which can be accessed, eg get, list, create, delete (these are the verb rules) against a set of resources (eg pod, services)
  • To change roles you need to remove the Binding, change the role, then rebind. Will get an error if you attempt to change a role when it’s already bound
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Other authentication strategies

A
  • Bootstrap tokens. Allow servers to authenticate for the first time
  • Static tokens: Only use for simple stuff
  • Authentication Proxy: K8s api server delegates the full authentication process to a proxy server
  • Webhook tokens: Similar to Auth Proxy
  • X.509 certs: Uses mTLS and extracts the username from the Common Name and the RBAC groups from the Organization fields. Each “user” has this cert and the k8s API verifies these certs using a certificate authority
    Impersonation:
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

RBAC

A
  • Adds security to a K8s cluster
  • Role Based Access Controls
  • RBAC defines roles, which are collections of permissions, and subjects which are service accounts or users. A subject is granted a role
  • What attributes on an object get used during RBAC authorization?
    User
    Group
    Resource
    Name

^ These attributes are validated against policies

Verbs: The operations we want to do with the resources

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Node Authentication and Authorization in EKS

A
  • Nodes in an EKS cluster authenticate themselves using AWS IAM.
  • Each node is associated with an IAM role that has specific permission
  • Node authorization is managed to K8s, specifically through the kubelet component running on each node. Can also use RBAC
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

K8s Components

A

Two primary components:

Control Plane (master nodes)
- API Server
- Control Manager
- Etcd
- Scheduler

Data Plane (worker nodes)
- Kube Proxy
- Kubelet
- Container runtime

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

K8s Component - API Server

A
  • All interactions with the cluster are made through the API server
  • The API server is responsible for validating all API requests and ensuring they are authorized
  • Adheres to a client-server model
  • Clients can be software developers, devops engineers, nodes, k8s scheduler, etc

Authentication: Verify the identity of the entity making the request, whether it’s a user, pod, or machine

Authorization: Checks if the requesting entity possesses adequate permissions for the operation

Only service which interacts directly with etc

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

K8s Component - Control Manager

A

Runs controllers to regulate the state of the cluster

Ensure desired state matches current state

Creates service accounts, new endpoints, manages load balancers, etc

Manage storage volumes

It encompasses several controllers, each focusing on a specific aspect of the cluster’s behavior. These controllers continuously monitor the cluster’s state and work towards reconciling any deviations from the desired state.

Examples: like the ReplicaSet controller ensuring the desired number of pod replicas are running, the Deployment controller managing the lifecycle of deployments, and the Node controller handling node-related tasks like detecting and responding to node failures.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

K8s Component - Etcd

A

Distributed KV store which only has cluster state

Always insert, never delete. Though runs compaction

No application data

Build on raft consensus algo

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

K8s Component - Scheduler

A

Watches for newly created Pods and assigns objects or pods to nodes

Knows about the quality and configuration of the nodes

It uses various policies to select the most suitable node for a pod based on factors such as resource requirements, node capacity, and pod affinity/anti-affinity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

K8s Component - Kubelet

A
  • An agent running on each worker node
  • Communicates with control plane
  • Receives pod definitions from master and interacts with container runtime. Connects to container runtime
  • Monitors health of the pods
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

K8s Component - Kube Proxy

A
  • The network agent
  • Runs on each node
  • Handles dynamic updates and maintenance of all networking rules on the node.
  • Configures iptables
  • Forwards networking requests to the pods
  • Allows services to communicate with each other
  • Managed by a DaemonSet
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Worker node

A

Has container runtime. Examples: docker, containerd, rkt

Pods are scheduled on worker nodes

Has networking

Pod is smallest scheduling unit in K8s

Worker node components
- Kubelet
- Kubeproxy

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Master node

A

Contains control plane components

Makes global decisions about the cluster

Manages cluster operations

Must be up all the time

If active master fails, then one of the replicas will take its place

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

K8s alternatives

A

Nomad
Docker swarm: very easy to install and use but designed for small scale

16
Q

K8s evolution history

A

K8s allows you to set boundaries on the applications running on physical servers. Without it apps would eat all resources, poor resource utilization

VM helped and allowed you to set resources utilization on each VM. Each VM is heavy, each have their own OS

Containers are light weight and can be ported across different clouds or environment

Portable isolated application environment

Can be densely packed, very good resources utilization

17
Q

Microservices

A

Light weight application which has its own environment requirements and dependencies

18
Q

K8s Namespace

A
  • A k8s cluster is comprised of multiple projects/applications with multiple teams
  • Allows isolation of resources within a cluster
  • A namespace is a logical isolation of resources, network policies, RBAC, and everything else
  • Each namespace can have different authentication policies, restrictions, etc
19
Q

How do you perform maintenance on the K8s node?

A
  • Often need to run administrative tasks due to upgrades, security patches, etc
  • To drain the k8s node can use the following

Marks the node as “un-schedulable”. Prevents pods from being scheduled on it
kubectl cordon

Removes existing pods from the node. Flag will skip over any pods that are part of a DaemonSet, which ensures that essential system services are not disrupted
kubectl drain –ignore-daemon set

20
Q

K8s Pod

A
  • A pod encapsulates one or more containers which share the same network namespace and can communicate with each other over the localhost interface
  • Each pod has a unique IP
  • All containers within a pod share the same IP
  • Can run a pod on a specific node by using node affinity. A node is assigned an arbitrary label and they are configured to be assigned to that node as per the label created
21
Q

Explain Daemonset

A
  • A type of controller that ensures a specific pod runs on each node in the cluster
  • A DaemonSet runs one pod per node
  • Used to run background services
  • Useful for system-level tasks like logging, monitoring, or node specific services.

Example: Use a DaemonSet to ensure that a logging agent runs on every node in the cluster to collect logs locally

22
Q

How to stop pods

A
  • K8s will first send a SIGTERM signal to the container’s main process and wait for a certain period of time, determined by the terminationGracePeriodSeconds parameter before issuing a SIGKILL
23
Q

What is a service?

A

An abstraction that defines a logical set of Pods and a policy by which to access them.

Enable communication between different parts of an application and provide load balancing and service discovery

Achieved through labels and selectors

How to find the Service?
1) env vars which automatically get added when resource is created
2) enable k8s add-on, core dns.
- Creates DNS record for each service
- Can find service in same namespace by just the name
- Other namespaces can access by adding suffix namespace

Cluster IP:
- Default service type
- Exposes the service on a cluster internal IP.
- Means that only the services inside the cluster can access it

Node Port:
- Exposes the service on a static port on each node in the cluster.
- Port is mapped to a particular service
- Node redirects to cluster IP of the service then request is forwarded to app
- This makes the service accessible from outside the cluster.
- Anyone who has the node IP/port can access it

Load Balancer:
- Provisions an external load balancer in the cloud infra and directs traffic to the K8s service.
- Allows you to expose your service to the internet

External IP:
- Use an external load balancer outside of K8s
- Not managed by K8s

Ingress: A K8s object which defines to expose services to external traffic. Can provide features like load balancing, SSL termination, etc

A service provides a stable and consistent IP address and DNS name for the pods

24
Q

Horizontal Pod Autoscaler (HPA)

A
  • A controller that automatically scales the number of pods in a ReplicaSet, Deployment, StatefulSet, or any other custom resources based on the observed CPU utilization of other metrics
  • Peregrine used keda (Kuberenetes Event Driven Auto scaling)
25
Q

Give an example of how K8s can be used to deploy a highly available application

A
  • Create a Deployment that defines the desired state of your application, such as the number of replicas, the image, the ports, etc.
  • Create a Service that exposes your application to other pods or external clients. The service acts as a load balancer that distributes traffic among your pods.
  • Create an Ingress that defines rules to expose your service to external traffic. The ingress controller routes the traffic to your service based on the host name or path.
  • Configure health checks for your pods using liveness and readiness probes. These probes allow Kubernetes to monitor the health and availability of your pods and restart them if they fail or become unresponsive.
  • Configure rolling updates for your deployment using updateStrategy parameters. These parameters allow you to perform zero-downtime updates to your pods without affecting the availability of your application.
  • Configure horizontal pod autoscaling for your deployment using HPA parameters. These parameters allow you to automatically scale your pods based on CPU utilization or other metrics.
26
Q

Admission Controller

A

Part of the auth flow

  1. Client Request
  2. Send to API server
  3. Authentication
  4. Authorization
  5. Optional admission controller
  6. Persist to etcd

Can accept or reject the request based on additional policies. For example resource quota, request count, custom business logic. Very granular

27
Q

K8s Job

A
  • One or more pods to create a task
  • Ensure task is completed successfully
28
Q

Affinity / Anti-Affinity

A

Pod affinity and pod anti-affinity allow you to specify rules about how pods should be placed relative to other pods

The rules are defined using custom labels on nodes and label selectors specified in pods

Anti-affinity rules allow you to prevent pods of a particular service from scheduling on the same nodes as pods of another service that are known to interfere with the performance of the pods of the first service. Or, you could spread the pods of a service across nodes or availability zones to reduce correlated failures

29
Q

Liveness / Readiness Probes

A

Liveness
- A probe which checks if an application is running based on any arbitrary business logic. If it fails the container gets restarted
- Is the container in a healthy state

Readiness
Readiness probes enable developers to instruct Kubernetes that a running container should not receive traffic until additional tasks are completed, such as loading files, warming caches, and establishing network connections

30
Q

Tainting a node

A

Taints are used to repel pods from being scheduled onto certain nodes unless the pods have corresponding tolerations. They are typically used to mark nodes with certain characteristics, such as hardware capabilities or organizational policies. Taints are set by administrators on nodes to influence where pods can be scheduled.

31
Q

ConfigMap

A
  • Has configuration data as key value pairs
  • Often get set as env vars on the pod
  • Can be used to mount a file. Key is the mount path, value is the contents of the file
  • We used sops for secrets.yaml
32
Q

Volume

A

A volume is a directory backed by a storage medium

Volume is attached to pods and outlives the life of the pod

Volume type decides the type of directory to create

EmptyVolume
- Tied to the life of the pod
- It’s created when the pod is assigned to a node
- Useful for sharing files between containers within the same pod or for temporary data
- eg “/tmp”, “/var/run/shared_sockets”

HostPath
- Mounts a file or directory from the host node’s filesystem into your pod
- Often not used in prod given security risks (pod has access to underlying node), node coupling (makes it difficult to migrate pots and limits flexibility), etc

PersistentVolumeClaim
- Allows Pods to request storage resources without needing to know the details of the underlying storage infra
- A way to provision storage dynamically
- Types include: ReadWriteOnce, ReadOnlyMany, ReadWriteMany,
- With a PVC a user requests storage

CSI
- Allows 3rd parties to develop their own plugins to manage storage systems

33
Q

Ingress

A

Allows one to route and expose multiple services through a single IP address and port combination

Used to manage external access to services within a cluster.

Provides HTTP/S routing to services based on hostnames and URL paths

Rules: Define rules that specify how incoming requests should be routed. Example, route requests with a specific hostname (example.com) to a particular service

Handles TLS termination

NGINX is a common ingress controller, ISTIO too

Allows you to decouple rules from the application

34
Q

Secrets

A

Secrets object allow us to encode sensitive information before sharing it

Same idea of ConfigMaps

Keep in mind that by default etcd data is not encrypted and therefore the secret can be viewed

However, on EKS you can encrypted the underlying EBS volume

35
Q

Helm / Helmfile

A

Helm is a package manager for Kubernetes that helps you manage, version, and deploy applications using pre-configured packages called charts

Helmfile allows you to manage Helm releases (deployments) as code. Makes it easy to manage and version your Helm deployments

36
Q

ArgoCD

A

We integrated Helmfile into ArgoCD to automate the deployment of Helm releases using GitOps principles

37
Q

What are resource quotas in K8s?

A

A tool for managing the resource consumption in a namespace. It sets hard limits on the amount of compute resources, such as CPU and memory, that can be used by the objects within a namespace. This ensures that no single team or application consumes more than its fair share of resources, promoting efficient utilization and preventing resource contention.

Can specify limits on:
- CPU and Memory
- Storage: limits on persistent volume claims
- Object count: maximum number of objects like pods, services, and secrets

38
Q

What is a namespace in K8s?

A

A namespace is a logical partition which allows cluster resources to be divided between teams and/or applications running in the cluster

Key points about namespaces:
- Isolation: Namespaces isolate resources (eg pods, deployments) from each other
- Resource quotas
- Access Control: Using RBAC to restrict user access to certain namespaces
-