K8s Flashcards
Service Account Tokens
- ServiceAccount: A resource that provides an identity for processes running in pods
- Every Pod has a service account. Created automatically
- Allows apps to authenticate with the K8s API and access cluster resources
- Service accounts are associated with access tokens that the pods can use to authenticate themselves to the k8s API server, enabling them to perform actions like reading or modifying resources within the cluster
- When a pods starts, K8s mounts a Service Account token at a predefined location, eg /var/run/secrets/k8s.io/serviceaccount/token
- This token is signed by the Kubernetes Certificate Authority. This token is signed
- When decoded the ServiceAccount token contains things like namespace, pod name and uuid, service account name
- Once the K8s API receives ServiceAccount token from the pod, it validates existence of the service account and checks if the system has any RBAC rules attached that allow/deny the current action
OIDC
Used the OIDC authentication layer to validate and authenticate our external users managed on our corporate identity provider, JumpCloud
We used OIDC integration to access the K8s cluster
K8s uses OIDC validation mechanism to confirm it’s the correct user
Use the OIDC group and/or claims which get returned for looking up the RoleBinding
Roles, Bindings, Subjects
Role: Grant permissions within a specific namespace. Contains rules that represent a set of permissions. There are no DENY rules, always additive
ClusterRole: Same idea but for the whole cluster and granting permissions for non-namespaced resources (like nodes)
RoleBinding: Grants the permissions defined in a Role to a user or set of users. Applies only within a specific namespace. Has a list of Subjects
ClusterRoleBinding: Same idea. Use this to grant permissions across the whole cluster
Subjects: Users, Groups, or ServiceAccounts
- These Bindings get bound to subjects and they can be authenticated users returned from the OIDC integration or ServiceAccounts with a corresponding namespace
- Within the Role you would define the rules which can be accessed, eg get, list, create, delete (these are the verb rules) against a set of resources (eg pod, services)
- To change roles you need to remove the Binding, change the role, then rebind. Will get an error if you attempt to change a role when it’s already bound
Other authentication strategies
- Bootstrap tokens. Allow servers to authenticate for the first time
- Static tokens: Only use for simple stuff
- Authentication Proxy: K8s api server delegates the full authentication process to a proxy server
- Webhook tokens: Similar to Auth Proxy
- X.509 certs: Uses mTLS and extracts the username from the Common Name and the RBAC groups from the Organization fields. Each “user” has this cert and the k8s API verifies these certs using a certificate authority
Impersonation:
RBAC
- Adds security to a K8s cluster
- Role Based Access Controls
- RBAC defines roles, which are collections of permissions, and subjects which are service accounts or users. A subject is granted a role
- What attributes on an object get used during RBAC authorization?
User
Group
Resource
Name
^ These attributes are validated against policies
Verbs: The operations we want to do with the resources
Node Authentication and Authorization in EKS
- Nodes in an EKS cluster authenticate themselves using AWS IAM.
- Each node is associated with an IAM role that has specific permission
- Node authorization is managed to K8s, specifically through the kubelet component running on each node. Can also use RBAC
K8s Components
Two primary components:
Control Plane (master nodes)
- API Server
- Control Manager
- Etcd
- Scheduler
Data Plane (worker nodes)
- Kube Proxy
- Kubelet
- Container runtime
K8s Component - API Server
- All interactions with the cluster are made through the API server
- The API server is responsible for validating all API requests and ensuring they are authorized
- Adheres to a client-server model
- Clients can be software developers, devops engineers, nodes, k8s scheduler, etc
Authentication: Verify the identity of the entity making the request, whether it’s a user, pod, or machine
Authorization: Checks if the requesting entity possesses adequate permissions for the operation
Only service which interacts directly with etc
K8s Component - Control Manager
Runs controllers to regulate the state of the cluster
Ensure desired state matches current state
Creates service accounts, new endpoints, manages load balancers, etc
Manage storage volumes
It encompasses several controllers, each focusing on a specific aspect of the cluster’s behavior. These controllers continuously monitor the cluster’s state and work towards reconciling any deviations from the desired state.
Examples: like the ReplicaSet controller ensuring the desired number of pod replicas are running, the Deployment controller managing the lifecycle of deployments, and the Node controller handling node-related tasks like detecting and responding to node failures.
K8s Component - Etcd
Distributed KV store which only has cluster state
Always insert, never delete. Though runs compaction
No application data
Build on raft consensus algo
K8s Component - Scheduler
Watches for newly created Pods and assigns objects or pods to nodes
Knows about the quality and configuration of the nodes
It uses various policies to select the most suitable node for a pod based on factors such as resource requirements, node capacity, and pod affinity/anti-affinity
K8s Component - Kubelet
- An agent running on each worker node
- Communicates with control plane
- Receives pod definitions from master and interacts with container runtime. Connects to container runtime
- Monitors health of the pods
K8s Component - Kube Proxy
- The network agent
- Runs on each node
- Handles dynamic updates and maintenance of all networking rules on the node.
- Configures iptables
- Forwards networking requests to the pods
- Allows services to communicate with each other
- Managed by a DaemonSet
Worker node
Has container runtime. Examples: docker, containerd, rkt
Pods are scheduled on worker nodes
Has networking
Pod is smallest scheduling unit in K8s
Worker node components
- Kubelet
- Kubeproxy
Master node
Contains control plane components
Makes global decisions about the cluster
Manages cluster operations
Must be up all the time
If active master fails, then one of the replicas will take its place