Kubernetes DevOps Flashcards

1
Q

How do you automate Kubernetes deployment?

A

Development flow

  • Code Repo
  • Dockerize (jenkins, codebuild, gitlab)
  • Container Image Repo
  • Deploy [Helm, KubeCtl] ((jenkins, codebuild, gitlab)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

How do you secure Kubernetes app?

A

When it comes to security in the Kubernetes application, there are two aspect of it

Application Security
DevSecOps = Security of the container DevOps Lifecycle

Kubernetes Security
- Application Security
- Pod, namespace, Node
- RBAC, IRSA

DevSecOps = Security of the container DevOps lifecycle

Authorization (IAM Role)
- Scan repository (SysDig Falco, ECR image scanner)
- Scan running container (Twist lock, Black duck)

Security Compliance
- FedRAMP, HIPAA, SOC, etc. (check with the service complaint with regulations)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

How do you cost / performance optimize Kubernetes app?

A

Kubernetes Cost

Control Plane cost (Fixed, not much room for improvement)

Worker Node number and types

  • Pod resource specification
  • Unused CPU / Memory allocation

Detect CPU / Memory waste - Utilize Metrics Server

  • CloudWatch Container Insights
  • Kubecost (Dollar amount for resource)
  • CloudHealth
  • Kubernetes Resource Report
  • Cloudability (dubizzle use this)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Tell me about a challenge you faced in k8s?

A

There are many challenges is running k8s

Run lots of ops components
- Metric server, Prometheus, Thanos, CoreDNS, Harbor for docker cache, Log aggregation, Ingress, etc.

Upgrade Kubernetes cluster very three months
- It grunt task to upgrade kubernetes cluster very three months to upkeep with the upsteam

More challenging to get the Total cost of the application
- In VM based solution like Elastic Beanstalk, EC2, LB, Storage can be determined per app, in Kubernetes, there are pods, which we can get cost but there is shared resource like ALB, Shared Ops component, which makes estimating TCO tricky

K8s Upgrade story
- Updating the Kubernetes version
- Self managed - node Challange Drain the node Respect pod disruption budget, Keep the application up and running while node are updated

Solution
Use managed node group AWS manage all the heavy lifting
One Click update

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

How do you scale Kubernetes?

A
  • Horizontal Pod Autoscaler (HPA)
  • Cluster Autoscaler
  • Cluster Over-provisioning (Real World App)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

How do you expose a Kubernetes microservice to consumers?

A

Go over services
- Nodeport
- Load Balancer
- ClusterIP

Ingress is what is used in actual application

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Name the component of the control plane?

A
  • Kube-api-server
  • etcd
  • kube-scheduler
  • kube-controller-manager
  • cloud-controller-manager
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Does Kubelet manage containers which aren’t created by Kubernetes?

A

No, kubelet doesn’t manage container which aren’t created by kubernetes

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is a running the container in the privileges mode mean?

A

This means that if you are root in a container you have the privileges of root on the host system. Is only meant for special cases such as running Docker in Docker and should be avoided.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is the difference between replication controllers and replica sets?

A

The only difference between replication controllers and replica sets is the selectors. Replication controllers don’t have selectors in their spec and also note that replication controllers are obsolete now in the latest version of Kubernetes.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Is there any other way to update configmap for deployment without pod restarts?

A

well you need to have some way of triggering the reload. ether do a check every minute or have a reload endpoint for an api or project the configmap as a volume, could use notify to be aware of the change.

Depends on how the configmap is consumed by the container.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is the ingress, is it something that runs as a pod or on a pod?

A

An ingress is an object that holds a set of rules for an ingress controller, which is essentially a reverse proxy and is used to (in the case of nginx-ingress for example) render a configuration file. It allows access to your Kubernetes services from outside the Kubernetes cluster. It holds a set of rules. An Ingress Controller is a controller. Typically deployed as a Kubernetes Deployment. That deployment runs a reverse proxy, the ingress part, and a reconciler, the controller part. the reconciler configures the reverse proxy according to the rules in the ingress object. Ingress controllers watch the k8s api and update their config on changes. The rules help to pass to a controller that is listening for them. You can deploy a bunch of ingress rules, but nothing will happen unless you have a controller that can process them.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

If a pod exceeds its memory “limit” what signal is sent to the process?

A

SIGKILL as immediately terminates the container and spawns a new one with OOM error. The OS, if using a cgroup based containerisation (docker, rkt, etc), will do the OOM killing. Kubernetes simply sets the cgroup limits but is not ultimately responsible for killing the processes.SIGTERM is sent to PID 1 and k8s waits for (default of 30 seconds) terminationGracePeriodSeconds before sending the SIGKILL or you can change that time with terminationGracePeriodSeconds in the pod. As long as your container will eventually exit, it should be fine to have a long grace period. If you want a graceful restart it would have to do it inside the pod. If you don’t want it killed, then you shouldn’t set a memory limit on the pod and there’s not a way to disable it for the whole node. Also, when the liveness probe fails, the container will SIGTERM and SIGKILL after some grace period.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is psp (Pod security policy)

A

Pod Security Policies (or PSPs) are objects that control security-sensitive aspects of pod specification (like root privileges).

If a pod does not meet the conditions specified in the PSP, Kubernetes will not allow it to start, and Rancher will display an error message of Pod is forbidden: unable to validate….

https://k8s-examples.container-solutions.com/examples/PodSecurityPolicy/PodSecurityPolicy.html

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What difference between Daemonset and Side Car? Apart for it runs on each node, how characteristic are different? Scheduling, IP?

A

Deamonset = certain pod running on every nodeSidecar = container in the same pod

Deamonset = Own separate IP
Sidecar = Same IP on different port

Deamonset example = ingress controller, log shippers like Datadog, etc
Sidecar example = log collector like fluentd or fluentbit which exports the logs from disk, envoy proxy for network

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What are benefits of Managed K8s service

A

Managed Kubernetes service has following advance

  • Highly Available and Scalable Control Plane
  • Console and UI
  • One Click Version Upgrade
  • Cloud Provider Managed AMIs with Up to

Date Security Patching
- Easy installation of tools and add-ons
- Amazon VPC CNI - To take advantage of multiple VPC features
- Amazon EBS CSI - Use EBS as persistent volume

  • Integration with Other service
  • EKS + EMR
  • EKS + Step functions
17
Q

What feature do you want to add to Managed K8s service?

A

Details view of resources running in the cloud
- Add Metric server addon while provisioning EKS

18
Q

Which tools you would use to backup Kubernetes cluster etcd?

A

You can use open source tools like
- portworx
- velero
- kasten by veeam

19
Q

What is network policies? What is needed to run them?

A

We need to use network policy agents like

  • Fannel by CoreOS
  • Calico
  • Canal by Weaveworks
20
Q

What is different between Service Mesh and Network policies?

A
  • Network policies operate at levels Level 3 and 4
  • Service mesh works on Layer 7
  • Service mesh has an additional features like
  • Rate limiting
  • Throttling
  • Circuit breaker
  • Proxy to Proxy security
  • tracing
21
Q

What are the volume mounts you can have in the pod

A
  • configmap
  • emptyDir
  • projected
  • secret
  • downwardAPI
  • persistentVolumeClaim
22
Q

At which two level most of the kubernetes policies are define

A
  • Cluster level
  • Namespace level
23
Q

What is admission controller in kubernetes?

A

An admission controller is a piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorized.
In that list, there are two special controllers: MutatingAdmissionWebhook and ValidatingAdmissionWebhook. These execute the mutating and validating (respectively) admission control webhooks which are configured in the API.

24
Q

What is Mutating Admission controller?

A

It is a type of admission controller. It can mutate the incoming requests to get a list of HashiCorp vault secrets when the controller starts.
There are two special controllers: MutatingAdmissionWebhook and ValidatingAdmissionWebhook. These execute the mutating and validating (respectively) admission control webhooks which are configured in the API.

25
Q

What are security measures you can take to make you cluster secure. Not DevSecOps pipeline, but something you need to do harden the cluster?

A
  • Network Policy
  • Secure image-building practices
  • Audit logging
  • Avoid mounting service accounts
  • Permissions on demand in RBAC
  • Use a container that actually contains, e.g. katacontainers, kubevirt, gvisor, etc.
  • Use the admission plugin DenyEscalatingExec
26
Q

What is no service account mounting

A

When the pod doesn’t need to talk to the kubeapi server, you can disable the automounting of the account token

apiVersion: v1
kind: Pod
metadata:
name: my-pod
spec:
automountServiceAccountToken: false

27
Q

What are the container patterns in kubernetes?

A
  1. The single-container design pattern
  2. The sidecar design pattern
  3. The ambassador design pattern : If you’re using the ambassador pattern, it means you have a proxy for other parts of the system
  4. The adapter design pattern : application exporting metrics in XML, adaptor container reads it and push as Prometheus metrics
  5. The leader election design pattern : If you’re using the leader election pattern, it means you’re providing redundancy for consumers of containers that need to have highly available systems.
  6. The work queue design pattern : split tasks into smaller tasks and put them into the queue. Worker container takes it from queue and process it.
  7. The scatter/gather design pattern : MapReduce
28
Q

What is difference between namespace quota and limit ranges?

A

Namespace quota defines how much cumulative resource namespace can consume and limit ranges.

With resource quotas, cluster administrators can restrict resource consumption and creation on a namespace basis. Within a namespace, a Pod or Container can consume as much CPU and memory as defined by the namespace’s resource quota. There is a concern that one Pod or Container could monopolize all available resources. A LimitRange is a policy to constrain resource allocations (to Pods or Containers) in a namespace.

A limit range provides constraints that can:

Enforce minimum and maximum compute resources usage per Pod or Container in a namespace.
Enforce minimum and maximum storage requests per PersistentVolumeClaim in a namespace.
Enforce a ratio between request and limit for a resource in a namespace.
Set default request/limit for computing resources in a namespace and automatically inject them to Containers at runtime.

29
Q

What is the difference between helm 2 and helm 3?

A
  • Tiller component is removed. As RBAC was introduced in the 1.6 version, now tiller component is not needed
  • Helm 2 is used to store the release information in the config map, and helm 3 uses the secret to store release information
  • Helm 3 has a JSON schema chart validator with isn’t there is Helm 2
30
Q

You made manual change made to manifest using kubectl. You change the replica count. This pod was create using Helm install. When you issue the rollback, will Helm rollback manually edited pod?

A

No
Helm 2 uses a two-way merge, where when you install, it will compare the new release YAML with the OLD release YAML. It will not check the present state of release
In Helm 3, it uses a three-way merge, it will compare the new release YAML with the OLD release YAML and the current state of the cluster. Based on the difference, it will issue a patch.

31
Q

What are the tools for smart provision the node instead cluster auto scaler

A
  • krapenter
  • cast.ai

If you have c5.4xlarge node define and it just needs one more pod, it would add the full one node with one pod running
This tools which provision t2.medium or spot instance base on the pending workload of pods. Smart scheduling.

32
Q

What is a Kubernetes Operator, and why is it useful?

A

A Kubernetes Operator is a method of packaging, deploying, and managing a Kubernetes application. It extends the Kubernetes API to create, configure, and manage complex stateful applications on behalf of the user.

  • Automation of Operational Tasks
  • Custom Resource Definitions (CRDs)
  • Self-Healing and Autonomous Operation
  • Application-Specific Knowledge
33
Q

Explain the concept of liveness and readiness probes in Kubernetes

A

Explain the concept of liveness and readiness probes in Kubernetes:In Kubernetes, liveness and readiness probes are mechanisms used to improve the reliability and availability of applications running in Pods. Here’s a breakdown of each:

Liveness Probe: A liveness probe is used to determine if a container within a Pod is alive and healthy. Kubernetes periodically executes a specified command inside the container or sends an HTTP request to a designated endpoint. If the probe fails (i.e., the command returns a non-zero status code or the endpoint is unreachable), Kubernetes considers the container to be unhealthy and takes remedial action, such as restarting the container. Liveness probes are crucial for ensuring that applications are responsive and can recover from transient failures.

Readiness Probe: A readiness probe is used to determine if a container is ready to serve traffic. Kubernetes uses readiness probes to decide whether to include a Pod in the pool of endpoints that can receive traffic from Services. Similar to liveness probes, readiness probes can execute commands or send HTTP requests to check the container’s readiness. If the probe fails, Kubernetes removes the Pod from the pool of endpoints until it becomes ready again. Readiness probes are essential for ensuring that Pods are only included in load balancing once they are fully initialized and capable of handling requests.
By configuring liveness and readiness probes appropriately, Kubernetes can automatically detect and handle container failures, ensuring that applications remain available and responsive to incoming traffic.

34
Q

What is different between Gateway API, Ingress and Service Mesh

A

GateWay API is new specification which will replace Ingress spec and Service Mesh spec

The problem is Ingress Specs

  • It was too limiting
  • Many implementations like Istio, Kong work with Annotation and Labels
  • It doesn’t address extensibility like WAF, protection against
    • DoS using rate limiting
    • JWT authentication
    • JS and SQL Injection protection

The problem is Service Mesh Specs

  • Every implementations has it’s own extension, it’s own way
  • It doesn’t unify Service Mesh ability like
    - canary traffic distribution
    - mTLS
    - Observability metrics

Gateway API specs address this and is extensible

35
Q

What are the pro’s of using GateWay API

A
  • Standard API Specs
  • Most of the Industry is behind it
  • It address the Separation of Concern
    • What Kube Admin does (Define Gateway classes)
    • What SRE / DevOps does (Define Gateway definition using Gateway classes)
    • What Developers does (Define Routes for their application using Gateway create by SRE)