Platform engineer technical fundamentals interview questions

As asked

A pod tries to resolve the hostname 'myservice'. Walk through the full DNS resolution path: what is in /etc/resolv.conf, how does the search domain list affect which queries go to CoreDNS, and what is the ndots setting and why does it cause extra DNS lookups?

Sample answer outline

The pod's /etc/resolv.conf is injected by kubelet and contains the CoreDNS ClusterIP as the nameserver and search domains: [namespace].svc.cluster.local, svc.cluster.local, cluster.local, plus any host search domains. With ndots:5 (the default), a hostname with fewer than 5 dots is first tried against each search domain before being tried as an absolute name. So 'myservice' generates 3-4 DNS queries before resolving. For external hostnames like 'example.com' (1 dot, less than 5), the same behavior applies, creating extra latency. Mitigation: use FQDN (trailing dot), lower ndots, or use autopath in CoreDNS.

Expect these follow-ups

• What is CoreDNS autopath and how does it reduce the number of DNS queries for service resolution?
• How do you diagnose DNS resolution failures in a pod? Which tools and commands do you use?

As asked

Explain how Kubernetes container resource limits are enforced by the Linux kernel. What is the difference between cgroups v1 and v2, which subsystem enforces CPU limits, and what happens at the kernel level when a container exceeds its memory limit?

Sample answer outline

Kubernetes sets cgroup limits for each container: the cpu.shares (requests) and cpu.cfs_quota_us (limits) in the CPU subsystem, and memory.limit_in_bytes in the memory subsystem. When a container exceeds its memory limit, the kernel OOM killer terminates the process, which Kubernetes surfaces as an OOMKilled restart. cgroups v2 unifies the hierarchy and adds memory.swap.max for swap control. Candidates should note that CPU limits cause throttling, not killing, which is why CPU throttling is a silent performance issue.

Expect these follow-ups

• What is CPU throttling in the context of cgroups, and how do you measure whether a pod is being throttled?
• Why do some platform engineers argue for removing CPU limits entirely for stateless services?

As asked

Explain the OCI image specification: what is in an image manifest, how are layers stored and deduplicated in a registry, and what happens when you run docker pull in terms of network requests and filesystem operations?

Sample answer outline

An OCI manifest references a config blob (JSON with environment variables, entrypoint, etc.) and a list of layer blobs by digest (SHA256). Layers are content-addressed tarballs stored in the registry as blobs. docker pull fetches the manifest, checks local cache for each layer digest, and downloads only missing layers. On disk, the container runtime (containerd) uses an overlay filesystem (overlayfs) to stack layers, with the top layer writable. Deduplication is implicit because two images sharing a base layer share the same digest.

Expect these follow-ups

• What is a multi-platform image manifest list (OCI image index), and how does docker pull know which variant to download?
• How does layer caching in Docker BuildKit differ from the registry-level layer deduplication?

As asked

Explain what happens at the TCP and HTTP level when a pod makes 1000 requests per second to another service through a Kubernetes Service. Why can connection reuse (HTTP keep-alive) dramatically reduce latency and CPU usage, and how does the kube-proxy load balancing interact with long-lived connections?

Sample answer outline

Without keep-alive, each request incurs a TCP handshake (1 RTT) plus TLS handshake (1-2 RTT), wasting latency and CPU on the server side for connection setup. With keep-alive, one TCP connection handles many requests. The problem: kube-proxy iptables DNAT balances per-connection, not per-request. So a client with one long-lived connection always hits the same backend pod, creating load imbalance. Solutions: use a client-side load balancing library, Istio sidecar with its own connection pool and per-request load balancing, or gRPC client-side LB.

Expect these follow-ups

• What is the TIME_WAIT state in TCP and why can it cause port exhaustion in a high-throughput microservice?
• How does HTTP/2 multiplexing change the relationship between connections and request-level load balancing?

As asked

Explain how OIDC-based authentication works for kubectl access to a Kubernetes cluster. What does the API server verify in the JWT token, what is the client-go credentials plugin flow, and what breaks if the OIDC provider certificate expires?

Sample answer outline

The API server is configured with --oidc-issuer-url and --oidc-client-id. kubectl uses a credential plugin (like kubelogin or oidc-login) that handles the OIDC flow (device code or browser redirect), returns an ID token, and the API server validates the token's signature against the OIDC provider's JWKS endpoint, checks the iss and aud claims, and extracts the sub and groups claims to map to RBAC subjects. If the OIDC provider certificate expires, the API server cannot fetch the JWKS to validate tokens and all OIDC-authenticated users are locked out (service accounts use their own tokens and are unaffected).

Expect these follow-ups

• What is the difference between an ID token and an access token in OIDC, and which one does Kubernetes use?
• How do you configure kubectl to automatically refresh an expired OIDC token without requiring the user to re-authenticate?

As asked

Explain how kube-proxy in iptables mode implements a ClusterIP Service with three backend pods. What iptables chains are involved, how is load balancing implemented, and what is the performance implication as the number of Services scales past 10,000?

Sample answer outline

kube-proxy creates KUBE-SERVICES chain entries that match the ClusterIP:port and jump to a KUBE-SVC-xxx chain. The SVC chain uses statistic module (random probability) rules to distribute traffic across KUBE-SEP-xxx chains, one per endpoint, each containing a DNAT rule replacing the destination with the pod IP. For 10,000 Services with 3 pods each, there are 30,000+ iptables rules that must be traversed linearly per packet, causing O(n) latency in the kernel. IPVS mode uses a kernel hash table for O(1) lookup. eBPF-based solutions like Cilium bypass iptables entirely.

Expect these follow-ups

• What is the specific iptables rule that implements the 33% / 50% / 100% probability load balancing across three endpoints?
• At what scale does the iptables mode become a measurable bottleneck, and how do you detect it?

As asked

Define immutable infrastructure and explain the operational principle behind it. How does containerization and GitOps implement the principle in practice, and what classes of bugs and audit problems does it eliminate compared to mutable server configurations?

Sample answer outline

Immutable infrastructure means that once a server or container is deployed, it is never modified in place: to change configuration or code, you build a new artifact and replace the old instance. In containers this means no SSH-and-apt-get; instead, rebuild the image and redeploy. In GitOps, the Git repo is the source of truth and the cluster state is always derived from it. This eliminates configuration drift, snowflake servers, and ad-hoc manual changes that cannot be audited or reproduced, which are the root cause of 'works on my machine' and 'works in staging but not prod' problems.

Expect these follow-ups

• What is configuration drift and how do you detect it in a Kubernetes cluster that has been running for two years?
• Are there workloads where immutable infrastructure is impractical? How do you handle stateful systems?

As asked

Explain how Istio implements mutual TLS between two sidecars. What is the role of Citadel (Istiod's CA), how are certificates distributed to sidecars, how does the Envoy proxy intercept traffic transparently, and what does PEER_AUTHENTICATION PeerAuthentication.STRICT mode enforce?

Sample answer outline

Istiod acts as a CA and issues X.509 certificates to each Envoy sidecar via xDS SDS (Secret Discovery Service), with the certificate encoding the pod's SPIFFE identity (spiffe://cluster.local/ns/namespace/sa/service-account). The sidecar intercepts all inbound and outbound TCP traffic via iptables rules injected during pod initialization (init container), performing TLS termination/origination transparently to the application. PeerAuthentication STRICT mode tells Envoy to require mTLS for all inbound connections, rejecting plaintext. Certificate rotation happens automatically every 24 hours without pod restart.

Expect these follow-ups

• What is SPIFFE and how does it provide workload identity independent of network location?
• How does Istio's AuthorizationPolicy use the SPIFFE identity in the mTLS certificate for L7 access control?

As asked

The entire Kubernetes ecosystem is built on the control loop pattern. Explain the observe-diff-act loop, why idempotency is critical to it, and give two examples of how a poorly implemented controller breaks this loop in subtle ways.

Sample answer outline

The control loop observes current state, computes the diff against desired state, and takes action to close the gap. Idempotency means running the loop ten times produces the same result as running it once. Poor implementations: (1) a controller that appends to a list (like annotations) each time it reconciles, causing unbounded growth; (2) a controller that creates a resource without checking if it already exists, causing duplicate errors that are interpreted as transient failures and cause infinite requeuing. Strong candidates also mention that edge-triggering (act only on change events) is insufficient; level-triggering (periodically reconcile even without events) is required for reliability.

Expect these follow-ups

• What is the difference between edge-triggered and level-triggered reconciliation, and why do Kubernetes controllers use level-triggered?
• How does the ResourceVersion optimistic lock prevent two concurrent reconcile calls from conflicting?

As asked

Kubernetes 1.25 removed PodSecurityPolicy and replaced it with Pod Security Admission and the three built-in Pod Security Standards: Privileged, Baseline, and Restricted. What does each level enforce, what breaks most commonly when you move a namespace from Baseline to Restricted, and how do you audit existing workloads before enforcing a stricter level?

Sample answer outline

Privileged allows everything. Baseline blocks the most dangerous settings: privileged containers, host namespaces, and dangerous capabilities like NET_ADMIN. Restricted adds seccompProfile (must be RuntimeDefault or Localhost), drops all capabilities, requires runAsNonRoot, and disallows hostPath volumes. Common breakage when moving to Restricted: init containers that run as root, apps that need NET_BIND_SERVICE capability, and images that do not have a non-root USER. Auditing uses the warn or audit enforcement mode (rather than enforce) to log violations without blocking workloads, giving teams time to fix issues before hard enforcement.

Expect these follow-ups

• How do you set Pod Security Admission to enforce Restricted for production namespaces but only warn for dev namespaces?
• What legitimate use cases require Privileged level, and how do you handle them securely on a shared cluster?

As asked

Platform engineers spend a lot of time in Git. Explain how Git stores content as objects using SHA-1 hashes, why this makes Git naturally immutable and audit-friendly, and what happens when you run git commit under the hood in terms of object creation.

Sample answer outline

Git stores four object types in .git/objects: blob (file content), tree (directory), commit (snapshot with parent reference), and tag. Every object is stored by the SHA-1 of its content, making it content-addressable and immutable. git commit creates a tree object for the current working tree, a commit object referencing the tree and the parent commit, and updates the branch ref to point to the new commit SHA. This immutability is what makes GitOps reliable: the commit SHA in the Argo CD Application is an immutable reference to a specific state.

Expect these follow-ups

• What is a Git shallow clone and when does it cause problems in a CI pipeline?
• How does git rebase change commit SHAs, and why does this matter for GitOps deployments tracking a branch?