DevOps engineer technical fundamentals interview questions

As asked

Explain the TCP three-way handshake. What are the SYN, SYN-ACK, and ACK states, and what does it mean when you see a lot of connections in SYN_SENT or TIME_WAIT on a production server?

Sample answer outline

Strong answers explain the handshake sequence, then explain SYN_SENT as the client waiting for a SYN-ACK (many of these suggest the target is unreachable or dropping packets, pointing to a firewall or overloaded server). TIME_WAIT is the 2*MSL period after connection close to handle delayed packets; many TIME_WAIT connections on a server under high connection churn is normal but can exhaust ports. Solutions include SO_REUSEADDR, shorter TIME_WAIT via tcp_fin_timeout tuning, or connection pooling to reduce churn.

Expect these follow-ups

• What is the purpose of the TIME_WAIT state and what happens if you disable it with tcp_tw_reuse?
• How does connection pooling in a database driver reduce TIME_WAIT pressure on an application server?

As asked

Walk me through exactly what happens when a pod in Kubernetes tries to resolve api.example.com. Include the in-cluster DNS, the chain to the authoritative nameserver, and how TTL caching works.

Sample answer outline

Strong answers describe the pod checking /etc/resolv.conf for the CoreDNS service IP (typically 10.96.0.10), CoreDNS checking its cache and then forwarding to the upstream resolver (the node's /etc/resolv.conf or a configured forwarder), the recursive resolver walking from the root servers through .com to example.com's authoritative nameserver, and the TTL controlling how long CoreDNS and the client cache the result. Mention ndots:5 in Kubernetes causing search domain appending and how it affects external DNS lookups.

Expect these follow-ups

• What is the effect of ndots:5 in the Kubernetes pod's /etc/resolv.conf and how can it cause unexpected latency?
• How does CoreDNS forward DNS to an on-premise nameserver for internal company domains?

As asked

Containers are often described as 'just processes.' Explain what Linux cgroups and namespaces actually do and how they together create the isolation properties that make a Docker container feel like a separate machine.

Sample answer outline

Strong answers explain namespaces (pid, net, mnt, uts, ipc, user) as providing the illusion of isolation: a process in a PID namespace sees only its own process tree, the network namespace gives it its own network stack. Cgroups limit resource consumption: memory, CPU, block I/O. Together they create a process tree that cannot see outside its namespace and cannot exceed its resource budget. Unlike a VM, there is no separate kernel; the container shares the host kernel, which is why kernel vulnerabilities can be container escapes.

Expect these follow-ups

• Why is a container not as isolated as a VM when it comes to kernel exploits?
• What does the --privileged flag do when starting a Docker container and why is it dangerous?

As asked

What is immutable infrastructure and why is it important? How does it change your approach to debugging and patching compared to mutable servers?

Sample answer outline

Strong answers explain that immutable infrastructure means never modifying a running server or container in place; instead, you rebuild and replace. This eliminates configuration drift, makes deployments reproducible, and simplifies rollback to replacing the instance. Debugging shifts from SSHing into a live box to reproducing the issue in a local container or by examining logs and metrics. Patching means rebuilding the base image and redeploying. The tradeoff is that incident debugging requires good observability because you cannot inspect the crashed process interactively.

Expect these follow-ups

• How do you handle a critical security patch that needs to go out in 30 minutes if rebuilding images takes 20 minutes?
• What is the golden image pattern and how does it relate to immutable infrastructure?

As asked

When would you prefer git rebase over git merge in the context of a CI/CD pipeline, and what are the risks of rebasing a branch that has already been pushed?

Sample answer outline

Strong answers explain that rebase rewrites commit history producing a linear graph, making it easier to bisect for bugs and keeping the main branch history clean. Merge preserves the branch topology but produces merge commits. Rebasing a pushed branch rewrites SHA hashes and requires a force push, which breaks other engineers who have based work on those commits. In CI/CD, a squash-and-rebase merge strategy on PRs is common to keep main linear while allowing messy feature branch history during development.

Expect these follow-ups

• How do you use git bisect to find which commit introduced a bug, and why is a linear history important for bisect?
• What is a fast-forward merge and when does it happen?

As asked

Explain the difference between a pub/sub messaging pattern and a point-to-point queue. Give me a concrete DevOps use case where each is the right choice.

Sample answer outline

Strong answers define a queue as a one-producer-one-consumer model where each message is processed once (e.g. an SQS queue driving a Lambda for image resizing). Pub/sub has one producer and many subscribers each receiving a copy (e.g. an SNS topic that fans out to multiple SQS queues for logging, alerting, and auditing from a single deployment event). In a DevOps context, a deployment event could publish to a topic and multiple downstream systems (Slack notifier, audit logger, deployment tracker) consume independently.

Expect these follow-ups

• How does Kafka support both queue and pub/sub semantics using consumer groups?
• What is the difference between at-least-once and exactly-once delivery and which is easier to achieve?

As asked

A production service starts failing with 'too many open files' errors under high load. Explain what file descriptors are in Linux, what limits govern them, how you diagnose the problem, and what you change to fix it.

Sample answer outline

Strong answers explain that in Linux everything is a file: sockets, actual files, pipes, and device handles all consume file descriptors. Limits are per-process soft and hard limits (ulimit -n shows soft; ulimit -Hn shows hard) and system-wide fs.file-max. Diagnosis: lsof -p <pid> | wc -l shows current FD count, /proc/<pid>/limits shows limits, check for FD leaks by comparing counts over time. Fix: increase limits in /etc/security/limits.conf, systemd service LimitNOFILE, or Kubernetes container resources. Fix leaks in the application by ensuring connections and files are closed.

Expect these follow-ups

• How do you set file descriptor limits for a process running in a Kubernetes container?
• How do you detect a file descriptor leak in a running process without restarting it?

As asked

Checkout latency jumps from 250ms p95 to 2s p95 after a deploy. Walk me through how you use observability data to find the cause.

Sample answer outline

Start with metrics to localise the symptom: which service, route, region, dependency, and percentile changed. Use deployment markers and service dashboards to correlate the regression with a version or config change. Then inspect traces for slow requests to see whether time is spent in app code, database calls, network hops, or downstream services. Use structured logs for exact errors, feature flags, customer segments, or payload patterns, not as the primary latency graph. Good candidates avoid jumping straight to rollback, but they know when rollback is the right mitigation while diagnosis continues.

Expect these follow-ups

• What dashboard panels would you expect every service to have?
• How do you distinguish client-side latency from server-side latency?
• What do you do if traces show missing spans in the suspected service?

As asked

Explain HTTP/2 multiplexing and how it solves HTTP/1.1's head-of-line blocking problem. Why does HTTP/3 still have a head-of-line blocking problem at a different layer?

Sample answer outline

Strong answers explain that HTTP/1.1 pipelines requests on a single TCP connection but the response to request 1 must arrive before request 2 (application-layer HOL). HTTP/2 uses binary framing with stream IDs so responses can interleave without ordering constraints, solving application-layer HOL. However, all HTTP/2 streams share one TCP connection, so a lost TCP packet blocks all streams until retransmitted (transport-layer HOL). HTTP/3 uses QUIC over UDP which maintains independent stream delivery, solving transport-layer HOL too.

Expect these follow-ups

• How does HTTP/2 HPACK header compression work and why does it matter for APIs with many requests?
• When running an ingress controller, how do you enable HTTP/2 between the client and the controller versus between the controller and the backend?

As asked

Walk me through the TLS 1.3 handshake. What has changed from TLS 1.2 and why does it matter for latency in high-throughput services?

Sample answer outline

Strong answers explain TLS 1.3's 1-RTT handshake (vs TLS 1.2's 2-RTT): the client sends key_share in the first message so the server can compute the shared secret immediately and respond with the certificate and finished in one round trip. TLS 1.3 also supports 0-RTT resumption for reconnecting sessions, reducing latency further but with replay attack risk. Removed weak cipher suites and the certificate is now encrypted. For a high-throughput service, the latency difference matters for connection-heavy workloads like gRPC.

Expect these follow-ups

• What is the 0-RTT replay attack risk in TLS 1.3 and how do you mitigate it?
• How does session resumption with session tickets work and what are the key management implications?

As asked

Explain eventual consistency. Give me a specific example of how it can cause a visible bug in a deployed system and how you would detect and handle it operationally.

Sample answer outline

Strong answers define eventual consistency as a guarantee that, absent new updates, all replicas will converge to the same value eventually but there is no bound on when. A concrete example: before December 2020, S3 offered only eventual consistency for LIST operations after a PUT, so a downstream service listing a bucket might not see a newly uploaded object for a short window. AWS made S3 strongly consistent for all object operations including GET, PUT, LIST, and DELETE in December 2020, removing this issue for S3 specifically. A current relevant example is DynamoDB global tables or a multi-region Aurora read replica, where a write to the primary may not be visible on a replica for tens of milliseconds. Detection uses read-your-writes checks or checksums. Operational handling: add retry with backoff, avoid read-after-write patterns across replicas, or use a strongly consistent read option when correctness is critical.

Expect these follow-ups

• How did S3's consistency model change in December 2020 and which operations became strongly consistent?
• What is the CAP theorem and how does it relate to the consistency vs availability tradeoff in distributed systems?