Site reliability engineer technical fundamentals interview questions

As asked

Explain the CAP theorem in concrete terms. Give an example of a real system that chooses AP and one that chooses CP, and explain what each gives up.

Sample answer outline

The candidate should explain that during a network partition you can choose consistency (returning an error if the node cannot confirm the latest state) or availability (returning a potentially stale response). AP example: Cassandra and DynamoDB (eventual consistency, always available). CP example: etcd and ZooKeeper (consistent, reject writes if quorum is lost). They should also mention that PACELC extends CAP to describe the latency versus consistency tradeoff in the normal (no-partition) case.

Expect these follow-ups

• What does eventual consistency actually mean in practice for a user reading their own writes?
• How does the PACELC model extend CAP theorem?

As asked

Walk me through every network step that occurs when a user types https://example.com in a browser and the page loads, with an SRE focus on what can go wrong at each step.

Sample answer outline

Strong answers cover DNS resolution (recursive lookup, TTL, potential failures), TCP connection to the resolved IP (SYN timeout, firewall drops), TLS handshake (certificate validation, OCSP, cipher negotiation), HTTP/2 or HTTP/3 request, CDN edge versus origin routing, load balancer selection, app server processing, and database queries. At each step they should name a failure mode SRE actually cares about: DNS misconfiguration, expired certificate, origin overload, slow DB query.

Expect these follow-ups

• How does HSTS (HTTP Strict Transport Security) change the first request after a user has visited before?
• What does the browser do when it receives a 503 versus a connection timeout?

As asked

Walk me through the TCP three-way handshake. What happens when the server's SYN-ACK is lost? How does this relate to SYN flood attacks and the SYN cookie mitigation?

Sample answer outline

The candidate should describe SYN, SYN-ACK, ACK. If SYN-ACK is lost, the client retransmits SYN with exponential backoff. On the server, the half-open connection sits in the backlog consuming memory. SYN flood exploits this by sending millions of SYNs. SYN cookies mitigate by encoding state in the sequence number so the server only allocates memory when the final ACK arrives. They should know that net.ipv4.tcp_syncookies is the Linux sysctl for this.

Expect these follow-ups

• What is the TCP listen backlog and how do you tune it?
• What happens to a TCP connection if no data is sent for a long time?

As asked

A server shows a load average of 32 on a 16-core machine. What does that number represent, what could be causing it, and does a high load average necessarily mean the machine is struggling?

Sample answer outline

Load average in Linux counts runnable processes (in TASK_RUNNING state) plus uninterruptible sleep processes (in TASK_UNINTERRUPTIBLE, typically waiting for I/O). So a load of 32 on 16 cores means an average of 2 processes queued per core, but this could be CPU-bound or I/O-bound. They should know that `iostat` or `sar` distinguishes the two cases. A high load from I/O wait does not mean the CPUs are busy. They should also explain the three exponential moving averages (1 min, 5 min, 15 min) and what a rising versus falling trend tells you.

Expect these follow-ups

• How does a load average spike that lasts only 30 seconds show up in the 5-minute average?
• What is the difference between CPU utilization and load average?

As asked

Explain the Raft consensus algorithm at a level that would let you understand why etcd behaves the way it does during a network partition or leader election.

Sample answer outline

The candidate should explain the three roles (leader, follower, candidate), how leader election works via election timeout and majority vote, how the leader replicates log entries to followers and commits once a majority acknowledges, and how split votes are resolved. Practically for etcd: during a partition, a quorum-less minority node stops accepting writes (safety over availability). They should know that a 5-node etcd cluster can tolerate 2 node failures and that reducing to 2 nodes makes the cluster non-functional.

Expect these follow-ups

• What happens to etcd clients during a Raft leader election?
• Why does etcd use Raft rather than Paxos?

As asked

Name the Linux kernel features that Docker and Kubernetes use to isolate containers, and explain what each one isolates. Then explain why containers sharing a kernel is a security concern that VMs do not have.

Sample answer outline

The candidate should list: pid namespace (process isolation), net namespace (network isolation), mnt namespace (filesystem isolation), uts namespace (hostname), ipc namespace, and user namespace (UID mapping). Cgroups handle resource accounting and limits. The shared-kernel concern is that a kernel vulnerability allows container escape (privileged container breakout, cgroup misconfiguration) which compromises all other containers on the host. VMs have separate kernels so a hypervisor vulnerability is needed for escape, which has a much smaller attack surface.

Expect these follow-ups

• What is a privileged container and when should you ever run one?
• What does seccomp do in a container context and what default profile does Kubernetes apply?

As asked

From an SRE perspective, what is the difference between counting 4xx versus 5xx errors in your SLI? Give an example where a spike in 4xx errors should trigger an alert, and one where it should not.

Sample answer outline

5xx errors are server-side faults and almost always indicate a problem the SRE should act on. 4xx errors are typically client errors (bad request, not found, unauthorized) and reflect client behavior rather than service health. However, a sudden spike in 400s could indicate a bad deploy that broke the request schema, or a spike in 401s could indicate an authentication service failure or a mass token expiry bug. The SLI should typically count only 5xx as errors unless specific 4xx codes indicate server-side problems.

Expect these follow-ups

• How do you handle 429 Too Many Requests in your SLI: is it a client fault or a server fault?
• What does a 504 Gateway Timeout tell you that a 503 Service Unavailable does not?

As asked

Define idempotency and explain why it is critical for retryable operations in distributed systems. Give an example of an operation that is idempotent by nature and one that requires explicit engineering to make idempotent.

Sample answer outline

Idempotency means calling an operation multiple times produces the same result as calling it once. Critical because networks are unreliable and clients retry. GET is naturally idempotent; PUT (set value to X) is idempotent; POST (increment balance) is not. To make POST idempotent: assign a client-generated idempotency key on the request, server stores it and returns the cached response on duplicate. The candidate should mention how payment APIs use this pattern to prevent double-charging.

Expect these follow-ups

• How do you generate a good idempotency key on the client side?
• What happens if two requests with the same idempotency key arrive at the same time on two different servers?

As asked

From an SRE standpoint, what specific reliability problems does immutable infrastructure solve that mutable, in-place infrastructure creates? What new operational challenges does it introduce, and how does it change your incident response and rollback strategy?

Sample answer outline

Mutable infrastructure is patched in place: configuration drift, snowflake servers, and unknown state accumulate over time. Immutable infrastructure builds a new artifact and replaces the old one: containers or AMIs baked with all dependencies included. Reliability wins: no drift, rollback is just deploying the previous artifact, and the blast radius of a bad change is bounded to pods using the new image. Challenges: build pipelines become critical infrastructure, stateful services do not fit the immutable model, and image builds must be fast.

Expect these follow-ups

• How do you handle operating system security patches in an immutable infrastructure model?
• What is the difference between immutable infrastructure and GitOps?

As asked

Explain CAP. Then tell me what CAP does and does not tell you when you are choosing a database in 2026.

Sample answer outline

CAP says: under a network partition, a distributed system must choose between consistency (every read sees the latest write) and availability (every request gets a response). It does NOT say you pick two out of three in normal operation. The more useful framing is PACELC: in a partition, choose A or C; else, choose latency or consistency. Real systems are not binary either. DynamoDB offers per-request consistency levels. Spanner uses synchronised clocks to give strong consistency at higher latency. The interview answer that matters: which of consistency or availability does your specific application require, with reasons.

Expect these follow-ups

• What does PACELC add over CAP?
• How does Spanner's TrueTime change the CAP discussion?
• Give an example where eventual consistency is the right call.

As asked

Users in one region cannot resolve api.example.com, while other regions are fine. Walk me through your DNS debugging process.

Sample answer outline

Check resolution from affected and unaffected recursive resolvers, then query authoritative nameservers directly to separate resolver cache problems from zone problems. Inspect TTLs, recent changes, DNSSEC validation failures, delegation, glue records, and geo-DNS or latency-based routing policies. Compare A, AAAA, CNAME, and ALIAS behaviour because IPv6 or a broken CNAME target can look regional. Use logs from managed DNS where available and confirm whether the issue is resolver-specific, ISP-specific, or authoritative. Weak answers stop at flushing cache and miss delegation or DNSSEC failures.

Expect these follow-ups

• How can a DNSSEC issue affect only some resolvers?
• What TTL would you choose before a planned cutover?
• How do you prove the application is healthy once DNS is fixed?

As asked

Explain the difference between hashing and encryption, give an example of where SREs use each in production systems, and explain what happens when a system stores passwords with MD5.

Sample answer outline

Hashing is one-way (SHA-256, bcrypt) used for passwords, checksums, and fingerprints. Encryption is two-way, requires a key, and used for data at rest (AES-256) and in transit (TLS). SRE contexts: checksum stored artifacts after download, verify log integrity, encrypt secrets in Kubernetes using a KMS provider. MD5 for passwords is a critical vulnerability because MD5 is fast (GPUs can crack billions per second) and should be replaced with bcrypt or Argon2.

Expect these follow-ups

• What is the difference between symmetric and asymmetric encryption and when would you use each?
• How does TLS certificate pinning work and why is it dangerous to deploy in a large-scale system?