Engineering manager system design interview questions

As asked

A highly anticipated Netflix original releases at midnight UTC. You expect 10 to 20 times normal play traffic in the first 30 minutes. Walk me through how the platform team prepares for this, what failure modes are most likely, and what your runbook looks like for the on-call engineer.

Sample answer outline

A strong answer covers: pre-seeding Open Connect appliances with the encoded assets the day before, load testing the API gateway and recommendation service under 20x traffic simulations, enabling feature flags to shed non-critical work (recommendations, social features) and return cached or default responses if services degrade, pre-scaling autoscaling groups based on expected request curves, and a runbook that defines the on-call escalation path, the synthetic transaction monitors that alert within 30 seconds of a play failure, and the rollback plan if a deploy is found to be the cause.

Expect these follow-ups

• The play request volume is normal but users are reporting the title is not appearing on their homepage. What do you investigate first?
• How do you communicate status to a global engineering team during an incident at 2am when half the team is asleep?

As asked

Oracle Cloud Infrastructure Object Storage stores trillions of objects for enterprise customers. Walk me through how you would design a distributed object storage service that must handle 99.999999999% durability, strong read-after-write consistency for the same region, and low-latency access for large objects up to 50 GB.

Sample answer outline

A strong answer covers the data plane (shard assignment, consistent hashing, erasure coding vs. replication for durability), the metadata plane (a distributed key-value store mapping object key to shard locations), multi-part upload for large objects, presigned URLs for client-direct upload, and how you achieve strong consistency within a region while accepting eventual consistency across regions. The candidate should discuss failure domains, scrubbing jobs for bit-rot, and compaction.

Expect these follow-ups

• How do you handle the case where a client writes an object and immediately reads it before all replicas have confirmed?
• Walk me through how you would migrate a petabyte of objects with zero downtime.

As asked

Oracle API Gateway handles over one million requests per second across thousands of customer API tenants. Design a distributed rate limiting system that enforces per-tenant request quotas, can be updated dynamically, adds less than 2 milliseconds of latency to each request, and remains accurate to within a few percent even when the service is scaled to hundreds of gateway nodes.

Sample answer outline

A strong answer covers the token bucket or sliding window algorithm, the choice between a centralized store (Redis cluster) versus a gossip-based local approximation (each node tracks a local counter and periodically syncs), the tradeoff between accuracy and latency (centralized is accurate but adds one network hop; local approximation is fast but may allow short bursts over quota), and how to handle quota updates propagating to all nodes within a bounded window. The candidate should also cover what to do when the rate limit store is unavailable (fail open vs. fail closed).

Expect these follow-ups

• How do you prevent a thundering herd when all nodes refresh their quota at the same time?
• How does your design change if some tenants have per-second limits and others have per-minute limits?

As asked

Oracle Fusion Applications is a multi-tenant SaaS platform serving thousands of enterprise customers from shared infrastructure. Walk me through the architecture you would design to ensure that one tenant's data, compute, and network activity never affects another tenant's experience or security posture.

Sample answer outline

A strong answer covers schema-per-tenant versus row-level security in a shared schema (with the tradeoffs of each), connection pooling that prevents cross-tenant context leakage, network-level isolation using VCNs or security groups, compute isolation via dedicated pods or VMs for large tenants, audit logging that is tenant-scoped, and how you ensure a noisy-neighbor tenant cannot exhaust shared caches or I/O bandwidth. The candidate should also discuss how you test isolation guarantees.

Expect these follow-ups

• How do you design the data model so a new column added for one tenant does not require a schema migration for all tenants?
• How do you handle a zero-day vulnerability that requires patching the shared application layer without notifying each tenant individually?

As asked

You own a compute cluster running on OCI that serves bursty analytical workloads. Average utilization is 20% but peaks hit 90% for 2-hour windows daily. Design an autoscaling strategy that keeps p99 latency under 1 second during peaks while minimizing the monthly OCI compute bill.

Sample answer outline

A strong answer covers predictive scaling (scheduled rules for the known daily peak) combined with reactive scaling (metric-based rules on CPU or custom queue depth), the warmup time for new instances and how to account for it in scale-out decisions, spot instances for non-latency-sensitive batch work within the cluster, and how you validate that the SLA is met before declaring scale-out complete. The candidate should mention OCI-specific constructs like Instance Pools and Autoscaling Configurations.

Expect these follow-ups

• How do you handle a sudden unexpected spike that is 3x the historical peak?
• What metrics besides CPU would you monitor to trigger scaling in an analytical query cluster?

As asked

SpaceX needs to ship flight software updates frequently without sacrificing safety. Design the CI/CD pipeline for flight software, including what gates must pass before code can run on an actual vehicle.

Sample answer outline

A strong answer covers: static analysis (MISRA C++, custom linters), unit tests with coverage thresholds, software-in-the-loop (SIL) simulation against a software model of the vehicle dynamics, hardware-in-the-loop (HIL) testing against actual flight computer hardware to catch timing and integration issues, formal verification for critical state machines, a staged gate review with named sign-offs, and a controlled rollout that starts with test vehicles and ground tests before operational flights. The candidate should be clear that SIL runs against a simulated plant model while HIL uses real flight computers talking to a simulated or physical environment, and that both gates must pass. They should also address rollback procedures and binary provenance so you know exactly what code flew.

Expect these follow-ups

• How do you manage a situation where the simulation passes but hardware reveals a difference?
• What does your rollback procedure look like when a software update causes an anomaly 60 seconds into flight?

As asked

Imagine Airbnb's booking, search, payments, and reviews are all in a single Rails monolith. You have been asked to lead the decomposition into services over 18 months. How do you approach it, and what do you do first?

Sample answer outline

Strong answers follow the strangler fig pattern rather than a big-bang rewrite, starting with the boundary that offers the highest autonomy benefit (typically payments because of its separate compliance and scaling requirements). They discuss how to route traffic at the edge, how to handle cross-service data consistency (event-driven with eventual consistency, or synchronous with circuit breakers), how to manage shared databases during the transition, and how to avoid the distributed monolith anti-pattern. They should talk about what makes a good service boundary (team ownership, data ownership, independent scaling).

Expect these follow-ups

• How do you handle a transaction that spans two services that previously shared a database?
• How do you measure whether the decomposition is actually making things better or worse?

As asked

OCI's Virtual Cloud Network gives tenants isolated Layer 3 networking inside a shared physical fabric. How would you design the software-defined networking layer that enforces tenant isolation, routes packets between subnets, and supports security lists and network security groups, all without a centralized chokepoint?

Sample answer outline

A strong answer describes the overlay network model (VXLAN or similar encapsulation), distributed control plane that programs each hypervisor's virtual switch, the policy enforcement points at the NIC or vNIC level, and how security rules are enforced without routing all traffic through a central firewall. The candidate should discuss how route tables are propagated, how Internet Gateways and NAT Gateways work at the data plane, and how you test isolation correctness.

Expect these follow-ups

• How do you detect and block a tenant who is attempting to probe the physical network?
• What changes when a tenant needs sub-millisecond latency between two VMs on different physical hosts?