Golang engineer system design interview questions

As asked

You are designing an orders topic with 50,000 events per second. How do you choose the partition key, and what can go wrong?

Sample answer outline

The partition key controls ordering, load distribution and consumer parallelism, so it should match the ordering requirement first. If all events for an order must be processed in sequence, orderId is a natural key; if customer-level ordering matters, customerId may be required but can create hot partitions. The number of partitions should allow enough consumer parallelism without creating unnecessary broker overhead. The answer should discuss skew, key cardinality, compaction needs and future repartitioning pain. Candidates often say random keys for balance while missing that this destroys per-entity ordering.

Expect these follow-ups

• How would you detect a hot partition in production?
• What happens to ordering when you increase partition count?
• When would you split one logical stream across multiple topics?

As asked

You are running 10 instances of a Go API service behind a load balancer. Design a rate limiter that enforces per-user limits across all instances. What storage backend would you use, how do you handle backend unavailability, and how do you minimize added latency?

Sample answer outline

A strong answer uses Redis with atomic Lua scripts or the INCR+EXPIRE pattern for sliding window or token bucket logic, ensuring atomicity. Each request makes one Redis call; under Redis unavailability, the service can fail open (allow) or fail closed (reject) with a circuit breaker. Latency is minimized by using a local in-process token bucket for the majority of traffic and only syncing with Redis periodically (approximate rate limiting) to reduce per-request latency. The candidate should discuss tradeoffs between precision and latency.

Expect these follow-ups

• How does this design change if you need hard guarantees rather than approximate limits?
• How would you implement this without Redis, using consistent hashing to route users to the same instance?

As asked

Design a system in Go that accepts work items via an HTTP API, persists them so they survive restarts, processes them with configurable concurrency, and supports retries with exponential backoff. What storage would you use, how do you prevent duplicate processing, and how do you monitor queue health?

Sample answer outline

A strong answer uses a database like Postgres for durability (advisory locks or SELECT FOR UPDATE SKIP LOCKED for dequeue to prevent duplicates), stores job state transitions (pending, running, failed, done), and implements exponential backoff by updating a next_attempt_at timestamp on failure. The processor uses a worker pool of goroutines. Monitoring tracks queue depth, processing latency, error rate, and dead-letter queue size. Alternatives like Redis sorted sets or a dedicated system like Asynq are worth mentioning.

Expect these follow-ups

• How does SELECT FOR UPDATE SKIP LOCKED prevent duplicate processing across multiple instances?
• At what queue depth would you switch from Postgres to a dedicated message broker?

As asked

You are building a Go microservice that calls three downstream gRPC services. Design the client-side architecture for connection management, load balancing, retries, circuit breaking, and distributed tracing. How do you handle a downstream that goes slow without taking down your service?

Sample answer outline

A strong answer covers using grpc.ClientConn with a DNS or service-discovery resolver for client-side load balancing (round-robin or least-loaded), per-call context deadlines to avoid waiting forever on a slow downstream, retries with exponential backoff on UNAVAILABLE errors, and a circuit breaker (like gobreaker) to stop calling a failing service. OpenTelemetry gRPC interceptors propagate trace context. The candidate should mention hedged requests as a technique for tail latency reduction.

Expect these follow-ups

• How does gRPC's built-in keepalive interact with cloud load balancers that close idle TCP connections?
• What is the difference between a service mesh handling retries versus doing it in Go application code?

As asked

Your Go API reads product catalog data that changes infrequently but is read millions of times per day. Design a caching strategy with an in-process layer, a Redis layer, and a Postgres backend. How do you handle cache invalidation, cold starts, and thundering herds?

Sample answer outline

A strong answer describes a three-tier approach: a local sync.Map or ristretto cache with a short TTL for the hottest items, Redis as the shared cache with a longer TTL, and Postgres as the source of truth. Cache invalidation on write uses a write-through or write-behind pattern and a Redis pub/sub to invalidate local caches across instances. Thundering herd on a cold cache is mitigated by singleflight. Cold starts after a deploy use cache warming by preloading from Postgres.

Expect these follow-ups

• How do you ensure cache and database consistency when a write partially fails?
• What is the cache-aside pattern and how does it differ from write-through?

As asked

You need to build a lightweight API gateway in Go that routes incoming HTTP requests to downstream microservices, validates JWT tokens, enforces rate limits, and logs request traces. How would you structure the code, and what would you use for the reverse proxy?

Sample answer outline

A strong answer uses httputil.ReverseProxy for proxying with custom Director functions per route. Middleware is composed with a chain pattern or a library like chi. JWT validation uses the jwt-go or golang-jwt library with RS256 or ES256 and key rotation via JWKS endpoints. Rate limiting per user uses the token bucket with Redis as described. Request tracing uses an OpenTelemetry middleware that propagates W3C trace context headers to downstream services.

Expect these follow-ups

• How do you handle circuit breaking when a downstream service is returning 500s?
• What is the latency overhead of a proxy hop, and how do you minimize it?

As asked

You are building a CLI tool in Go that DevOps engineers use to deploy services, query logs, and manage secrets. How do you structure the project, handle configuration from files and environment variables, implement subcommands, and make the tool testable?

Sample answer outline

A strong answer uses cobra for subcommands and viper for configuration layering (flag overrides env overrides file). The command handlers are thin and delegate to service structs that accept interfaces, making them testable without real network calls. Config is loaded early and validated; missing required values fail fast with a descriptive error. The candidate should mention golden file tests for CLI output, how to handle signals for graceful cancellation, and binary size considerations with Go build tags.

Expect these follow-ups

• How do you distribute a Go CLI binary across Linux, macOS, and Windows?
• How would you implement shell tab completion for subcommands and flags?

As asked

Design a Go service that maintains thousands of WebSocket connections and fans out events to all connected clients that are subscribed to a given topic. How do you structure the hub goroutine, prevent slow clients from blocking fast ones, and scale beyond one instance?

Sample answer outline

A strong answer uses a central hub goroutine that owns a map of topic to set of clients, receiving on register/unregister/broadcast channels to avoid concurrent map access. Each client has a buffered send channel; if the buffer is full the client is considered slow and dropped. For multi-instance scale, a pub/sub backend (Redis, Kafka, NATS) fans out events to all hub instances so each instance only needs to deliver to its locally connected clients. The candidate should discuss backpressure and graceful disconnect.

Expect these follow-ups

• How do you detect a dead WebSocket connection that has not sent a ping in 30 seconds?
• How does sticky session load balancing interact with this design?

As asked

You are building an order management service using event sourcing. Describe how you would implement the event store in Go with Postgres, how you reconstruct aggregate state from events, how you handle optimistic concurrency, and how you emit domain events to Kafka for downstream consumers.

Sample answer outline

A strong answer describes an events table with aggregate_id, version, event_type, and payload columns. Optimistic concurrency uses INSERT with a version check or a UNIQUE constraint on (aggregate_id, version). State is reconstructed by loading all events for an aggregate and folding them with a reduce function. After the transaction commits, a transactional outbox table or CDC (Debezium) publishes events to Kafka to avoid the dual-write problem. The candidate should mention snapshot optimization for long event streams.

Expect these follow-ups

• What is the dual-write problem and how does the outbox pattern solve it?
• How do you test that an aggregate's event stream correctly rebuilds its state?

As asked

Design a backend service in Go that provides autocomplete suggestions for a product search box. The service must return top 5 results in under 50ms p99 for millions of queries per day. Describe your data structure, storage, caching, and how you keep suggestions fresh as the catalog changes.

Sample answer outline

A strong answer uses a trie or sorted set in Redis (ZADD with score as popularity) for fast prefix lookup. The application layer reads from Redis and applies in-process caching for the hottest prefixes. The catalog is indexed into Redis asynchronously on change events via Kafka. The candidate should discuss sharding the trie by first letter if the dataset is too large for a single Redis node, and mention that Redis modules like Redisearch provide prefix search natively.

Expect these follow-ups

• How do you personalize suggestions per user without blowing up your cache hit rate?
• How would the design change if you needed to support fuzzy matching?