Agent SDK engineer system design interview questions

As asked

Design a multi-agent system where a user types a research question and gets back a comprehensive written report. The system should parallelize research across multiple sources and then synthesize the results. Walk me through the agent topology, data flow, and failure handling.

Sample answer outline

Strong answers describe a planner agent that decomposes the question into sub-queries, a set of parallel researcher agents each calling different search or database tools, a merging step that aggregates chunks and deduplicates, a writer agent that drafts the report, and an editor agent that fact-checks citations. Candidates should address fan-out rate limiting, partial failure handling, and how the final context is kept under the window limit.

Expect these follow-ups

• How do you make sure the writer agent does not hallucinate facts that were not in any researcher's output?
• How would you scale this to 1000 concurrent report requests?

As asked

You need an LLM to extract invoice fields into JSON for downstream automation. How do you make the output reliable enough for production?

Sample answer outline

Define a strict schema first, including nullable fields, enums, currency handling, and confidence or evidence fields. Use model-native structured output or function calling where available, then validate every response with a real parser before accepting it. Add deterministic post-processing only for well-defined normalisation, not for fixing arbitrary model text. The pipeline should route low-confidence, invalid, or high-value invoices to human review and should store the original document and model version for audit. Strong candidates discuss evaluation by field-level precision and recall rather than a vague accuracy score.

Expect these follow-ups

• How do you handle a field that is visible in the invoice but ambiguous?
• What changes if the invoice may be handwritten?
• How do you version the schema without breaking downstream consumers?

As asked

A browser-use agent keeps searching, clicking, and returning to the same page without completing the task. How do you debug and fix the orchestration?

Sample answer outline

Start with a trace of observations, model messages, tool calls, screenshots or DOM summaries, and stop reasons for each step. Look for missing state, ambiguous success criteria, poor page abstraction, tool errors hidden as normal observations, or prompts that reward continuing rather than deciding. Fixes include explicit task state, step budgets, progress checks, better tool return values, and a final-answer condition that is tested in evals. The agent should fail closed with a useful reason rather than loop until a timeout. Candidates often tune the prompt before proving which observation caused the bad decision.

Expect these follow-ups

• What fields belong in an agent trace?
• How do you distinguish recoverable exploration from a loop?
• What eval would catch this before release?

As asked

A founder asks you to 'add team invites and billing seats' using a coding agent. How do you turn that prompt into a plan the agent can execute safely?

Sample answer outline

A strong answer decomposes the request into product rules, data model changes, API contracts, UI states, and migration or billing provider concerns before asking an agent to edit code. The candidate should identify risk areas: authorisation, duplicate invites, race conditions around seat limits, webhook idempotency, and auditability. Good agent instructions include file scope, acceptance criteria, test commands, and explicit non-goals. They should iterate in small diffs and verify each step rather than requesting a sweeping feature in one prompt. Interviewers look for product judgement and engineering control, not prompt bravado.

Expect these follow-ups

• What would you ask the founder before writing code?
• Which part would you implement manually rather than delegate to an agent?
• How do you check that billing webhooks are idempotent?

As asked

Design a memory service that allows agents across multiple sessions to store, retrieve, and forget user-specific facts. It must support semantic search, time-based decay, and a per-user storage quota.

Sample answer outline

Strong answers describe a write path that embeds new memories and stores them with metadata (user_id, timestamp, access_count, importance_score), a semantic retrieval path using vector similarity filtered by user_id, a decay function that lowers importance over time without deleting (soft forgetting), hard deletion when quota is exceeded using least-important-first eviction, and encryption at rest per user. Candidates should discuss the embedding model choice and indexing strategy.

Expect these follow-ups

• How do you handle a user who requests deletion of all their memories under a GDPR right-to-erasure request?
• How do you prevent one user's memories from leaking to another user's agent session?

As asked

Design an observability platform that captures, stores, and makes searchable every step of every agent run: prompts, model responses, tool calls, errors, and costs. It should support replay and diff across runs.

Sample answer outline

Strong answers describe a structured event log (Kafka or a similar stream) with one event per agent step, a write path that stores raw events in object storage (S3) and indexes metadata in a columnar store (ClickHouse or BigQuery) for query, a UI that reconstructs a run timeline from events, a cost computation layer that maps token counts to dollar amounts per model, and a diff view that shows what changed in prompts or tool calls between two run versions.

Expect these follow-ups

• How do you handle the storage cost of storing the full prompt for every step at 10M runs per day?
• How do you replay a run from the stored events to reproduce a bug?

As asked

Design a system that runs a suite of golden-set agent tasks after every code or prompt change and flags regressions before they reach production. Cover test case management, scoring, CI integration, and cost controls.

Sample answer outline

Strong answers describe a golden set stored in a database with input, expected_output, and acceptable_tool_calls fields, a runner that parallelizes task execution using the candidate agent version, an LLM-as-judge scorer for open-ended tasks plus exact-match for structured outputs, a CI step that computes a weighted regression score and fails the build if it exceeds a threshold, and a cost gate that estimates the eval run cost and skips large models on draft PRs.

Expect these follow-ups

• How do you handle flaky tests where the same golden task passes sometimes and fails others?
• How do you prevent the golden set from becoming stale as the agent's capabilities grow?

As asked

Design the infrastructure that runs user-provided code (generated by an agent) safely at scale. Cover container lifecycle, network isolation, file system access, output streaming, and cleanup.

Sample answer outline

Strong answers describe a pool of pre-warmed containers to reduce cold start, each container started with a per-session network namespace that blocks all external traffic, a read-only base image with a writable overlay for the session scratch directory, a sidecar that streams stdout over gRPC back to the orchestrator, a watchdog that kills the container after a configurable timeout, and a cleanup job that removes containers and volumes after the session ends.

Expect these follow-ups

• How do you handle a container that consumes all available memory on the host?
• How do you let the agent's generated code read a file the user uploaded without mounting the entire upload bucket?

As asked

Design a workflow engine where an agent can pause at designated checkpoints and wait for a human to review and approve before continuing. The human may be offline for hours, so the system must be durable.

Sample answer outline

Strong answers describe a state machine per run with a paused_for_review state, serializing the full agent context to durable storage when pausing, sending a notification (email, Slack, webhook) with a one-time approval link, verifying the approval token on resume, restoring the context and continuing the run, and handling expiration of approval windows with a configurable escalation path.

Expect these follow-ups

• How do you handle a scenario where the action the human approved is no longer safe to execute because external state has changed in the meantime?
• How do you audit who approved which actions and when?

As asked

Design a system that tracks LLM API spend per user, per agent run, and per model in real time, enforces per-user budgets, and produces monthly billing reports broken down by usage type.

Sample answer outline

Strong answers describe intercepting every LLM API response to extract token counts and model id, writing a usage event to a stream for async aggregation, maintaining a running total per user in Redis for real-time budget checks, flushing events to a data warehouse (BigQuery) for monthly reporting, and a billing aggregation job that joins usage with model pricing tables to produce invoice line items.

Expect these follow-ups

• How do you handle a race condition where two concurrent requests both read the budget as within limit but together would exceed it?
• How do you deal with provider pricing changes that affect historical runs?

As asked

Design a system where third-party developers can publish tools that agents can discover and call. Cover authentication, schema validation, billing, abuse prevention, and how agents discover relevant tools at runtime.

Sample answer outline

Strong answers describe an OAuth-based developer onboarding flow, a tool registry with versioned JSON Schema manifests and semantic tags for discovery, a call proxy that injects developer credentials from secrets storage and measures usage for billing, rate limits per tool to prevent abuse, a sandboxed verification environment for new submissions, and a semantic tool search API that agents call to find relevant tools given a task description.

Expect these follow-ups

• How do you handle a tool that is suddenly unavailable but many agents depend on it?
• How do you prevent a malicious tool publisher from injecting prompt injection payloads in their tool's response?

As asked

An agent's run spans multiple microservices: an LLM gateway, a tool executor, and a memory service. Design how agent state is consistently shared so that any service can read the current run state without stale data.

Sample answer outline

Strong answers describe a centralized run state store (Redis or Postgres) as the single source of truth, each service reading and writing via an SDK client with optimistic locking using a version counter, an event bus (Kafka) for broadcasting state transitions so services can react without polling, and a reconciliation job that detects runs whose state has not been updated within a timeout.

Expect these follow-ups

• How do you handle a case where the LLM gateway crashes mid-write and leaves the state partially updated?
• How do you roll back a run to a previous state after detecting a bug in the tool executor?

As asked

Design a multi-tenant platform where different companies each run their own agent deployments on shared infrastructure. Cover tenant isolation, resource quotas, data segregation, and billing.

Sample answer outline

Strong answers describe tenant namespacing at every layer (database schemas or separate Postgres databases per tenant, separate vector collections, separate Redis key prefixes), resource quotas enforced at the API gateway and LLM proxy layers, audit logs tied to tenant id, billing meters that aggregate token usage per tenant per model, and soft vs. hard limits with notification before hard enforcement.

Expect these follow-ups

• How do you prevent a noisy tenant from degrading performance for others?
• How do you handle a tenant that requests a dedicated cluster rather than shared infrastructure?