Agent SDK engineer system design interview questions

As asked

You need an LLM to extract invoice fields into JSON for downstream automation. How do you make the output reliable enough for production?

Sample answer outline

Define a strict schema first, including nullable fields, enums, currency handling, and confidence or evidence fields. Use model-native structured output or function calling where available, then validate every response with a real parser before accepting it. Add deterministic post-processing only for well-defined normalisation, not for fixing arbitrary model text. The pipeline should route low-confidence, invalid, or high-value invoices to human review and should store the original document and model version for audit. Strong candidates discuss evaluation by field-level precision and recall rather than a vague accuracy score.

Expect these follow-ups

• How do you handle a field that is visible in the invoice but ambiguous?
• What changes if the invoice may be handwritten?
• How do you version the schema without breaking downstream consumers?

As asked

A browser-use agent keeps searching, clicking, and returning to the same page without completing the task. How do you debug and fix the orchestration?

Sample answer outline

Start with a trace of observations, model messages, tool calls, screenshots or DOM summaries, and stop reasons for each step. Look for missing state, ambiguous success criteria, poor page abstraction, tool errors hidden as normal observations, or prompts that reward continuing rather than deciding. Fixes include explicit task state, step budgets, progress checks, better tool return values, and a final-answer condition that is tested in evals. The agent should fail closed with a useful reason rather than loop until a timeout. Candidates often tune the prompt before proving which observation caused the bad decision.

Expect these follow-ups

• What fields belong in an agent trace?
• How do you distinguish recoverable exploration from a loop?
• What eval would catch this before release?

As asked

A founder asks you to 'add team invites and billing seats' using a coding agent. How do you turn that prompt into a plan the agent can execute safely?

Sample answer outline

A strong answer decomposes the request into product rules, data model changes, API contracts, UI states, and migration or billing provider concerns before asking an agent to edit code. The candidate should identify risk areas: authorisation, duplicate invites, race conditions around seat limits, webhook idempotency, and auditability. Good agent instructions include file scope, acceptance criteria, test commands, and explicit non-goals. They should iterate in small diffs and verify each step rather than requesting a sweeping feature in one prompt. Interviewers look for product judgement and engineering control, not prompt bravado.

Expect these follow-ups

• What would you ask the founder before writing code?
• Which part would you implement manually rather than delegate to an agent?
• How do you check that billing webhooks are idempotent?