Agent SDK engineer interview questions

As asked

Write a TypeScript tool definition for creating a support ticket, then explain how you would make the schema safe for agent use.

Sample answer outline

The schema should make required fields explicit, restrict enums, and avoid accepting vague free-form blobs for important state. Tool descriptions should say when to call the tool and when not to call it, but the server must still validate permissions and business rules. A good answer includes idempotency keys or deduplication because agents may retry after timeouts. The tool should return a small structured result the agent can reason over, not an entire database record. Candidates often make tools too broad, which increases accidental side effects and makes eval failures hard to diagnose.

Reference implementation (typescript)

import { z } from "zod";

export const createSupportTicketTool = {
  name: "create_support_ticket",
  description:
    "Create one support ticket after the user has described a concrete problem. Do not call for general questions.",
  inputSchema: z.object({
    customerId: z.string().min(1),
    idempotencyKey: z.string().uuid(),
    severity: z.enum(["low", "normal", "high", "urgent"]),
    subject: z.string().min(5).max(120),
    description: z.string().min(20).max(4000),
    orderId: z.string().optional(),
  }),
  async execute(input: unknown) {
    const args = this.inputSchema.parse(input);
    return {
      ticketId: await createTicket(args),
      status: "created" as const,
    };
  },
};

Expect these follow-ups

• How do you version this tool without breaking old agents?
• Where does human approval fit?
• What should the tool return on validation failure?

As asked

You are asked to red team a tool-using support agent that reads web pages, calls internal APIs, and drafts customer replies. How would you test it for indirect prompt injection?

Sample answer outline

Start by mapping the trust boundaries: user instructions, retrieved web content, tool outputs, system prompts, and privileged internal data. Build payloads where hostile content appears in external pages or documents, then check whether the agent treats that content as instructions rather than data. A strong answer separates exfiltration, unauthorised tool use, data corruption, and policy bypass as distinct failure classes. Good candidates discuss both manual probing and an eval harness that measures attack success rate across model and prompt versions. The common trap is testing only direct jailbreaks in chat and missing the real agentic path through retrieved content.

Expect these follow-ups

• What signals prove the model followed the web page rather than the user task?
• How would you turn one successful exploit into a regression test?
• Which mitigations belong in prompting and which belong in the tool layer?

As asked

You need an LLM to extract invoice fields into JSON for downstream automation. How do you make the output reliable enough for production?

Sample answer outline

Define a strict schema first, including nullable fields, enums, currency handling, and confidence or evidence fields. Use model-native structured output or function calling where available, then validate every response with a real parser before accepting it. Add deterministic post-processing only for well-defined normalisation, not for fixing arbitrary model text. The pipeline should route low-confidence, invalid, or high-value invoices to human review and should store the original document and model version for audit. Strong candidates discuss evaluation by field-level precision and recall rather than a vague accuracy score.

Expect these follow-ups

• How do you handle a field that is visible in the invoice but ambiguous?
• What changes if the invoice may be handwritten?
• How do you version the schema without breaking downstream consumers?

As asked

A browser-use agent keeps searching, clicking, and returning to the same page without completing the task. How do you debug and fix the orchestration?

Sample answer outline

Start with a trace of observations, model messages, tool calls, screenshots or DOM summaries, and stop reasons for each step. Look for missing state, ambiguous success criteria, poor page abstraction, tool errors hidden as normal observations, or prompts that reward continuing rather than deciding. Fixes include explicit task state, step budgets, progress checks, better tool return values, and a final-answer condition that is tested in evals. The agent should fail closed with a useful reason rather than loop until a timeout. Candidates often tune the prompt before proving which observation caused the bad decision.

Expect these follow-ups

• What fields belong in an agent trace?
• How do you distinguish recoverable exploration from a loop?
• What eval would catch this before release?

As asked

You are adding computer-use actions to an agent SDK so agents can operate web apps. What sandboxing, permissions, and audit controls do you require?

Sample answer outline

Run agents in isolated browser or desktop sessions with scoped credentials, network controls, and no ambient access to the user's local files unless explicitly granted. Separate observation from action and require confirmation for sensitive actions such as purchases, messages, deletes, or permission changes. Store an audit trail of observations, actions, screenshots where allowed, and user approvals. The SDK should expose policy hooks so product teams can block actions by domain, selector, data class, or user role. Strong answers treat model behaviour as untrusted automation inside a controlled runtime.

Expect these follow-ups

• What actions should always require confirmation?
• How do you redact secrets from screenshots and traces?
• How would you test that policies cannot be bypassed through prompt injection?

As asked

What makes a sample app credible enough that developers copy from it without carrying bad patterns into production?

Sample answer outline

A credible sample app is small, runnable, and honest about what is production-grade and what is simplified. It should use current SDK APIs, secure secret handling, typed configuration, meaningful error handling, and tests around the integration path. The candidate should avoid building a toy that hides authentication, pagination, retries, or deployment concerns if those are central to real adoption. Strong answers include maintenance: version pinning, dependency updates, CI, and ownership when the API changes. Interviewers look for judgement about where completeness helps learning and where it becomes distracting.

Expect these follow-ups

• What is the first thing you would remove from an overbuilt sample app?
• How do you stop copied sample code from leaking secrets?
• How do you keep examples current across SDK versions?