Security engineer role-specific interview questions

As asked

Your production database password has leaked in a log file. Walk me through rotating it with zero downtime across a fleet of 200 application pods.

Sample answer outline

Two passwords on the database account, or a dual-credential pattern (Vault dynamic credentials or AWS RDS IAM auth). Sequence: add the new credential, push it to the secret store, trigger a rolling restart so pods pick it up via the projected secret or a sidecar, verify all pods are on the new credential, then revoke the old one. Audit logs to confirm no connections used the old credential before revoking. Treat the leak itself separately: rotate, then investigate scope.

Reference implementation (bash)

# Postgres supports a window where two passwords are valid (via pgbouncer auth_query
# or by issuing both old and new for the same role). For RDS IAM auth, no static
# password is needed at all.

# 1. Stage new credential in the secret manager
aws secretsmanager update-secret \
  --secret-id prod/api/db \
  --secret-string '{"username":"api","password":"NEW_PASSWORD"}'

# 2. Trigger a rolling restart so pods pick up the new secret
kubectl rollout restart deployment/api -n prod
kubectl rollout status deployment/api -n prod --timeout=10m

# 3. Confirm no active sessions are using the old credential
psql -c "select usename, count(*) from pg_stat_activity group by 1;"

# 4. Revoke the old credential on the database
psql -c "alter user api with password 'NEW_PASSWORD';"

Expect these follow-ups

• What if your database does not support two simultaneous passwords?
• How do you prove the leak did not lead to data exfiltration?
• How would you automate this so the next rotation is one click?

As asked

Your product is adding a feature where users can upload arbitrary files to share with other users. Threat model it.

Sample answer outline

Use STRIDE or a similar framework. Spoofing: who can upload, is auth required, is the user-id on the upload trusted. Tampering: file type lying about MIME (sniff the content, do not trust the header), filename traversal, content-disposition tricks. Repudiation: audit log of uploads. Information disclosure: are files private by default, signed URLs with expiry. Denial of service: max file size, rate limits, virus scanning. Elevation of privilege: stored XSS via uploaded HTML, malware delivery, hosting attacker-controlled JS. Concrete mitigations: serve user content from a separate origin, force download for unknown types, scan async, signed URLs.

Expect these follow-ups

• How do you serve user-uploaded images without enabling stored XSS?
• What is your virus scanning policy and what do you do on a positive hit?
• How would you handle the same feature for a regulated industry (healthcare, finance)?

As asked

You have 30 services running in one AWS account. Design an IAM strategy that gives each service least privilege and keeps the policies maintainable.

Sample answer outline

One IAM role per service, assumed via the workload identity (IRSA for EKS, instance profile for EC2, task role for ECS). Policies generated from infrastructure code, not hand-edited. Use IAM Access Analyzer to flag overly permissive policies and unused permissions. Avoid wildcard resource ARNs. Separate the prod and non-prod accounts to remove the worst blast-radius failure modes. Service control policies at the org level for guardrails like 'no public S3'. Rotate credentials never used (CIS benchmarks). Treat IAM as code with review.

Reference implementation (json)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadOwnConfigOnly",
      "Effect": "Allow",
      "Action": ["s3:GetObject"],
      "Resource": "arn:aws:s3:::prod-config/api/*"
    },
    {
      "Sid": "PublishToOwnTopic",
      "Effect": "Allow",
      "Action": ["sns:Publish"],
      "Resource": "arn:aws:sns:eu-west-2:123456789012:api-events"
    },
    {
      "Sid": "DenyAllOtherS3",
      "Effect": "Deny",
      "NotAction": ["s3:GetObject"],
      "Resource": "*"
    }
  ]
}

Expect these follow-ups

• How do you handle a service that needs cross-account access?
• What is your strategy for human access (engineers) to production?
• How would you scope a policy down from a wildcard that someone shipped 3 years ago?

As asked

An alert fires: a user account logged in from a new country, then immediately created a new API key. Walk me through your investigation.

Sample answer outline

First, contain: kill the active session and rotate the API key while investigating. Then verify: contact the user through an out-of-band channel to confirm if it was them. Look at the login: IP, device fingerprint, prior history for the user. Look at the broader pattern: are other users from the same IP, is there a credential stuffing burst, does the IP appear in threat feeds. Check what the API key was used for in the window between creation and detection. Document the timeline. If confirmed compromise: full account recovery flow, audit of data the attacker could have read, notify the user and (depending on data) regulators.

Expect these follow-ups

• How would you have detected this earlier?
• What is your policy for password resets after a confirmed compromise?
• What changes if this is an internal employee account?

As asked

Design and implement a rate-limiting strategy on a login endpoint that mitigates credential stuffing without locking out legitimate users.

Sample answer outline

Multiple keys, not just IP. Rate-limit per username (slows stuffing against a single account), per IP (slows distributed attempts from a single source), and globally on failure rate (signals an attack in progress). Use exponential backoff per username after consecutive failures. Add a CAPTCHA after N failures, do not lock out (lockout is itself a denial-of-service vector). Watchlist credential dumps with HaveIBeenPwned-style hash prefix checks and force a password reset for known-compromised passwords. Discuss residential proxy networks that defeat per-IP rate limits and why per-username matters more.

Reference implementation (typescript)

import { Redis } from "ioredis";

export async function checkLoginAttempt(
  redis: Redis,
  username: string,
  ip: string,
): Promise<{ allowed: boolean; backoffMs: number }> {
  const userKey = `login:user:${username}`;
  const ipKey = `login:ip:${ip}`;
  const [userFails, ipFails] = await Promise.all([
    redis.incr(userKey),
    redis.incr(ipKey),
  ]);
  await Promise.all([redis.expire(userKey, 900), redis.expire(ipKey, 900)]);
  if (userFails > 10 || ipFails > 50) {
    const backoffMs = Math.min(60_000, 2 ** Math.min(userFails, 10) * 100);
    return { allowed: false, backoffMs };
  }
  return { allowed: true, backoffMs: 0 };
}

Expect these follow-ups

• What if attackers come from a botnet with 100k unique IPs?
• When would you reach for device fingerprinting?
• What is the false positive rate of locking based on geo?

As asked

An engineer commits an AWS access key to a public GitHub repo. Walk me through what you do in the next hour.

Sample answer outline

Treat the key as compromised. Rotate immediately, do not wait. Revoke the old key, generate a new one, push it via the secret manager so dependent services pick it up. Check CloudTrail for any usage of the key in the window between commit and rotation, especially from unfamiliar IPs. Remove the key from history (git filter-repo or BFG) but understand that GitHub may have already cached it and bots scrape new commits within seconds, so rotation is what matters, not rewriting history. Talk to the engineer without blame, and add pre-commit hooks (gitleaks, trufflehog) so it does not happen again.

Expect these follow-ups

• Why is rewriting history less important than rotating?
• How would you scan all current repos for past leaks?
• What is your policy for repeat offenders?