Question 1

Explain what a Kafka consumer group rebalance is, what triggers one, and what the practical impact is on a production streaming pipeline. What strategies do you use to minimize disruption?

Accepted Answer

A rebalance redistributes partition ownership among consumers in a group. Triggers include a consumer joining or leaving, a heartbeat timeout, or a topic partition count changing. During the rebalance all consumers stop consuming (stop-the-world by default with eager rebalancing), which causes latency spikes and can cause duplicate processing if offsets were not committed. Strong answers mention cooperative incremental rebalancing introduced in Kafka 2.4 that only moves affected partitions, static group membership to avoid rebalances on rolling restarts, tuning session.timeout.ms and max.poll.interval.ms, and idempotent consumers to handle duplicates safely.

Question 2

dbt supports multiple incremental strategies: append, delete+insert, merge, and insert_overwrite. Walk me through when you choose each one and what can go wrong with the merge strategy on large tables in BigQuery.

Accepted Answer

Append is simplest and fastest but only correct when rows are immutable (event logs). Delete+insert drops and re-inserts the affected partition, which is atomic but requires a reliable partition column. Merge is the most flexible for slowly changing records but on BigQuery performs a full table scan on the target unless the predicate includes the partition column, which makes it expensive at scale. Insert_overwrite replaces entire partitions and is good for hour or day granularity pipelines where reprocessing a whole partition is acceptable. Strong answers mention the is_incremental() macro, the unique_key configuration, and testing incremental models with dbt test to catch late-arriving data issues.

Question 3

You need to track historical changes to a customer dimension table in Snowflake. Describe how you would implement SCD Type 2: the schema design, the merge logic, and how you handle large daily change volumes efficiently.

Accepted Answer

SCD Type 2 adds effective_from, effective_to, and is_current columns to the dimension. A new row is inserted for each changed record; the previous row's effective_to is set to the change date. The MERGE statement in Snowflake or BigQuery matches on the natural key, closes the current row and inserts a new one atomically. For large volumes, candidates should mention staging the incoming changes first, then applying a two-step approach: first UPDATE to close open rows, then INSERT new rows, which is often faster than a single MERGE with complex logic. Strong answers also address the surrogate key generation strategy (sequence vs. hash) and how fact tables should join to dimension snapshots using the date range.

Question 4

You are designing a Kafka topic for an order events stream that will be consumed by multiple downstream services. Walk me through how you decide on partition count, which field to use as the partition key, and how both decisions affect ordering guarantees and consumer throughput.

Accepted Answer

Partition count determines maximum parallelism for consumers. A good starting rule is to set it to the target peak consumer throughput divided by the throughput per partition. Partition count cannot easily be reduced later, so sizing up is safer. The partition key controls which partition a message lands on; messages with the same key always go to the same partition, which is the only ordering guarantee Kafka provides. For orders, keying by order_id preserves per-order ordering, but keying by customer_id may be better if downstream needs all events for a customer in order. Strong answers mention the hotspot risk: a viral customer with thousands of orders will starve one partition, so sometimes a two-level key or null key with consumer-side sorting is better.

Question 5

Walk me through how you would add data quality validation to a daily batch pipeline loading data into a warehouse. What tool or framework would you use, what categories of checks would you write, and how do you handle failures without breaking the pipeline entirely?

Accepted Answer

A strong answer covers at least three categories: schema checks (column presence, data types), completeness checks (null rates, row counts vs. expected ranges), and semantic checks (values within known domains, referential integrity, statistical distribution drift). Frameworks like Great Expectations, dbt tests, or Soda Core integrate into pipelines at load or transform time. Candidates should describe a severity model: critical failures (wrong schema, row count near zero) halt the pipeline and alert, while soft failures (elevated null rate) log a warning and continue but trigger a downstream quality flag. This prevents all-or-nothing fragility while still surfacing issues.

Question 6

Your daily ETL job extracts from an API, transforms data, and writes to a Snowflake table. On reruns (due to failures or backfills), you must not create duplicates. Walk me through the design choices that make this pipeline fully idempotent.

Accepted Answer

Idempotency means running the same job twice produces the same result as running it once. For a batch pipeline, this requires: using a deterministic extraction window (e.g., yesterday's date as a parameter, not a rolling lookback), writing to a staging table or stage first, then applying a MERGE or DELETE+INSERT that is keyed on a natural unique identifier for that day's data. Avoid appending to a table without deduplication logic. Strong candidates mention partitioned writes: if the target table is partitioned by date, replacing the entire partition atomically (Snowflake TRUNCATE+INSERT in a transaction, BigQuery partition replace) is the simplest idempotent pattern. They also mention that extraction idempotency (the API must return the same data for the same window each time) is equally important and not always guaranteed.

Question 7

Traditional ETL transformed data before loading it. Modern cloud warehouses push for ELT where raw data is loaded first and transformed inside the warehouse. Walk me through when you would choose each approach and how pricing models like Snowflake's credit system affect the decision.

Accepted Answer

ETL makes sense when: the source data is too large to load raw (reducing it pre-load is necessary), PII must be masked before it touches the warehouse for compliance, or the transformation logic depends on ML models running outside the warehouse. ELT is preferable when raw data has value on its own (auditing, reprocessing), when transformation logic changes frequently (easier to iterate in SQL or dbt than rebuilding ETL code), and when the warehouse is cost-effective for compute. For Snowflake, virtual warehouse credits are billed per second of compute used, so heavy SQL transformations are cost-predictable. Strong candidates note that GDPR and CCPA compliance may mandate that personal data is never stored in its raw form, which forces some ETL masking before load.

Question 8

Monday morning your daily pipeline fails with a column not found error. You trace it to a column that a source team dropped over the weekend without notifying anyone. Three downstream analytics dashboards are already showing no data. Walk me through your response.

Accepted Answer

First: assess impact (which dashboards, which tables, how long data is missing). Contact the source team to understand if the column is permanently dropped or temporarily missing. If permanently dropped, implement a schema-safe fallback: add a NULL coalesce for the column, update the pipeline to remove the dependency or map to a new column name, test, and redeploy. Backfill the missing hours. Notify downstream stakeholders. Long-term: implement schema validation in CI that tests the pipeline against a schema compatibility check before deploying, add a schema drift alert that fires when the source schema changes, and introduce a data contract with the source team that requires advance notice of breaking changes.

Question 9

The finance team reports $5.2M in last month's revenue, and the growth team reports $4.9M for the same period, both claiming to use the data warehouse. Your VP notices the discrepancy in a board meeting. What is your investigation process?

Accepted Answer

First, get the exact SQL or BI report definition from both teams. Differences usually come from: different filter criteria (refunds included or excluded, specific product lines), different date definitions (order date vs. payment date vs. recognition date), different aggregation granularity, or each team using a different source table or model. Systematically walk through each definition side by side. Once the discrepancy is found, document the canonical definition in a metrics layer (dbt metrics, Looker LookML, or a semantic layer) so all consumers use the same logic. Report the root cause and the fix to the VP with a clear explanation. Strong candidates mention implementing a certified metrics library that prevents this class of divergence.

Question 10

You need a streaming layer for event ingestion. Walk me through how you would decide between self-hosted Kafka, MSK, Kinesis, and Confluent Cloud.

Accepted Answer

Self-hosted Kafka gives full control and the lowest unit cost at scale, in return for the operational cost of running a clustered storage system. MSK takes the operational burden but you still own scaling and tuning. Confluent Cloud is fully managed including schema registry, connectors, and ksqlDB, at the highest unit cost. Kinesis is simpler and integrates well with the AWS ecosystem, but caps out on throughput per shard and has a shorter retention. Decision drivers: total throughput, retention needs, ecosystem fit (Kafka has more connectors), and how many engineers you have to operate it.

Question 11

A 30-person Series A SaaS company asks you to pick their data stack from scratch. Walk me through your choices and the tradeoffs.

Accepted Answer

Warehouse: Snowflake or BigQuery; both work, pick based on engineering team familiarity. Ingestion: Fivetran or Airbyte for SaaS sources; build custom only where there is no connector. Transformation: dbt Core (with dbt Cloud if you want managed scheduling and IDE). BI: Mode, Hex, or Looker depending on user persona. Reverse ETL: Hightouch or Census if marketing or ops needs warehouse data in their tools. Observability: Monte Carlo or open-source elementary. Total monthly cost in the low thousands; complexity is the team's, not the stack's. Avoid premature distributed compute (Spark, Trino) at this stage.

Question 12

Walk me through what happens internally during a Spark shuffle. What triggers one, how does data move between executors, and what are the most common performance problems you run into with large shuffles?

Accepted Answer

A strong answer covers the two phases: map side (writing shuffle files to local disk partitioned by reducer key) and reduce side (fetching those files across the network). Candidates should mention sort-based shuffle vs. hash shuffle, the shuffle service, and why wide transformations like groupByKey or join cause shuffles while narrow ones do not. Key problems include shuffle spill to disk when executor memory is insufficient, skewed partitions causing one task to take 10x longer than others, and network I/O bottlenecks. Strong candidates mention using broadcast joins to avoid shuffles entirely for small tables, repartitioning before joins, and salting keys to handle skew.

Question 13

Snowflake is not a traditional row-store database. Explain how micro-partitions work and walk me through how you decide whether to define a clustering key on a table, what columns to choose, and what the maintenance cost is.

Accepted Answer

Snowflake stores data in compressed micro-partitions (roughly 50 to 500 MB compressed) with metadata about min/max values per column per partition. Queries prune partitions at scan time using that metadata. A clustering key is only worth defining when natural insertion order diverges significantly from query filter patterns, such as a table inserted in user_id order but always queried by date. Candidates should mention that Automatic Clustering is a background service that re-clusters over time and incurs compute costs, so it should be reserved for very large tables with predictable, repetitive filter patterns. Choosing high-cardinality columns that appear in WHERE or JOIN conditions is the goal, but avoid columns with extreme cardinality that result in one row per micro-partition.

Question 14

You are using Debezium to capture changes from a Postgres source and writing events to Kafka. Explain the failure modes in this pipeline and how you ensure downstream consumers handle the at-least-once delivery guarantee correctly.

Accepted Answer

Debezium reads from the Postgres WAL using logical replication. If Debezium crashes after reading a WAL record but before committing the Kafka offset, it replays from the last committed position and produces duplicates. Downstream consumers must be idempotent, typically by using the event's LSN or transaction ID plus table primary key as a deduplication key. Strong answers mention that Kafka Streams and Flink both support exactly-once via transactions and that a MERGE or UPSERT in the sink (Snowflake, BigQuery, or Postgres) keyed on the source primary key is the most common approach. Candidates should also mention slot lag monitoring in Postgres to prevent WAL accumulation from filling disk.

Question 15

BigQuery is a columnar store that charges by bytes scanned. Explain how partition pruning reduces cost and query time, and describe the common patterns where pruning silently does not happen.

Accepted Answer

BigQuery supports date or timestamp partitioning, integer range partitioning, and ingestion-time partitioning. When a query filters on the partition column directly (e.g., WHERE event_date = '2024-01-01'), BigQuery reads only matching partitions. Pruning fails when: the filter wraps the column in a function (CAST, FORMAT_DATE, DATE_TRUNC used incorrectly), when a subquery or CTE is used and the optimizer cannot push the predicate, or when the table is not partitioned at all. Strong candidates also mention clustering as a complementary technique that sorts within partitions and further reduces bytes scanned, and note that INFORMATION_SCHEMA.PARTITIONS can be queried to inspect which partitions exist.

Question 16

Describe the components required for exactly-once semantics in Kafka, from producer to consumer. What are idempotent producers, transactional APIs, and how does a Kafka Streams application guarantee exactly-once processing?

Accepted Answer

Idempotent producers assign a sequence number to each batch; the broker deduplicates retries within a session. Transactions extend this to atomic writes across multiple partitions: the producer begins a transaction, writes to multiple topics, then commits atomically. On the consumer side, read_committed isolation level means consumers only see committed transactions. Kafka Streams uses a combination of the transactional producer and changelog topics backed by RocksDB to atomically process input, update state, and produce output. Strong answers note that exactly-once is only within Kafka itself, and that sinking to an external system (Snowflake, Postgres) requires an idempotent sink operation to extend the guarantee end to end.

Question 17

You have inherited a large Airflow deployment with hundreds of DAGs that is causing scheduler performance problems and frequent task failures. What design anti-patterns would you look for first, and how would you fix them?

Accepted Answer

Common anti-patterns include: top-level code in DAG files that runs on every scheduler heartbeat (e.g., database connections, API calls outside of tasks), which slows DAG parsing; overly large DAGs with thousands of tasks that tax the scheduler; not setting depends_on_past or wait_for_downstream correctly, causing tasks to run when upstream data is not ready; using LocalExecutor in a multi-worker setup that creates contention; and not setting task retries and timeouts, causing hung tasks to block slots. Fixes include lazy imports, splitting large DAGs, using task groups, enabling the Airflow 2.x scheduler with DAG file processing pools, and profiling parsing time with airflow dags parse.

Question 18

You need to serve both real-time dashboards (sub-minute latency) and historical batch reports from the same data. Compare Lambda and Kappa architectures for this requirement. Which would you recommend and what are the trade-offs?

Accepted Answer

Lambda maintains separate batch and speed layers with a serving layer that merges results. The batch layer (Spark on S3 or HDFS) provides correctness over large historical windows, the speed layer (Kafka plus Flink or Spark Streaming) provides recency, and the query layer unions them. The main problem is maintaining two codebases that must produce identical results. Kappa simplifies to a single streaming layer, replaying historical data through the same streaming job when reprocessing is needed. Kappa works well when the streaming framework can handle reprocessing at batch speeds (e.g., Flink with Kafka replay). Kappa is increasingly preferred with mature tools but requires enough Kafka retention to replay history, which is expensive. Strong candidates mention Lakehouse architectures (Delta Lake, Iceberg) as a modern alternative that can serve both latencies from one storage layer.

Question 19

You run EXPLAIN ANALYZE on a query joining three large tables in Postgres and see a Seq Scan on a 50-million-row table and a nested loop join with an estimated row count of 1 vs actual 120,000. Walk me through what you would investigate and how you would fix it.

Accepted Answer

The seq scan on a large table suggests a missing index or a predicate that prevents index use (e.g., a function wrapping the column). The large estimate vs. actual row count discrepancy (1 vs 120,000) indicates stale or inaccurate table statistics, which causes the planner to choose a nested loop instead of a hash join, turning an O(n) operation into O(n*m). Fixes: run ANALYZE or VACUUM ANALYZE on the table to refresh statistics, increase default_statistics_target for the column if the distribution is skewed, add an appropriate index, or use a query hint (pg_hint_plan) as a last resort. Strong candidates mention that nested loops are efficient for small inner sets but catastrophic here, and explain how to confirm the join type selected in the plan.

Question 20

Your Flink streaming job aggregates click events by 5-minute windows using event time. Some events arrive up to 3 minutes late due to mobile app buffering. Explain how watermarks work and how you configure the job to handle this correctly without dropping valid late events.

Accepted Answer

Flink uses watermarks to track event-time progress. A watermark with timestamp T tells the system that no event with event time earlier than T will arrive. A periodic watermark strategy emits watermarks every N milliseconds based on the maximum observed event timestamp minus a configured lag (e.g., 3 minutes). When the watermark passes the end of a window, that window is triggered and its result emitted. Events arriving after the watermark are late; they can be directed to a side output for separate handling rather than silently dropped. Allowed lateness on the window can hold the window state open for an additional duration, emitting an updated result each time a late event arrives. Strong answers distinguish between the watermark lag and the allowed lateness, and describe the memory cost of keeping window state open.

Question 21

Snowflake and BigQuery both store data in a columnar format with compression. Explain how columnar compression differs from row-based compression, what compression algorithms are typically used, and how this affects query performance and storage costs.

Accepted Answer

In row-based storage, each row stores all column values together, so compression must handle heterogeneous data types. Columnar storage stores each column independently, enabling type-specific compression: run-length encoding works well for low-cardinality string columns (e.g., status, country), dictionary encoding for repeated string values, delta encoding for monotonically increasing timestamps. These give far higher compression ratios than row-based gzip. For queries that read only a few columns, columnar storage skips all other columns entirely (projection pushdown), dramatically reducing I/O. Strong candidates mention that Snowflake uses proprietary ZSTD-based compression and that choosing the right data types (DATE vs. VARCHAR for dates) significantly affects compression ratio and query speed.

Question 22

You are responsible for data quality in a dbt project with 80 models feeding executive dashboards. Describe your full testing strategy: which built-in dbt tests you use, where you add custom tests, how you handle test failures in CI, and how you catch statistical anomalies that schema tests miss.

Accepted Answer

Built-in tests cover unique, not_null, accepted_values, and relationships. These should be applied to every primary key and every foreign key. Beyond that, custom generic tests using Jinja macros can check things like date range validity, percentage completeness, or that a sum of a fact column is within an expected range. For statistical anomalies, dbt-expectations (a package mirroring Great Expectations) or dbt-utils.recency checks are useful. In CI, failing tests should block a PR merge. For production, a separate monitoring run on a schedule (not just during deploys) catches data drift between deploys. Strong answers also mention elementary or re_data for automated anomaly detection on metric trends over time.

Question 23

Explain Spark's join strategies: broadcast join, sort-merge join, and shuffle hash join. When does the optimizer choose each one automatically, and when do you force a specific strategy and why?

Accepted Answer

Spark chooses a broadcast join when one side is smaller than spark.sql.autoBroadcastJoinThreshold (default 10 MB). The smaller table is sent to every executor, eliminating the shuffle entirely. Sort-merge join is used for large tables: both sides are sorted by join key, then merged in a single pass, requiring a shuffle but no full-table memory. Shuffle hash join builds a hash table from the smaller side in memory during the shuffle reduce phase; it is faster than sort-merge when the smaller side fits in memory per partition but is larger than the broadcast threshold. You force a broadcast join with the broadcast() hint when you know a table is small but the stats are stale, or when autoBroadcastJoinThreshold is disabled for safety. Strong answers note that broadcast joins fail with OOM if the table is actually large.

Question 24

Your organization has 200 dbt models, 50 Airflow DAGs, and multiple Kafka topics. When an executive dashboard shows wrong numbers, how do you trace the root cause? What tools and practices give you data lineage and observability?

Accepted Answer

Data lineage maps the path of data from source to dashboard. dbt exposes a manifest.json with a full DAG of model dependencies, which tools like Atlan, Datahub, or OpenLineage can ingest. For runtime observability, emit OpenLineage events from Airflow operators and Spark jobs so a lineage backend can trace which source record contributed to which output. For Kafka pipelines, schema registry enforces contracts between producers and consumers. When debugging a wrong number, the first step is to check the dbt model DAG for the metric, then check Airflow task run history for failures or late runs, then verify row counts and checksums at each transform stage. Strong candidates mention setting up data observability tools like Monte Carlo or Metaplane for automated anomaly detection.

Question 25

You need to add a new required field to a Kafka topic's Avro schema. Downstream consumers are written in three different languages. Walk me through how you manage this change safely using Confluent Schema Registry.

Accepted Answer

Schema Registry enforces compatibility rules. Adding a required field without a default value is a breaking change (BACKWARD incompatible) because existing consumers reading old messages will fail deserialization. The safe approach is to make the new field optional with a default value (null or a sensible default in Avro), which is BACKWARD compatible and allows old consumers to handle new messages gracefully. Schema Registry's compatibility modes: BACKWARD means new schema can read old data, FORWARD means old schema can read new data, FULL means both. Strong candidates describe the migration plan: register the new schema, deploy producers to start writing new schema, deploy consumers to handle both old and new schemas (using default values for missing fields), then eventually deprecate old schema. They also mention using schema subjects per topic vs. per record type.

Question 26

You have a Spark Structured Streaming job reading from Kafka. Explain how checkpointing works, what is stored in the checkpoint directory, and what happens when the job restarts after a failure.

Accepted Answer

Checkpointing stores three categories of data: offsets (the Kafka partition offsets the stream has committed to reading), committed offsets (offsets whose micro-batch has completed and been written to the sink), and state (for stateful operations like aggregations or joins, stored in RocksDB or HDFS). On restart, Spark reads the latest committed offset and resumes from there, re-processing any micro-batch that started but did not commit. End-to-end exactly-once requires an idempotent sink or a transactional sink like Delta Lake's foreachBatch with transaction support. Strong answers mention that changing a streaming query's logic requires deleting the checkpoint (losing state) or using the migration mechanism, and that HDFS or S3 with strong consistency is required for reliable checkpointing.

Question 27

Describe how Snowflake's zero-copy cloning works under the hood and walk me through three concrete data engineering scenarios where you would use it.

Accepted Answer

Zero-copy cloning creates a clone of a table, schema, or database at a point in time by copying only the metadata pointer to the underlying micro-partitions, not the data itself. It is nearly instant and costs nothing in storage until the clone diverges from the source. Practical scenarios: (1) creating a development copy of production data for engineers to test transformations without risk; (2) creating a pre-deployment snapshot before running a destructive migration, so you can roll back by simply swapping back; (3) cloning a table to a staging environment for QA testing with production-sized data. Strong candidates note that clones do not inherit row access policies from the source by default and that both the source and clone consume storage credit once either diverges.

Question 28

At 6 AM, you discover that a source system sent corrupted data for the past 12 hours and your pipeline has already loaded it into the warehouse and refreshed dashboards that executives see at 8 AM. The source system cannot resend clean data until the afternoon. What do you do?

Accepted Answer

Immediate actions: identify exactly which tables and time windows are affected, revert to the last known-good snapshot (using Snowflake time travel or a backup), or remove the affected rows from affected partitions. Then notify stakeholders proactively before the 8 AM meeting with a clear description of what is wrong and what the timeline is for correction. If reverting is not possible, take down affected dashboards or add a data quality warning banner. When clean data arrives, rerun the pipeline and validate before republishing. Strong candidates also describe adding a source data validation step that catches this class of corruption before loading.

Question 29

A Spark job that has been running successfully for 6 months started failing last week with java.lang.OutOfMemoryError: GC overhead limit exceeded. The input data size has grown 40%. Walk me through your debugging process and the fixes you would try.

Accepted Answer

Start in the Spark UI: check which stage is failing, look at the task details for the failing executor (memory spill, GC time, task size). Common causes at 40% data growth: a broadcast join table that was under the threshold before is now too large and causes OOM when materialized on every executor; a groupBy that accumulates state per key in memory is now exceeding executor heap; insufficient executor memory for the data volume. Fixes in order of effort: increase spark.executor.memory, reduce parallelism to give each task more memory, replace the broadcast join with a sort-merge join, switch the aggregation from groupByKey to reduceByKey, or repartition before the heavy aggregation to reduce per-task data volume. Check if executor memory overhead (spark.executor.memoryOverhead) is set adequately for non-heap use.

Question 30

You get an alert that your Kafka consumer group processing a payment events topic has 10 million messages of lag and is growing by 500,000 per minute. The downstream pipeline enriches payments for a fraud detection system. What do you do?

Accepted Answer

Immediate triage: check consumer CPU and memory (are consumers healthy?), check if consumer count matches partition count (if fewer consumers than partitions, some partitions are unassigned), check if a recent deployment changed processing logic to be slower. Mitigation options: scale out consumers (must not exceed partition count), increase batch size for more efficient processing, check if an expensive external call in the processing loop (database lookup, API call) can be batched or cached, and temporarily route the consumer to a higher-resource deployment. If lag is unrecoverable in time for the fraud system SLA, consider a fast-path that uses simplified enrichment to process the backlog while the full enrichment catches up in parallel. Post-incident: add lag-per-partition alerting, consider adding more partitions (with the ordering implications).

Questions

What happens during a Kafka consumer group rebalance?Role-specificmediumVery common

As asked

Sample answer outline

Expect these follow-ups

Compare dbt incremental strategies and when you would use eachRole-specificmediumVery common

As asked

Sample answer outline

Expect these follow-ups

Implement SCD Type 2 in a modern data warehouseRole-specificmediumVery common

As asked

Sample answer outline

Expect these follow-ups

How do you choose a Kafka partition count and key strategy?Role-specificmediumVery common

As asked

Sample answer outline

Expect these follow-ups

How do you implement data quality checks in a production pipeline?Role-specificmediumVery common

As asked

Sample answer outline

Expect these follow-ups

How do you make a batch pipeline idempotent?Role-specificmediumVery common

As asked

Sample answer outline

Expect these follow-ups

When would you use ETL versus ELT and how does cloud warehouse pricing affect that decisioRole-specificeasyVery common

As asked

Sample answer outline

Expect these follow-ups

A source team drops a column your pipeline depends on without warning. What do you do?Role-specificmediumVery common

As asked

Sample answer outline

Expect these follow-ups

Two teams report different revenue numbers from the same warehouse. What do you do?Role-specificmediumVery common

As asked

Sample answer outline

Expect these follow-ups

When would you choose Kafka over Kinesis?Role-specificmediumCommon

As asked

Sample answer outline

Expect these follow-ups

Pick a modern data stack for a Series A startupRole-specificmediumCommon

As asked

Sample answer outline

Expect these follow-ups

Explain Spark shuffle internals and when it causes performance problemsRole-specifichardCommon

As asked

Sample answer outline

Expect these follow-ups

When and why would you define a clustering key in Snowflake?Role-specificmediumCommon

As asked

Sample answer outline

Expect these follow-ups

How does Debezium handle failures, and how do you deal with duplicates downstream?Role-specifichardCommon

As asked

Sample answer outline

Expect these follow-ups

How does BigQuery partition pruning work and when does it fail?Role-specificmediumCommon

As asked

Sample answer outline

Expect these follow-ups

Explain how Kafka achieves exactly-once semantics end to endRole-specifichardCommon

As asked

Sample answer outline

Expect these follow-ups

What are the most common Airflow DAG design mistakes and how do you avoid them?Role-specificmediumCommon

As asked

Sample answer outline

Expect these follow-ups

Compare Lambda and Kappa architectures for a real-time analytics use caseRole-specifichardCommon

As asked

Sample answer outline

Expect these follow-ups

Read and act on a Postgres EXPLAIN ANALYZE outputRole-specificmediumCommon

As asked

Sample answer outline

Expect these follow-ups

Explain Flink watermarks and how they handle out-of-order eventsRole-specifichardCommon

As asked

Sample answer outline