Scala engineer role-specific interview questions

As asked

Walk me through covariance and contravariance in Scala. When would you mark a type parameter as +A versus -A, and what does the compiler actually enforce when you do that?

Sample answer outline

A strong answer explains that +A means the type constructor is covariant, so List[Cat] is a subtype of List[Animal], while -A means contravariant, so Function1[-A, +B] accepts a wider input type. The candidate should explain that the compiler enforces positions: covariant type params can only appear in output positions, contravariant only in input positions. Bonus for mentioning invariance in mutable containers like Array[A] and why mutation breaks the substitution principle.

Expect these follow-ups

• Why is Array[T] invariant in Scala even though List[+T] is covariant?
• How does variance interact with bounds, like +A <: Animal?

As asked

How did Scala 3 change the implicit system? Walk me through the differences between Scala 2 implicits and Scala 3 givens and using clauses, and tell me about a real migration problem you would face going from 2 to 3.

Sample answer outline

Strong answer covers the renaming from implicit val/def to given/using, the new import syntax (import given), and how Scala 3 restricts implicit conversions to reduce footguns. Should mention the canonical given derivation pattern and how given instances replace implicit objects. A good candidate also mentions that implicit search rules changed and that some Scala 2 library implicits require a compatibility layer.

Expect these follow-ups

• How do you handle a library that still uses Scala 2 implicits when your codebase is on Scala 3?
• What scalacOption would you enable to get warnings about implicit conversions being applied silently in Scala 2, and what replaced that concern in Scala 3?

As asked

Show me how you would define a Show type class in Scala and provide instances for Int, String, and a case class Person. Then explain how the compiler resolves which instance to use at the call site.

Sample answer outline

Candidate should write a trait Show[A] with a method show(a: A): String, then provide implicit (or given in Scala 3) instances. The call site uses an implicit parameter or context bound syntax [A: Show]. They should describe the Scala 2 implicit search order: (1) local definitions and block scope including method parameters, (2) inherited members of the enclosing class, (3) implicit scope of types involved, which means the companion objects of A and Show. Explicitly imported implicits also fall in tier 1 and beat companion objects. Bonus for explaining ambiguity: two instances at the same priority tier produce a compile error, not a silent pick.

Expect these follow-ups

• What is a context bound and how does it desugar at the bytecode level?
• How would you derive a Show instance for a generic case class automatically in Scala 3 using Mirror?

As asked

Walk me through Akka's supervision tree model. What are the built-in supervision strategies, when would you use OneForOneStrategy versus AllForOneStrategy, and how do you avoid silently swallowing failures?

Sample answer outline

Strong answer covers the hierarchy: every actor has a parent that supervises it. OneForOneStrategy applies the directive (Resume, Restart, Stop, Escalate) only to the failed child; AllForOneStrategy applies it to all siblings too. Should mention the Decider partial function and how Escalate bubbles up. A common pitfall is not logging inside the decider, so failures vanish silently. Good candidates also mention Akka Typed's behavior-based supervision.

Expect these follow-ups

• How does supervision change in Akka Typed compared to classic actors?
• What happens to messages in the mailbox when an actor restarts?

As asked

Explain what happens during a shuffle in Spark. What triggers a shuffle, what are the performance costs, and what are three concrete techniques you use in production to reduce shuffle overhead?

Sample answer outline

Shuffle is triggered by wide transformations like groupByKey, join, or repartition. Data is written to local disk by map tasks and read by reduce tasks across the network. Costs include disk I/O, serialization, and network transfer. Concrete optimizations: (1) use reduceByKey instead of groupByKey to combine locally before shuffle, (2) broadcast join when one side is small enough to fit in memory, (3) partition on a relevant key beforehand to avoid re-shuffling. Bonus: mention AQE (Adaptive Query Execution) in Spark 3.

Expect these follow-ups

• How does a sort-merge join differ from a broadcast hash join in Spark, and when does AQE switch between them?
• How do you diagnose excessive shuffle spill in the Spark UI?

As asked

Why do libraries like Cats Effect and ZIO define their own IO type instead of using Scala's built-in Future? What are the concrete problems with Future that IO solves?

Sample answer outline

Key problems with Future: it is eager (starts executing on creation, not on composition), it is not referentially transparent (two references to the same Future represent the same running computation), and cancellation is not built in. IO is lazy (a description of a computation), referentially transparent, and supports cancellation and resource safety. A strong answer gives a concrete example: val x = Future(sideEffect()) called twice reuses one computation, while IO(sideEffect()).replicateA(2) runs it twice.

Expect these follow-ups

• What does referential transparency mean in practice for testing async code?
• How does Cats Effect's IO runtime differ from Scala's global ExecutionContext?

As asked

How does Scala use sealed traits to enable exhaustive pattern matching, and what does the compiler actually check? Give me an example where a match is not exhaustive and show what happens at runtime versus compile time.

Sample answer outline

Sealed traits restrict subclasses to the same file, so the compiler knows all cases at compile time. A non-exhaustive match produces a warning (not an error by default) and throws MatchError at runtime for unhandled cases. Use -Xfatal-warnings or the @nowarn annotation intentionally. A strong candidate also knows about enum in Scala 3 as a cleaner sealed hierarchy and about using @exhaustive in some effect libraries.

Expect these follow-ups

• How do you turn the non-exhaustive match warning into a compile error without -Xfatal-warnings globally?
• What is the difference between sealed and final in Scala?

As asked

Why do DataFrames generally outperform equivalent RDD code in Spark, even though they run on the same JVM? What specific optimizations does the Catalyst optimizer apply that an RDD pipeline cannot get?

Sample answer outline

Catalyst optimizer transforms a logical plan through analysis, optimization, and physical planning. Key optimizations RDDs miss: predicate pushdown (filter before a scan), column pruning (read only needed columns from Parquet), join reordering, and code generation (Tungsten whole-stage codegen produces tight JVM bytecode). RDDs treat data as opaque objects so the optimizer has nothing to work with. A strong answer also notes that Datasets[T] give type safety at a slight performance cost versus plain DataFrames.

Expect these follow-ups

• When would you still choose an RDD over a DataFrame in a real Spark job?
• How does Tungsten's off-heap memory model reduce GC pressure compared to standard JVM serialization?

As asked

How do you decide how many partitions a Spark job should have, and what are the symptoms of too few versus too many partitions? What is spark.sql.shuffle.partitions and when should you change it?

Sample answer outline

Too few partitions: not enough parallelism, tasks run long, memory pressure per task. Too many: task scheduling overhead dominates, small files on write. Rule of thumb: 2 to 4 partitions per CPU core in the cluster, with tasks running 100ms to a few minutes. spark.sql.shuffle.partitions defaults to 200 and is often wrong for small or very large datasets. AQE in Spark 3 can coalesce small shuffle partitions automatically. A strong answer mentions that Parquet output partitions directly affect downstream read performance.

Expect these follow-ups

• How does AQE's coalescing of shuffle partitions work and what is its minimum partition size?
• What is the relationship between partition count and the number of output files on S3 or HDFS?

As asked

Walk me through exactly how the Scala compiler desugars a for-comprehension with a yield. Show me a three-step for-comprehension and its desugared form, and explain what requirements this places on the types involved.

Sample answer outline

A for-comprehension with yield desugars to nested flatMap calls with a final map. The type must have flatMap(A => M[B]): M[B] and map(A => B): M[B]. A guard (if condition) desugars to withFilter. This means any type with those methods works, not just monads from Cats. A strong candidate writes the desugared form explicitly and notes that this is why IO, Future, and List all work in for-comprehensions even though they are unrelated types.

Expect these follow-ups

• What happens if one step in a for-comprehension has type Option[A] and another has type Future[B]?
• How do you use monad transformers or flatMapF to mix effects inside a for-comprehension?

As asked

In Cats Effect 3, how does fiber cancellation work, and what can go wrong if you do not handle it correctly in a resource-acquisition scenario? Give me a concrete example where cancellation causes a resource leak and how you fix it.

Sample answer outline

A strong answer explains that IO is cancelable by default and that cancellation is delivered as a finalizer after the current step completes. The danger is acquiring a resource (like opening a DB connection) and being canceled before the release finalizer runs. The fix is Resource.make or bracket, where the release is guaranteed to run even on cancellation. Bonus for mentioning uncancelable regions and when to use them.

Expect these follow-ups

• When would you use IO.uncancelable and what are the risks of marking a block uncancelable?
• How does Cats Effect's scheduler differ from Java's thread pool and why does it matter for CPU-bound work?

As asked

Explain how ZIO's ZLayer system works. How do you wire up a service graph, and how does the type system tell you at compile time that a dependency is missing?

Sample answer outline

Candidate should explain that ZIO[R, E, A] carries its environment R in the type, and ZLayer[In, E, Out] transforms one environment into another. Wiring uses ZLayer.make or provide, and the compiler checks that all required environment types are satisfied. A missing dependency produces a type error. Good candidates mention that ZLayer is lazy by default and supports horizontal (++) and vertical (>>>) composition.

Expect these follow-ups

• How do you scope a ZLayer to a request versus the whole application lifetime?
• Compare ZIO layers with a Guice-style DI framework in terms of testability.

As asked

How does Akka Streams implement backpressure, and what happens when a downstream stage cannot keep up with a fast producer? Describe the flow of demand signals in a linear graph.

Sample answer outline

Candidate should explain that Akka Streams uses the Reactive Streams protocol: downstream stages pull by sending demand (request(n)) upstream, and the source only emits when demand exists. When downstream is slow, demand stops flowing and the source naturally slows or buffers. Overflow strategies like dropHead, dropTail, or fail handle the case when buffers fill. A strong answer also covers async boundaries and how bufferSize affects latency versus throughput.

Expect these follow-ups

• How do you add a fixed-size buffer between two stages with a specific overflow strategy?
• What is the difference between a mapAsync stage and a plain map stage in terms of backpressure?

As asked

You are running a Spark join and 80 percent of the data lands on two partitions, causing those tasks to run 50 times longer than the others. Walk me through how you diagnose this and what techniques you use to fix it.

Sample answer outline

Diagnosis involves checking the Spark UI task duration distribution and looking for stage stragglers. Fixes include salting the join key (adding a random prefix to spread keys, then stripping it after joining), using AQE's skew join hint in Spark 3, or pre-aggregating before joining. A strong answer distinguishes between skew in groupByKey versus skew in join and applies the right fix to each. Also worth mentioning the skewHint API and when a full repartition is actually cheaper.

Expect these follow-ups

• How does salting work specifically, and what do you do on the broadcast side when you salt?
• What Spark 3 AQE feature handles skew automatically and what are its limits?

As asked

Your Scala service experiences GC pauses of 3 to 5 seconds under load, causing timeouts. Walk me through how you diagnose which GC is running, what is causing the pauses, and what changes you make first.

Sample answer outline

Start with GC logs (-Xlog:gc*) to identify pause times and GC type (G1, ZGC, Shenandoah). Long pauses in G1 often mean humongous allocations or region evacuation failures. Fixes include increasing -Xms to match -Xmx to avoid heap expansion, tuning -XX:MaxGCPauseMillis, or switching to ZGC/Shenandoah for lower latency. For Scala specifically, watch out for boxed collections producing large amounts of short-lived objects. Heap profiling with JFR or async-profiler reveals allocation hot spots.

Expect these follow-ups

• How do you detect object promotion failures in G1 and what causes them?
• When would you choose ZGC over G1 for a Spark driver process?

As asked

Explain monad transformers in Cats. When would you use EitherT[IO, AppError, A] versus a plain IO[Either[AppError, A]], and what are the downsides of deep transformer stacks?

Sample answer outline

EitherT wraps IO[Either[E,A]] to allow flatMap to short-circuit on Left without manually unwrapping. The upside is cleaner for-comprehension syntax. Downsides are performance overhead from extra wrappers, harder stack traces, and compile-time slowness with deep stacks. Many production codebases now prefer IO[Either[E,A]] directly or use ZIO[R, E, A] whose E channel is built in. A strong answer discusses when the ergonomic benefit outweighs the cost.

Expect these follow-ups

• How do you lift an IO[A] into an EitherT[IO, E, A] without losing error information?
• What are mtl-style type classes (MonadError, MonadState) and how do they avoid some transformer downsides?

As asked

What is a higher-kinded type in Scala, and why does the language require the -language:higherKinds flag (or Scala 3's equivalent) to use them? Give me a concrete example where you need HKTs to write a generic abstraction.

Sample answer outline

A higher-kinded type is a type that takes a type constructor as a parameter, for example trait Functor[F[_]]. This lets you write map once for any F that is a Functor, whether List, Option, or IO. The flag (now a warning in Scala 2, removed in Scala 3) signals intent to use this advanced feature. A strong answer demonstrates writing a Functor[F[_]] trait with a map method and providing instances for List and Option.

Expect these follow-ups

• How does kind-projector simplify the syntax for partial type application in Scala 2?
• In Scala 3, how do match types and type lambdas replace some uses of kind-projector?

As asked

Describe the full implicit search priority order in Scala 2. When you have two valid implicit values in scope, which one wins, and how do you use this to provide default instances that can be overridden?

Sample answer outline

Priority order (high to low): local scope including method parameters, inherited scope (if inside a class), then implicit scope (companion objects of types involved). The classic two-priority trick defines a LowPriorityInstances trait with a fallback instance; the companion object extends it and provides a higher-priority specific instance. When two implicits at the same priority are in scope, the compiler errors. A candidate should be able to explain why this matters for type class libraries like Cats.

Expect these follow-ups

• What is the difference between implicit scope and lexical scope for implicit resolution?
• How does Scala 3 change implicit search priority with given instances?

As asked

When would you use Cats Validated[E, A] instead of Either[E, A] for error handling? Give me a realistic example like validating a form or a config object.

Sample answer outline

Either short-circuits on the first error. Validated is an applicative, not a monad, so it accumulates all errors when combined with mapN or traverse. For form validation where you want to report all field errors at once, Validated is the right choice. The answer should show ValidatedNel (NonEmptyList of errors) and how to convert to Either when you need sequential steps that depend on earlier results. Bonus: explain why Validated cannot have a Monad instance.

Expect these follow-ups

• Why can Validated not be a monad given that it accumulates errors?
• How do you combine a Validated[Nel[E], A] result with an IO that fetches data from a database?

As asked

What are the dangers of implicit conversions in Scala 2, and how did Scala 3 address them? Give me an example where an implicit conversion causes a bug that is hard to spot.

Sample answer outline

Implicit conversions can silently convert types, hiding type errors and making code hard to reason about. Classic example: implicit def intToString(i: Int): String means passing an Int where a String is expected compiles silently. Scala 3 requires explicit import scala.language.implicitConversions and marking conversion objects with Conversion[A, B]. This makes conversions opt-in and visible. A strong candidate also mentions extension methods in Scala 3 as the preferred alternative for adding methods.

Expect these follow-ups

• How do extension methods in Scala 3 replace the enrichment pattern from Scala 2?
• In Scala 2, which scalacOption warns you when an implicit conversion is inserted by the compiler, and why is that warning useful?

As asked

How do watermarks work in Spark Structured Streaming? What happens to a late event that arrives after the watermark has passed, and how do you set the watermark correctly for your use case?

Sample answer outline

A watermark tells Spark how late events can be. It is computed as max(event_time) minus the configured delay. Events arriving after the watermark for their window are dropped. Setting it too small drops valid late data; too large delays output and increases state size. A strong answer explains that windowed aggregations are held in state memory until the watermark passes the window end, at which point results are emitted. Also covers withWatermark and how the late data is handled in append versus update output modes.

Expect these follow-ups

• What is the difference between append mode and update mode for a windowed aggregation?
• How do you test that your watermark setting is correct in a staging environment before going to production?

As asked

When would you use ZIO Ref versus ZIO STM (Software Transactional Memory) for shared mutable state? Give me a scenario where Ref is enough and one where STM is necessary.

Sample answer outline

Ref wraps an AtomicReference and is good for single-variable updates using modify or update with CAS semantics. When you need to update two related values atomically without a lock, STM is the right tool. Example: transferring funds between two accounts requires decrementing one and incrementing the other atomically. STM.atomically wraps TRef updates in a transaction that retries automatically on conflict. A strong answer explains that STM is optimistic and can be slower under high contention.

Expect these follow-ups

• What does STM.retry do and when would you call it explicitly inside a transaction?
• How does ZIO STM's retry mechanism differ from a traditional lock-based approach?

As asked

Describe the Akka HTTP routing DSL. How does a request travel from the network to your handler, and what is the role of the Directive abstraction?

Sample answer outline

Akka HTTP routes are a function RequestContext => Future[RouteResult]. Directives are composable building blocks that filter, extract, or transform the context before passing it to the inner route. For example, path extracts URL segments, get filters by method, and entity(as[T]) unmarshals the body. The request is handled by an Akka actor system running the HTTP server using Akka Streams under the hood. A strong answer covers rejection handling and how unmatched routes produce Rejected results that combine up the route tree.

Expect these follow-ups

• How do you handle authentication in Akka HTTP without blocking the dispatcher thread?
• How do you stream a large response body from Akka HTTP without loading it all into memory?

As asked

Walk me through the Cats type class hierarchy from Functor through Applicative to Monad. What does each add over the previous, and when is Applicative sufficient where Monad is overkill?

Sample answer outline

Functor adds map (lift a pure function into the context). Applicative adds pure (lift a value) and ap or mapN (combine independent effects). Monad adds flatMap (chain effects that depend on previous results). Applicative is enough when effects are independent, enabling parallelism (parMapN in Cats). Monad is needed when the next effect depends on the previous value. A strong answer gives a concrete example: validating two fields independently uses Applicative, but fetching a user and then their posts requires Monad.

Expect these follow-ups

• What is the difference between Applicative and ApplicativeError in Cats?
• Why can Validated not be a Monad even though it has an Applicative instance?

As asked

Explain the typeclass pattern in Scala and give a practical example where it improves an API without making the codebase harder to maintain.

Sample answer outline

A typeclass represents behaviour for a type without requiring inheritance or modifying the original type. In Scala this is usually encoded as a trait plus given or implicit instances and extension methods. It improves APIs when behaviour needs to be added across types from different modules, such as JSON encoding, ordering or database binding. The candidate should discuss discoverability and error messages because excessive implicit abstraction can hurt maintainability. Strong answers show restraint: use typeclasses at stable boundaries, not for every helper function.

Reference implementation (scala)

trait CsvEncoder[A] {
  def encode(value: A): List[String]
}

object CsvEncoder {
  given CsvEncoder[User] with {
    def encode(value: User): List[String] =
      List(value.id.toString, value.email)
  }
}

def writeCsv[A](rows: List[A])(using encoder: CsvEncoder[A]): List[String] =
  rows.map(row => encoder.encode(row).mkString(","))

Expect these follow-ups

• How do given instances in Scala 3 improve or worsen implicit search?
• When is inheritance simpler than a typeclass?
• How do you debug an ambiguous implicit error?