Scala Magic

Today we were discussing the benefits of handling errors or missing values without resorting to exceptions via Scala's Option data type.

Imagine, you have a list of options on Strings and you want to work with the values (the Somes) and ignore the absent values (the Nones). One way of doing that would be something like this:

val names = List(Some("Peter"), None, Some("Mary"))
  
names.flatMap(n => n.map("Hello " + _))

You might remember faintly what the type signature of flatMap was and think: hang on a minute, what's going on here?

def flatMap[A,B](f: A => List[B]): List[B]

This is obviously simplified compared to the actual Scala standard library version of flatMap (look for TraversableLike#flatMap). But still flatMap takes a function from A to List[B], while Option's map looks like this:

@inline final def map[B](f: A => B): Option[B] =
    if (isEmpty) None else Some(f(this.get))

So how does the result of map which is of type Option[String] turn magically into List[Option[B]]? Implicit conversion is the answer. If scalac finds a type mismatch between what we are passing in and what the function expects it looks for an implicit view definition to satisfy the expression.

We can use scalac's print option to see what's going on under the hood:

p_brc@brc-mbp ~/s/s/S/s/t/s/intro> scalac -print Tester.scala 

//lots of code removed 
// here is the call to flatMap

      Tester.this.names().flatMap({
        (new anonymous class anonfun$1(): Function1)
      }, immutable.this.List.canBuildFrom());


// more synthetic classes  removed
// here is the synthetic class representing the outer lambda function:

  @SerialVersionUID(0) final  class Tester$$anonfun$1 extends runtime.AbstractFunction1 with Serializable {
    final def apply(n: Option): Iterable = scala.this.Option.option2Iterable(n.map({
      (new anonymous class anonfun$apply$1(Tester$$anonfun$1.this): Function1)
    }));
    final  def apply(v1: Object): Object = Tester$$anonfun$1.this.apply(v1.$asInstanceOf[Option]());
    def (): anonymous class anonfun$1 = {
      Tester$$anonfun$1.super.();
      ()
    }
  }

You can see how the lambda is represented by a synthetic class Tester$$anonfun$1. In its apply method the code we defined is wrapped in a call to option2Iterable.

Option.option2Iterable is the implicit view definition that was in scope and was automagically injected by the compiler. It is quite simple and looks like this:

implicit def option2Iterable[A](xo: Option[A]): Iterable[A] = xo.toList

Riddle solved. Are we happy now? Maybe, but implicit conversions are certainly one of Scala's features that can lead to confusing if not unintelligible code quite easily. But then again, that is nothing new and if you look at the related literature, be it Odersky, Spoon, Venners: Programming in Scala or Suereth: Scala in Depth, the chapters covering implicit conversions are full of warnings and calls for restraint. Therefore: advance with caution.