Georg Heiler Georg Heiler - 9 days ago 8
Scala Question

Why does custom Estimator end up overriding Nothing type?

I am implementing a custom estimator which has troubles accessing a parameter

ava.util.NoSuchElementException: Failed to find a default value for isInList
. It is defined as follows:

trait PreprocessingParams extends Params {
final val isInList = new Param[Array[String]](this, "isInList", "list of isInList items")
}


To better debug the problem I created a minimal example here https://gist.github.com/geoHeil/8dc7a6b6938a517f068e7fd6a981ed12 the
ExampleTrans
works just fine. However, I would rather like to include the functionality of the transformer into an estimator which performs some data cleaning as well.

But now I face strange compile issues
overriding method has wrong type - expecting Nothing


What is wrong with the return types of my
ExampleEstimator
?

Answer

You didn't specify the generic types of Estimator type constructor and so Nothing has been used instead.

Use:

class ExampleEstimator(override val uid: String) 
  extends Estimator[ExampleTransModel] with PreprocessingParams {
  ...
}

The full definition of Estimator is as follows:

abstract class Estimator[M <: Model[M]] extends PipelineStage

Note the generic type M that extends Model.