Georg Heiler Georg Heiler - 9 months ago 65
Scala Question

Spark custom estimator override Nothing?

I am implementing a custom estimator which has troubles accessing a parameter

ava.util.NoSuchElementException: Failed to find a default value for isInList
. It is defined as follows:

trait PreprocessingParams extends Params {
final val isInList = new Param[Array[String]](this, "isInList", "list of isInList items")

To better debug the problem I created a minimal example here the
works just fine. However, I would rather like to include the functionality of the transformer into an estimator which performs some data cleaning as well.

But now I face strange compile issues
overriding method has wrong type - expecting Nothing

What is wrong with the return types of my

Answer Source

You didn't put generic types.


class ExampleEstimator(override val uid: String) extends Estimator[ExampleTransModel] with PreprocessingParams {

Estimator is parametrized with generic type M that extends Model. fit() should return type M. Without specifying explicity, Nothing is used