Paul Reiners Paul Reiners - 1 year ago 210
Scala Question

sryza/spark-timeseries: NoSuchMethodError: scala.runtime.IntRef.create(I)Lscala/runtime/IntRef;

I have a Scala project that I build with sbt. It uses the sryza/spark-timeseries library.

I am trying to run the following simple code:

val tsAirPassengers = new DenseVector(Array(

val period = 12
val model = HoltWinters.fitModel(tsAirPassengers, period, "additive", "BOBYQA")

It builds fine, but when I try to run it, I get this error:

Exception in thread "main" java.lang.NoSuchMethodError: scala.runtime.IntRef.create(I)Lscala/runtime/IntRef;
at com.cloudera.sparkts.models.HoltWintersModel.convolve(HoltWinters.scala:252)
at com.cloudera.sparkts.models.HoltWintersModel.initHoltWinters(HoltWinters.scala:277)
at com.cloudera.sparkts.models.HoltWintersModel.getHoltWintersComponents(HoltWinters.scala:190)

The error occurs on this line:

val model = HoltWinters.fitModel(tsAirPassengers, period, "additive", "BOBYQA")

My build.sbt includes:

name := "acme-project"
version := "0.0.1"
scalaVersion := "2.10.5"

libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-hive" % "1.6.0",
"net.liftweb" %% "lift-json" % "2.5+",
"com.github.seratch" %% "awscala" % "0.3.+",
"org.apache.spark" % "spark-mllib_2.10" % "1.6.2"

I have placed sparkts-0.4.0-SNAPSHOT.jar in the lib folder of my project. (I would have preferred to add a libraryDependency, but spark-ts does not appear to be on Maven Central.)

What is causing this run-time error?

Answer Source

The library requires Scala 2.11, not 2.10, and Spark 2.0, not 1.6.2, as you can see from


in pom.xml. You can try changing these and seeing if it still compiles, find which older version of sparkts is compatible with your versions, or update your project's Scala and Spark versions (don't miss spark-mllib_2.10 in this case).

Also, if you put the jar into lib folder, you also have to put its dependencies there (and their dependencies, etc.) or into libraryDependencies. Instead, publish sparkts into your local repository using mvn install (IIRC) and add it to libraryDependencies, which will allow SBT to resolve its dependencies.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download