Himanshu Yadav Himanshu Yadav - 1 month ago 14
Scala Question

Gradle: Setting up Scala project with Apache Spark in Eclipse

I am not able to setup a Scala project with Apache Spark dependency in Eclipse. Using a Scala IDE plugin and Gradle plugins in Eclipse.

build.gradle
project looks like this:

apply plugin: 'scala'
apply plugin: 'eclipse'

repositories{
mavenCentral()
mavenLocal()
}

dependencies{
compile 'org.slf4j:slf4j-api:1.7.5'
compile "org.scala-lang:scala-library:2.11.2"
compile 'com.sparkjava:spark-core:2.3'
testCompile "junit:junit:4.11"
}

task run(type: JavaExec, dependsOn: classes) {
main = 'Main'
classpath sourceSets.main.runtimeClasspath
classpath configurations.runtime
}


Under the
Referenced Libraries
I can see spark-core-2.3.jar. But I can't import any Spark library into Scala class.

I did try running
gradle eclipse
command but no luck.

Answer

You're referencing the wrong dependency - instead of com.sparkjava:spark-core:2.3 (which belongs to another project, Spark web framework), you should include:

compile 'org.apache.spark:spark-core_2.11:2.0.1'

This uses latest stable version (2.0.1).