Himanshu Yadav Himanshu Yadav - 1 year ago 146
Scala Question

Gradle: Setting up Scala project with Apache Spark in Eclipse

I am not able to setup a Scala project with Apache Spark dependency in Eclipse. Using a Scala IDE plugin and Gradle plugins in Eclipse.

project looks like this:

apply plugin: 'scala'
apply plugin: 'eclipse'


compile 'org.slf4j:slf4j-api:1.7.5'
compile "org.scala-lang:scala-library:2.11.2"
compile 'com.sparkjava:spark-core:2.3'
testCompile "junit:junit:4.11"

task run(type: JavaExec, dependsOn: classes) {
main = 'Main'
classpath sourceSets.main.runtimeClasspath
classpath configurations.runtime

Under the
Referenced Libraries
I can see spark-core-2.3.jar. But I can't import any Spark library into Scala class.

I did try running
gradle eclipse
command but no luck.

Answer Source

You're referencing the wrong dependency - instead of com.sparkjava:spark-core:2.3 (which belongs to another project, Spark web framework), you should include:

compile 'org.apache.spark:spark-core_2.11:2.0.1'

This uses latest stable version (2.0.1).