toofrellik toofrellik - 2 months ago 120
Scala Question

SBT assembly falis

I am running a spark job through intellij. Job executes and gives me output. i need to take this job as jar file to server and run, but when i try to do

sbt assembly
it throws below error:

[error] Not a valid command: assembly
[error] Not a valid project ID: assembly
[error] Expected ':' (if selecting a configuration)
[error] Not a valid key: assembly
[error] assembly


my sbt version is
0.13.8


below is my build.sbt file:

import sbt._, Keys._
name := "mobilewalla"
version := "1.0"
scalaVersion := "2.11.7"
libraryDependencies ++= Seq("org.apache.spark" %% "spark-core" % "2.0.0",
"org.apache.spark" %% "spark-sql" % "2.0.0")


i added a file assembly.sbt under project dir. it contains:

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.3")


what am i missing here

Answer

Add these lines in your build.sbt

assemblyMergeStrategy in assembly := {
  case PathList("META-INF", xs @ _*) => MergeStrategy.discard
  case x => MergeStrategy.first
}
mainClass in assembly := Some("com.SparkMain")
resolvers += "spray repo" at "http://repo.spray.io"
assemblyJarName in assembly := "streaming-api.jar"

and include these lines in your plugins.sbt file

addSbtPlugin("io.spray" % "sbt-revolver" % "0.7.2")

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.13.0")