user2330778 user2330778 - 2 months ago 58
Scala Question

Error: Invalid or corrupt jarfile sbt/sbt-launch-0.13.5.jar

I have been trying to install spark using the tutorial and everytime I run the command sbt/sbt assembly, I get the error "Error: Invalid or corrupt jarfile sbt/sbt-launch-0.13.5.jar"

I have tried everything: seperately adding the sbt file to the sbt folder in the spark folder, installing sbt individually, checking the download and reinstalling it over again, but in vain. Any advice about what I am doing wrong? Thanks.

Answer

Ok, After playing around for a while I finally got it and hopefully this will work for you aswell. That tutorial builds spark, where they do provide prebuilt binaries. I'm using Spark 1.2.0 just as a note (1.4.1 wouldn't work for me)

This is on Ubuntu 15.04 but should work on 14.04 the same

1) Remove the following lines from your bashrc

export SCALA_HOME=/usr/local/src/scala/scala-2.10.4
export PATH=$SCALA_HOME/bin:$PATH

2) Remove and reinstall scala

sudo rm -rf /usr/local/src/scala
# The following line is only needed if you installed scala another way, if so remove the #
# sudo apt-get remove scala-library scala
wget http://www.scala-lang.org/files/archive/scala-2.11.7.deb
sudo dpkg -i scala-2.11.7.deb
sudo apt-get update
sudo apt-get install scala

3) Download PreBuilt Spark and extract

wget http://d3kbcqa49mib13.cloudfront.net/spark-1.2.0-bin-hadoop2.4.tgz
tar -xzvf spark-1.2.0-bin-hadoop2.4.tgz 

4) Run spark-shell

cd spark-1.2.0-bin-hadoop2.4/
./bin/spark-shell

Sources (basically where I've read from, this solution has been trial and error)

https://chongyaorobin.wordpress.com/2015/07/01/step-by-step-of-installing-apache-spark-on-apache-hadoop/
https://gist.github.com/visenger/5496675