Dimebag Dimebag - 1 year ago 166
Scala Question

Use library in Spark-shell

I want to use this library in spark-shell and/or in a .scala file to manipulate some data. How do I do that? I cannot use maven.

EDIT for possible dupl: I also do not have a jar; if that is part of the solution, how do I make a jar from that library?

Answer Source

The library you reference is available on Maven Central, and spark-shell can automatically download libraries from Maven Central and a few other popular repositories if you give it the correct Maven coordinates. You don't need to explicitly use Maven. (In fact, it even lets you specify your own additional Maven repositories and searches those as well.) See http://spark.apache.org/docs/latest/programming-guide.html#using-the-shell

In your case specifically, the command should be something like

./bin/spark-shell --master local[4] --packages "dk.tbsalling:aismessages:2.2.1"
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download