asmaspark asmaspark - 5 months ago 10
Java Question

how to submit spark application on cmd

I have a Spark Application which I initially created using maven (on windows). I converted my maven project into an Eclipse project, and I am now working on it via Eclipse.

I am able to utilize and verify this by running the application through Eclipse.

Now, I am trying to submit my application through the command line. I use the spark-submit command as follows:

bin\spark-submit \
--class C:\Users\pc\Desktop\eclipse\myapp\src\main\java\myapp\Main.java \
--master local[8] \
C:\Users\pc\Desktop\eclipse\myjar.jar


When I submit my jar, I get an error saying can't load main class from Jar file?

what i am doing wrong?

is it the path of my main class (
--class C:\Users\pc\Desktop\eclipse\myapp\src\main\java\myapp\Main.java
)

Answer

--class where you put your class like packagename.ClassName
So I guess myapp.Main

  ./bin/spark-submit --class myapp.Main --master local[8] C:\Users\pc\Desktop\eclipse\myjar.jar
Comments