Aviral Kumar Aviral Kumar - 1 year ago 184
Scala Question

Hive table not found in Spark-SQL

I am facing a weird problem.
I am trying to connect to a table in my spark-sql java code by :

JavaSparkContext js = new JavaSparkContext();
SQLContext sc = new SQLContext(js);

DataFrame mainFile = sc.sql("Select * from db.table");

It gives me a table not found exception.
But when I do that in

spark-shell using scala, it works fine.The table gets accessed and I can print out the data also.
Any inputs on this issue?

Answer Source

Spark-shell provides HiveContext. If you want to use HiveContext in Java code then add dependency for that in your application and then use in java program. Please refer http://spark.apache.org/docs/1.6.2/sql-programming-guide.html#hive-tables

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download