firvin firvin - 1 year ago 175
Scala Question

How to Scala Eclipse IDE to connect to existing Hive

I've setup the following in my laptop.

  1. Apache 2.7.1 Hadoop, single node

  2. Hive 2.1.0 running on local mode with metastore on MySQL. It is not in thriftserver mode

  3. Spark 2.0.0

  4. Scala 2.1.1

I've placed hive-site.xml from hive/conf to spark/conf. When i goto spark-shell, i can use sql context to create tables in hive and query them. I can access all the tables in hive. My issue is, when I use Eclipse IDE, i'm not able to connect to existing hive.

Eclipse can connect to existing spark master to submit the jobs, i can see the jobs in UI but when i use sparkSession to connect to hive, it always creates its own Derby database. I've searched extensively, but i cannot figure out.

  1. Should the hive be setup in thriftserver mode for eclipse to connect. Why is eclipse not using existing hive installation.

  2. Is there a way to connect to my existing hive installation from eclipse.

I'm not sure what else to look for.

Answer Source

Your hive-site.xml is not in path. Try adding config to classpath

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download