eric_kernfeld eric_kernfeld - 8 months ago 14
Java Question

Communicate correct Java version to Hadoop

I have basically this problem, except that when I go to the folder where hadoop is installed, there is no

, so I don't know where to look for
. How can I get Hadoop to look for my up-to-data Java rather than my mandatory OSX legacy Java?


I've tried hadoop-2.7.2 locally, in my experience, you can specify your configuration directory by environment HADOOP_CONF_DIR.

To put it simple, you can just export HADOOP_CONF_DIR before you start HDFS by At this point, you can add your own JAVA_HOME at $HADOOP_CONF_DIR/ By the way, $HADOOP_CONF_DIR usually points to a directory copy from the hadoop release. For example, in hadoop-2.7.2 you can just copy hadoop-2.7.2/etc/hadoop/ and update the necessary configuration and specify it as your own HADOOP_CONF_DIR.

Thanks for eric_kernfeld's improvement:

UPDATE: since OP used a Homebrew installation, there were multiple directories named hadoop, and OP had to go through <version>/libexec/etc/hadoop.

At the end, hadoop-2.7.2/etc/hadoop/(please use your own version instead of 2.7.2) is the default HADOOP_CONF_DIR, which means you can just update hadoop-2.7.2/etc/hadoop/ as you wish.

However, there might be some difference if your hadoop release early than 2.x.x.