manuel mourato manuel mourato - 1 year ago 86
Java Question

Spark 1.6-Failed to locate the winutils binary in the hadoop binary path

I know that there is a very similar post to this one(Failed to locate the winutils binary in the hadoop binary path), however, I have tried every step that was suggested and the same error still appears.

I'm trying to use the Apache Spark version 1.6.0 on Windows 7 to perform the tutorial on this page, specifically using this code:

./bin/run-example streaming.JavaNetworkWordCount localhost 9999

However, this error keeps appearing:
enter image description here

After reading this post
Failed to locate the winutils binary in the hadoop binary path

I realized I needed the winutils.exe file, so I have downloaded a hadoop binary 2.6.0 with it, defined an Environment Variable called HADOOP_HOME:

with value C:\Users\GERAL\Desktop\hadoop-2.6.0\bin

and placed it on Path like this: %HADOOP_HOME%

Yet the same error still appears when I try the code. Does anyone know how to solve this?

Answer Source

If you are running Spark on Windows with Hadoop, then you need to ensure your windows hadoop installation is properly installed. to run spark you need to have winutils.exe and winutils.dll in your hadoop home directory bin folder.

I would ask you to try this first:

1) You can download .dll and .exe fils from the bundle in below link.

2) Copy winutils.exe and winutils.dll from that folder to your $HADOOP_HOME/bin.

3) Set the HADOOP_HOME either in your or at the command, and add HADOOP_HOME/bin to PATH.

and then try running.

If you need any assistance for hadoop installation help, there is a nice link, you can try it.

But, that can wait. you can try the first few steps.