woot woot - 6 months ago 31
Python Question

Using pyleus: NoClassDefFoundError: backtype/storm/topology/IRichBolt

Seems like a

NoClassDefFoundError
would be an easy thing to fix, however I can't seem to figure this out.

I'm using pyleus on the word_count example. I am able to build using:

pyleaus build word_count/pyleus_topology.yaml


Then I attempt to run local (or even if I attempt to submit, I get the same error):

pyleus local --debug word_count.jar

Running: /mypath/jre/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/mypath/storm -Dstorm.log.dir=/mypath/storm/logs -Djava.library.path=/mypath/jre/lib/ -Dstorm.conf.file= -cp /mypath/storm/lib/storm-core-1.0.1.jar:/mypath/storm/lib/asm-5.0.3.jar:/mypath/storm/lib/log4j-over-slf4j-1.6.6.jar:/mypath/storm/lib/log4j-slf4j-impl-2.1.jar:/mypath/storm/lib/log4j-core-2.1.jar:/mypath/storm/lib/kryo-3.0.3.jar:/mypath/storm/lib/storm-rename-hack-1.0.1.jar:/mypath/storm/lib/objenesis-2.1.jar:/mypath/storm/lib/reflectasm-1.10.1.jar:/mypath/storm/lib/log4j-api-2.1.jar:/mypath/storm/lib/minlog-1.3.0.jar:/mypath/storm/lib/slf4j-api-1.7.7.jar:/mypath/storm/lib/clojure-1.7.0.jar:/mypath/storm/lib/servlet-api-2.5.jar:/mypath/storm/lib/disruptor-3.3.2.jar:word_count.jar:/mypath/storm/conf:/mypath/storm/bin -Dstorm.jar=word_count.jar com.yelp.pyleus.PyleusTopologyBuilder --local --debug
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/topology/IRichBolt
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: backtype.storm.topology.IRichBolt
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more


What's confusing to me here is that it does look like all the storm libs are included in the classpath if you scroll on my error log. What am I missing here? I'm using Storm 1.0.1, pyleus 0.3.0, Java 1.8, Python 2.7.

Answer

I guess I'll answer my own question. Based on what I'm seeing in github.com and the javadocs... I believe pyleus does not support 1.0.1. The javadocs show that in previous revisions, IRichBolt was in backtype.storm.topology however now it exists in org.apache.storm.topology. Who knows what else is incompatible.

So I guess for now, run an older version of Storm (I only saw a reference to 0.9.4 being compatible, so perhaps 0.9.6 would also work). I did see that there is an open pull request for 0.10.0 being worked on, so I guess it might be awhile before 1.0.1 is supported.

Comments