user3354890 user3354890 - 25 days ago 9
Java Question

How does apache storm create its bolt implementations

After implementing a very simple apache storm topology in java I ran into a NPE which caused some confusion.

This is basically my main method:

TologyBuilder builder = new TopologyBuilder();
builder.setSpout("spout", new CustomSpout(true), 4);
final CustomBolt bolt = new CustomBolt();
builder.setBolt("bolt", bolt, 1).shuffleGrouping("spout");

LocalCluster cluster = new LocalCluster();
Map conf = new HashMap();
conf.put(Config.TOPOLOGY_WORKERS, 10);
cluster.submitTopology("test", conf, createTopology());


The
CustomBolt
extends
AbstractBasic
class and implements
IRichBolt
. They have these fields:

abstract class AbstractBasic {
A a;
B b;
}

CustomBolt extends AbstractBasic implements IRichBolt {
A anotherA;
C c;
CustomBolt() {
this.anotherA = new A(true);
this.a = new A(false);
this.b = new B();
this.c = new C();
}
}


On the
execute
method of
CustomBolt
there will be some insertions into all classes. However, the fields that are inherited from
AbstractBasic
are not instantiated, but the other ones, the ones that live directly in
CustomBolt
, are...

Does anyone know how a bolt is being contructed and how it's possible that some fields are initialized and some are not.

Answer

When bolts are submitted to the topology, they are serialized and sent to each worker where they are deserialized. Since AbstractBasic does not implement Serializable, its fields will not be serialized when CustomBolt is serialized.

With the example you give, you should be able to fix your problem by declaring that AbstractBasic implements Serializable:

abstract class AbstractBasic implements Serializable ...