user2524707 user2524707 - 1 month ago 26
Java Question

Kryo serialization caused fatal java runtime error with custom objects

I am trying to serialize hashmaps and collections of two custom classes (containing more hashmaps and collections).

Class1: NodeStorage.java

@NotNull
private final String id;

@Nullable
private String type;

@Nullable
private HashMap<String, String> properties;


Class2: RelationshipStorage.java

@NotNull
private final String id;

@Nullable
private String type;

@Nullable
private HashMap<String, String> properties;

@NotNull
private final NodeStorage startNode;

@NotNull
private final NodeStorage endNode;


The collections to serialize:

private HashMap<NodeStorage, NodeStorage> readsSetNode;
private HashMap<NodeStorage, NodeStorage> updateSetNode;
private ArrayList<NodeStorage> deleteSetNode;
private ArrayList<NodeStorage> createSetNode;

private HashMap<RelationshipStorage, RelationshipStorage> readsSetRelationship;
private HashMap<RelationshipStorage, RelationshipStorage> updateSetRelationship;
private ArrayList<RelationshipStorage> deleteSetRelationship;
private ArrayList<RelationshipStorage> createSetRelationship;


What I tried until now:

kryo.register(NodeStorage.class, 1);
kryo.register(RelationshipStorage.class, 2);
kryo.register(HashMap.class, mapSerializer);

mapSerializer.setKeyClass(NodeStorage.class, kryo.getSerializer(NodeStorage.class));
mapSerializer.setKeyClass(RelationshipStorage.class, kryo.getSerializer(RelationshipStorage.class));
mapSerializer.setValuesCanBeNull(false);
mapSerializer.setKeysCanBeNull(false);
listSerializer.setElementClass(NodeStorage.class, kryo.getSerializer(NodeStorage.class));
listSerializer.setElementClass(RelationshipStorage.class, kryo.getSerializer(RelationshipStorage.class));
listSerializer.setElementsCanBeNull(false);


public byte[] serialize()
{
ByteArrayOutputStream stream = new ByteArrayOutputStream();
Output output = new Output(stream);
mapSerializer.write(kryo, output, readsSetNode);
byte[] bytes = output.toBytes();
output.close();
return bytes;
}


I tried it with kryo.writeclassandobject but it didn't work as well. I get:

> > #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x00007f92f7f6efe0, pid=4637, tid=0x00007f92f94fd700
> #
> # JRE version: OpenJDK Runtime Environment (8.0_102-b14) (build 1.8.0_102-b14)
> # Java VM: OpenJDK 64-Bit Server VM (25.102-b14 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # V [libjvm.so+0x787fe0]
> #
> # Core dump written


Full code at: https://github.com/Raycoms/thesis

Declaration:

private Kryo kryo = new Kryo();
MapSerializer mapSerializer = new MapSerializer();
CollectionSerializer listSerializer = new CollectionSerializer();

cxw cxw
Answer

Kryo is not thread-safe, and practically any Java program has multiple threads going. Try the thread-pool approach, quoted below, detailed in the Kryo readme:

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.pool.*;

KryoFactory factory = new KryoFactory() {
  public Kryo create () {
    Kryo kryo = new Kryo();
    // configure kryo instance, customize settings
    return kryo;
  }
};
// Build pool with SoftReferences enabled (optional)
KryoPool pool = new KryoPool.Builder(factory).softReferences().build();

Then, in your serialize() function:

Kryo kryo = pool.borrow();
<... the rest of your code in serialize() before the return...>
pool.release(kryo);
return bytes;

As the README points out, you can alternatively use a callback:

// or use a callback to work with kryo - no need to borrow/release,
// that's done by `run`.
String value = pool.run(new KryoCallback() {
  public String execute(Kryo kryo) {
    return kryo.readObject(input, String.class);
  }
});
Comments