vijar vijar - 6 months ago 22
Java Question

Observer Pattern for operations performed on Chronicle Map

I am looking forward to use Chronicle Map as a datastore/data cache and intend to share it with other JVM processes running on the same box to reduce memory footprint of each of the other JVM processes else each JVM process will load the same data. Is it possible to get notification on each JVM process whenever an entry is added or deleted from the datastore? Is it really going to reduce the memory footprint? Because, each JVM process anyway going to create some domain objects.

I looked at the API and the documentation but it wasn't too clear to me how to implement my use case. MapMethods and remoteOperations came closest to it and maybe, the way to go. I want to know what's the right way to get the intended functionality or am I way off.

If I am on the right track, then, I am guessing that the mapMethods/remoteOperations only need to be provided on the observer side not on the main data-store. Am I correct?

Answer

Is it really going to reduce the memory footprint? Because, each JVM process anyway going to create some domain objects.

Yes, it is. First of all, you could use flyweight pattern to avoid actual serialization/deserialization and access off-heap memory directly, see the example in the Chronicle Map tutorial and Chronicle Values.

But even if you find this impossible or too complicated to implement, you could reuse objects when deserializing objects to access them within JVM, via map.getUsing(), or better inside context sections where object reuse is automatic (if value type serializers indeed allow reusing objects).

But even if you cannot reuse value objects, e. g. because they are immutable like String, it is likely that objects created when you deserialize off-heap memory will be short-living and will not be promoted from young generation, and GC will collect them efficiently and you will effectively reuse memory. While if you have per-JVM maps, each copy permanently resides memory which is not reused.

So in any case, having a Chronicle Map shared between JVMs is a good thing for overall memory usage in the machine and consequently the performance.

Is it possible to get notification on each JVM process whenever an entry is added or deleted from the datastore?

It is not possible out of the box in the open source version. It is possible to order such a feature. The same comes to remoteOperations that you have mentioned, this feature is not longer in the open source support. However remoteOperations is not exactly what you need for your case.

EDIT Note that depending on your throughput requirements and update frequency, you could implement this yourself relatively easily by copying udpates that you make to Chronicle Map to multi-cast IPC like Chronicle Queue or Aeron IPC, and then reading them in other JVMs. You could copy only the updated keys, without values.

This solution might be worse (or even non-functioning) than what Chronicle Map could provide "natively", if your Map update frequency is so high (100s of thousands of updates per second and above) that you cannot really process all updates and want to naturally throttle events by forgetting stale updates to some keys when newer updates to the same key has already been made. But in your case this might be not needed at all.

MapMethods and remoteOperations came closest to it and maybe, the way to go. I want to know what's the right way to get the intended functionality or am I way off.

MapMethods aims a narrow thing -- overriding default impelementations of Map's methods using Chroncile Map's contexts and lower level operations. It has nothing in common with remote communication, events and listening. You can override MapMethods.put() e. g. to add some logging or event notification, but you could do this by logging or notification after each call to chronicleMap.put() with almost the same results.

RemoteOperations allow to re-define the action that should happen when an entry is put/removed/updated on some node and this event arrives to another node. By default the event is re-played on the reciever node, if it's timestamp is later than the timestamp of the last update for the entry with the event's key in the reciever node, i. e. "last write wins". But it could be overriden to define different strategies, i. e. if the value contains some count, the value with the highest count is written.