Obicere Obicere - 11 days ago 8
Java Question

Java Serialization to transfer data between any language

Question:

Instead of writing my own serialization algorithm; would it be possible to just use the built in Java serialization, like I have done below, while still having it work across multiple languages?




Explanation:

How I imagine it working, would be as follows: I start up a process, that will be be a language-specific program - written in that language. So I'd have a CppExecutor.exe file, for example. I would write data to a stream to this program. The program would then do what it needs to do, then return a result.

To do this, I would need to serialize the data in some way. The first thing that came to mind was the basic Java
Serialization
with the use of an
ObjectInputStream
and
ObjectOutputStream
. Most of what I have read has only stated that the Java serialization is Java-to-Java applications.

None of the data will ever need to be stored in a file. The method of transferring these packets would be through a
java.lang.Process
, which I have set up already.

The data will be composed of the following:


  • String
    - Mostly containing information that is displayed to the user.

  • Integer
    - most likely 32-bit. Won't need to deal with times.

  • Float
    - just to handle all floating-point values.

  • Character
    - to ensure proper types are used.

  • Array
    - Composed of any of the elements in this list.



The best way I have worked out how to do this is as follows: I would start with a 4-byte magic number - just to ensure we are working with the correct data. Following, I would have an integer specifying how many elements there are. After that, for each of the elements I would have: a single byte, signifying the data type (of the above), following by any crucial information, e.x:
length
for the
String
and
Array
. Then, the data that follows.




Side-notes:

I would also like to point out that a lot of these calculations will be taking place, where every millisecond could matter. Due to this, a text-based format (such as JSON) may produce far larger operation times. Considering that non of the packets would need to be interpreted by a human, using only bytes wouldn't be an issue.

Answer

I'd recommend Google protobuf: it is binary, stable, proven, and has bindings for all languages you've mentioned. Moreover, it also handles structured data nicely.