(client) node-js http request <-----------> node-js http listen (server)
^ 100/1000 MBit/s ^
| LAN |
a file in ramdrive a file in ramdrive
C# program requesting C# or Java program
compute on data computing
(client)C# tcp ip <--------------------------------->C#/Java tcp ip (server)
socket 100/1000 Mbit/s LAN socket
in terms of error handling(such as half-open connections, acknowledgement packet management and picking right window/buffer size) and scalability (for 10-100 servers using tree-like connections)?
If node-js layer is to be used just for I/O, would it be better than MPI for a cluster on basic switches/routers(not infiniband for example)?
For now, I'm trying tcp-ip option and already have too many exceptions to handle and far from finished
. There are max connections because of desktop os, the serialization of arrays also need checking against endianness of server and client and many more to counterpoise before sending/computing the data. Also everywhere it says async is best. I just started with a 1 thread for 1 connection version. Threaded version also has persistent local data to hold compute device information and compiled kernels(for opencl).
If I understand correctly, both mpi and node-js need their runtime be installed on servers and clients but tcp-ip doesn't need such thing. But this wouldn't balance other difficulties would it?
Adding ramdrive between node-js and C# came into my mind because it looks easy to access a file from node-js(and C#) with much less things to worry about than a tcp connection. Http listening is also nearly one-liner in node-js when it is just passing data.