James Franco James Franco - 5 months ago 24
Python Question

python sending TCP/IP data instantly

I am currently sending data over TCP/IP in myserver using something like this

for str in lst:
data = str + "\n"
self._conn.sendall(data)


Now suppose my list has the following two string in it

1-This is statement 1 in list
2-This is statement 2 in list


My client is receiving half of line 2 like this.

This is statement 1 in list
This is


I would like to send line1 and then line 2 in the list individually. I understand that TCP/IP works this way in which it will send the entire data that is available to send. I think I could put a delay in after calling
self._conn.sendall(data)
but i wanted to know what other options I have. I cannot make changes to the receiver of the data and I can only make changes to the sender. So far my only option is adding a delay after each send.

Answer

TCP works with streams of data, not individual packets. It's like reading data from a file. The sender puts data in its send buffer, and TCP can decide for itself when to send it. The timing of the arrival at the receiving application depends on when the data was sent and on (often unpredictable) network conditions.

TCP deliveries can be made more predicable if you use the TCP_NODELAY flag in your socket (something like socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1). This would cause TCP to send out data as soon as it arrives in its buffer. But still, there would be no guarantees as to arrival times. This is why any time based solution would break, at least in some cases.

The solution is to divide the data stream into chunks yourself. There are several ways of doing that. Here are a few:

  1. Use fixed length messages - if all messages have a fixed length, the receiver just has to recv() the right number of bytes, process the message, then wait for the same number of bytes.

  2. Send the length of the message before each message. If you want to send the string "blah", encode it as "0004blah" or something similar. The receiver will (always) read the first four bytes (which are 0004) to figure out the number of remaining bytes to read. It will then read the required number of bytes, process the message, and then wait for the next one. It's a robust solution that's also easy to implement.

  3. Use a delimiter. Lines in text files are divided by newline characters (\n). Similarly, you can add a special delimiter byte (or bytes) between messages. For example, you can define that messages always end with a dollar sign ($). Then all the receiver has to do is read from the socket byte by byte until it receives a dollar sign. If you take this approach, you have to make sure though that the messages themselves don't contain the delimiter. Otherwise, they would be