Ned Rockson Ned Rockson - 6 months ago 23
Python Question

Websocket client on linux cuts off response after 8192 bytes

I've created a docker image based on Ubuntu 14.04 which runs a python websocket client to read from a 3rd party service that sends variable length JSON encoded strings down. I find that the service works well until the encoded string is longer than 8192 bytes and then the JSON is malformed, as everything past 8192 bytes has been cut off.

If I use the exact same code on my mac, I see the data come back exactly as expected.

I am 100% confident that this is an issue with my linux configuration but I am not sure how to debug this or move forward. Is this perhaps a buffer issue or something even more insidious? Can you recommend any debugging steps?

Answer

So it turns out the problem came from the provided websocket module from google cloud sdk. It has a bug where after 8192 bytes it will not continue to read from the socket. This can be fixed by supplying the websocket library maintained by Hiroki Ohtani earlier on your PYTHONPATH than the google cloud sdk.