Wacław Borowiec Wacław Borowiec - 10 months ago 180
Linux Question

Linux UDP max size of receive buffer

What's the maximum size of Linux UDP receive buffer? I thought it's limited only by available RAM, but when I set

5GB for rmem_max:

echo 5000000000 > /proc/sys/net/core/rmem_max

and 4GB for the actual socket buffer (in Erlang):

gen_udp:listen(Port, [{recbuf, 4000000000}])

When I measure the buffer utilization, it shows:

# netstat -u6anp | grep 5050
udp6 1409995136 0 :::5050 :::* 13483/beam.smp

I can't exceed this 1.4GB. For smaller buffer sizes, like e.g. 500MB, actual buffer size matched the configured value. My system is Debian 6.0, the machine has 50GB RAM available.

Answer Source

It seems that there is a limit in linux. I have tried setting rmem_max to 2^32-1 with success.

   root@xxx:/proc/sys/net/core# echo 2147483647 > rmem_max
   root@xxx:/proc/sys/net/core# cat rmem_max

2^32 was too much:

   root@xxx:/proc/sys/net/core# echo 2147483648 > rmem_max
   root@xxx:/proc/sys/net/core# cat rmem_max

Setting to 5000000000 yields:

   root@xxx:/proc/sys/net/core# echo 5000000000 > rmem_max
   root@xxx:/proc/sys/net/core# cat rmem_max

I have tested in python that setting and getting socket receive buffer with

   ss.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, bufferSize)
   print ss.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF)

If 'bufferSize' is less then 1024^3 program prints doubled 'bufferSize', otherwise it falls back to 256.

The value 705032704*2 = 1410065408 is close to the 1409995136 obtained by netstat.