brokenfoot brokenfoot - 5 months ago 18
Linux Question

What is the effect of setting a linux socket - high priority?

From the linux socket manpage:


SO_PRIORITY

Set the protocol-defined priority for all packets to be
sent on this socket. Linux uses this value to order the networking
queues: packets with a higher priority may be processed first
depending on the selected device queueing discipline.


And this is set using:

int optval=7 // valid values are in the range [1,7]
// 1- low priority, 7 - high priority
setsockopt(socket, SOL_SOCKET, SO_PRIORITY, &optval, optlen)


And say, the process has:

a. 10 low priority sockets (priority=4) from
socket_1
-
socket_10
,

b. 1 high priority socket (priority=7) -
socket_11


What will happen in the following scenarios:


  1. send()
    : process sends multiple msgs on
    socket_1
    -
    socket_10
    & on
    socket_11
    , IMO msgs on the
    socket_11
    will be given preference over the msgs being sent over
    socket_1
    -
    socket_10
    .

  2. recv()
    : if there are mutilple msgs being received at all the sockets described above, does
    socket_11
    gets higher priority in reading the messages over
    socket_1
    -
    socket_10
    ?

  3. Is there a way to measure the socket priority from command line using tools such as
    lsof
    ,
    netstat
    etc?


Answer

Every Linux network interface has a qdisc (queuing discipline) attached to it. And the answer to your questions depends on the qdisc configured on the relevant network interface. Some, like pfifo and bfifo, have no concept of priority so if they're used, answer is simple - nothing will happen (well, nothing interesting at least).

However, if used with an appropriate qdisc, such pfifo_fast which is the default qdisc on every Linux machine I ever saw, the priority might have an effect.

This image describes what's going on in a pfifo_fast qdisc:

enter image description here

We see that packets are placed in queues depending on their priority. When time comes for the interface to send the next packet (frame actually, but let's not get into that), it will always choose to send the packet with the highest priority among those waiting in the queues. Other qdiscs have different structures and policies. For example an SFQ qdisc:

enter image description here

With that in mind, let's get back to your questions:

  1. Depending on the qdisc, yes, packets from socket_11 may be sent ahead of packets from other sockets. If pfifo_fast is used, and if socket_11 sends large amounts of traffic that saturate the outbound network interface, the packets from the other sockets might even not be sent at all. However, practically, this is unlikely. It's usually hard to saturate a network interface before saturating some other resource (such as the speed of your Internet connection, or your own CPU) unless of course we're talking about a wireless link. If the network interface is not close to its saturation point, and always available to send packets the queues will (almost) always be empty, so no prioritization will happen. There has to be contention for the prioritization to kick in!

  2. The path packets take from the machine's network interface to the socket is much faster than the network itself. And, as you recall, for prioritization to have any effect, there has to be contention. In a typical scenario, packets that reached as far as your network interface have already passed the bottleneck of their journey across the network. You can of course use an ingress qdisc or other mechanisms to artificially create a bottleneck, and prioritize incoming traffic, but why would you? That only makes sense if you're building a traffic shaper or a similar network device. Plus, since this queuing is a low level mechanism that happens well below the higher level sockets (even before bridging or routing), I doubt that the socket's priority could have any effect on in.

  3. Not that I'm aware of, but I'd be happy to learn if anyone does know such a command. This kernel module comes close, but it doesn't seem to be able to show priority flags, just regular socket options.