Jadzia Jadzia - 17 days ago 5
C Question

Sending objects via sockets from Python to C/Fortran

The following codes are snippets of a larger software package. I want to understand how the code manages to send objects like numpy arrays, cells. What I wonder about is that the code does not seem to serialize the objects, but it does work. Why does it work?

Python client code (snippet):

class DriverSocket(socket.socket):

def __init__(self, _socket_interface):
super(DriverSocket,self).__init__(_sock=_socket_interface)


def sendpos(self, pos, cell):
"""Sends the position and cell data to the driver.

Args:
pos: An array containing the atom positions.
cell: A cell object giving the system box.

Raises:
InvalidStatus: Raised if the status is not Ready.
"""

if (self.status & Status.Ready):
try:
self.sendall(Message("posdata"))
self.sendall(cell.h)
self.sendall(cell.ih)
self.sendall(np.int32(len(pos)/3))
self.sendall(pos)
except:
self.poll()
return
else:
raise InvalidStatus("Status in sendpos was " + self.status)


Receiving code in C (snippet):

void open_socket_(int *psockfd, int* inet, int* port, char* host)
/* Opens a socket.

Note that fortran passes an extra argument for the string length, but this is
ignored here for C compatibility.

Args:
psockfd: The id of the socket that will be created.
inet: An integer that determines whether the socket will be an inet or unix
domain socket. Gives unix if 0, inet otherwise.
port: The port number for the socket to be created. Low numbers are often
reserved for important channels, so use of numbers of 4 or more digits is
recommended.
host: The name of the host server.
*/

{
int sockfd, portno, n;
struct hostent *server;

struct sockaddr * psock; int ssock;

if (*inet>0)
{ // creates an internet socket
struct sockaddr_in serv_addr; psock=(struct sockaddr *)&serv_addr; ssock=sizeof(serv_addr);
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd < 0) error("Error opening socket");

server = gethostbyname(host);
if (server == NULL)
{
fprintf(stderr, "Error opening socket: no such host %s \n", host);
exit(-1);
}

bzero((char *) &serv_addr, sizeof(serv_addr));
serv_addr.sin_family = AF_INET;
bcopy((char *)server->h_addr, (char *)&serv_addr.sin_addr.s_addr, server->h_length);
serv_addr.sin_port = htons(*port);
if (connect(sockfd, psock, ssock) < 0) error("Error opening socket: wrong host address, or broken connection");
}
else ...



void readbuffer_(int *psockfd, char *data, int* plen)
/* Reads from a socket.

Args:
psockfd: The id of the socket that will be read from.
data: The storage array for data read from the socket.
plen: The length of the data in bytes.
*/

{
int n, nr;
int sockfd=*psockfd;
int len=*plen;

n = nr = read(sockfd,data,len);

while (nr>0 && n<len )
{ nr=read(sockfd,&data[n],len-n); n+=nr; }

if (n == 0) error("Error reading from socket: server has quit or connection broke");
}


And then there is Fortran code which used the C-socket code

CALL open_socket(socket, inet, port, host)
...
CALL readbuffer(socket, msgbuffer, nat*3*8)


And this receiving code indeed gets a two dimensional arrays and so on. The same works in the opposite direction.

Answer

If you try to send an arbitrary object via send/sendall you will see an exception saying that a bytes-like is expected. So the idea is simple, numpy structures you send provide bytes-like interface, and conversion to bytes leaves just the raw binary data.

The easiest implementation is to inherit from bytes:

class A: pass
sock.sendall(A())  # exception

class B(bytes): pass
sock.sendall(B())  # no exception

However, numpy is a complex framework written in Python, Cython and C. They may use C API to provide the similar functionality.

It's also worth noting that duck-typing doesn't work here as I would expect:

class C:
    def __bytes__(self):
        return bytes()

sock.sendall(C())  # exception