Jadzia Jadzia - 17 days ago 5
C Question

Sending objects via sockets from Python to C/Fortran

The following codes are snippets of a larger software package. I want to understand how the code manages to send objects like numpy arrays, cells. What I wonder about is that the code does not seem to serialize the objects, but it does work. Why does it work?

Python client code (snippet):

class DriverSocket(socket.socket):

def __init__(self, _socket_interface):

def sendpos(self, pos, cell):
"""Sends the position and cell data to the driver.

pos: An array containing the atom positions.
cell: A cell object giving the system box.

InvalidStatus: Raised if the status is not Ready.

if (self.status & Status.Ready):
raise InvalidStatus("Status in sendpos was " + self.status)

Receiving code in C (snippet):

void open_socket_(int *psockfd, int* inet, int* port, char* host)
/* Opens a socket.

Note that fortran passes an extra argument for the string length, but this is
ignored here for C compatibility.

psockfd: The id of the socket that will be created.
inet: An integer that determines whether the socket will be an inet or unix
domain socket. Gives unix if 0, inet otherwise.
port: The port number for the socket to be created. Low numbers are often
reserved for important channels, so use of numbers of 4 or more digits is
host: The name of the host server.

int sockfd, portno, n;
struct hostent *server;

struct sockaddr * psock; int ssock;

if (*inet>0)
{ // creates an internet socket
struct sockaddr_in serv_addr; psock=(struct sockaddr *)&serv_addr; ssock=sizeof(serv_addr);
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd < 0) error("Error opening socket");

server = gethostbyname(host);
if (server == NULL)
fprintf(stderr, "Error opening socket: no such host %s \n", host);

bzero((char *) &serv_addr, sizeof(serv_addr));
serv_addr.sin_family = AF_INET;
bcopy((char *)server->h_addr, (char *)&serv_addr.sin_addr.s_addr, server->h_length);
serv_addr.sin_port = htons(*port);
if (connect(sockfd, psock, ssock) < 0) error("Error opening socket: wrong host address, or broken connection");
else ...

void readbuffer_(int *psockfd, char *data, int* plen)
/* Reads from a socket.

psockfd: The id of the socket that will be read from.
data: The storage array for data read from the socket.
plen: The length of the data in bytes.

int n, nr;
int sockfd=*psockfd;
int len=*plen;

n = nr = read(sockfd,data,len);

while (nr>0 && n<len )
{ nr=read(sockfd,&data[n],len-n); n+=nr; }

if (n == 0) error("Error reading from socket: server has quit or connection broke");

And then there is Fortran code which used the C-socket code

CALL open_socket(socket, inet, port, host)
CALL readbuffer(socket, msgbuffer, nat*3*8)

And this receiving code indeed gets a two dimensional arrays and so on. The same works in the opposite direction.


If you try to send an arbitrary object via send/sendall you will see an exception saying that a bytes-like is expected. So the idea is simple, numpy structures you send provide bytes-like interface, and conversion to bytes leaves just the raw binary data.

The easiest implementation is to inherit from bytes:

class A: pass
sock.sendall(A())  # exception

class B(bytes): pass
sock.sendall(B())  # no exception

However, numpy is a complex framework written in Python, Cython and C. They may use C API to provide the similar functionality.

It's also worth noting that duck-typing doesn't work here as I would expect:

class C:
    def __bytes__(self):
        return bytes()

sock.sendall(C())  # exception