Manolete Manolete - 3 months ago 11
C Question

Alternative to MPI_Sendrecv_replace

I am trying to get an alternative code to

if(rank %2==0 && rightNeighbour != MPI_PROC_NULL)
MPI_Sendrecv_replace(&bufferRight[0], len, MPI_DOUBLE, rightNeighbour, 1,
rightNeighbour, 1, MPI_COMM_WORLD, &status);
else if(rank%2 ==1 && leftNeighbour != MPI_PROC_NULL)
MPI_Sendrecv_replace(&bufferLeft[0], len, MPI_DOUBLE, leftNeighbour, 1,
leftNeighbour, 1, MPI_COMM_WORLD, &status);

if (rank % 2 == 1 && rightNeighbour != MPI_PROC_NULL)
MPI_Sendrecv_replace(&bufferRight[0], len, MPI_DOUBLE, rightNeighbour, 1,
rightNeighbour, 1, MPI_COMM_WORLD, &status);
else if (rank % 2 == 0 && leftNeighbour != MPI_PROC_NULL)
MPI_Sendrecv_replace(&bufferLeft[0], len, MPI_DOUBLE, leftNeighbour, 1,
leftNeighbour, 1, MPI_COMM_WORLD, &status);


using
MPI_Send
and
MPI_Recv
but it seems it's deadlocking. Any easy way of doing the same with
MPI_Send
and
MPI_Recv
?

I have tried using

if(rank %2==0 && rightNeighbour != MPI_PROC_NULL){
MPI_Recv(&bufferRight[0], len, MPI_DOUBLE, rightNeighbour, 1,
MPI_COMM_WORLD, &status);

MPI_Send(&bufferRight[0], len, MPI_DOUBLE, rightNeighbour, 1,
MPI_COMM_WORLD);
}

Answer

This code:

if(rank % 2 == 0 && rightNeighbour != MPI_PROC_NULL)
   MPI_Sendrecv_replace(&bufferRight[0], len, MPI_DOUBLE, rightNeighbour, 1, 
                       rightNeighbour, 1, MPI_COMM_WORLD, &status);
else if(rank % 2 == 1 && leftNeighbour != MPI_PROC_NULL)
   MPI_Sendrecv_replace(&bufferLeft[0], len, MPI_DOUBLE, leftNeighbour, 1, 
                        leftNeighbour, 1, MPI_COMM_WORLD, &status);

is equivalent to:

if(rank % 2 == 0)
{
   MPI_Send(&bufferRight[0], len, MPI_DOUBLE, rightNeighbour, 1,
            MPI_COMM_WORLD);
   MPI_Recv(&bufferRight[0], len, MPI_DOUBLE, rightNeighbour, 1,
            MPI_COMM_WORLD, &status);
}
else if(rank % 2 == 1)
{
   double *temp = malloc(len * sizeof(double));
   MPI_Recv(temp, len, MPI_DOUBLE, leftNeighbour, 1,
            MPI_COMM_WORLD, &status);
   MPI_Send(&bufferLeft[0], len, MPI_DOUBLE, leftNeighbour, 1,
            MPI_COMM_WORLD, &status);
   memcpy(&bufferLeft[0], temp, len * sizeof(double));
   free(temp);
}

Note that order of the send and receive calls is reversed in the odd ranks. Also, the receive uses a temporary buffer in order to implement the semantics of MPI_Sendrecv_replace, which guarantees that the data in the buffer is first sent and only then overwritten with the received one.

Note that the check whether a rank is not MPI_PROC_NULL is pointless since a send to/receive from MPI_PROC_NULL is essentially a no-op and will always succeed. One of the key ideas of the semantics of MPI_PROC_NULL is to facilitate the writing of symmetric code that doesn't contain such if's.

Comments