0x56794E 0x56794E - 1 month ago 7
C++ Question

MPI_Isend: how to keep data safe until they're received?

I'm to send many messages using

MPI_Isend
, but I'm not sure how to ensure my data are safe until they are received, and at the same time, do not use up all available mem.

Currently, I'm doing something like this:

vector<int*> outbox;
vector<MPI_Request> stats;

void run()
{
while (!done())
{
//some magic
//...

sendMsg(rand(), getRecipRank());
}
}

void sendMsg(int n, int recipRank)
{
//Need to gen n random integers and send it to proc with rank recipRank
//Can't block until the numbers are reveived.
//So how do I make sure these numbers are safe until then?


int* data = (int*)malloc(sizeof(int) * n);
//fill the data array with random numbers here

MPI_Request req;
MPI_Isend(data, n, MPI_INT, recipRank, 0, MPI_COMM_WORLD, &req);

//Keeping the pointer to the array to free it later
outbox.push_back(data); //Does this help keep the data safe until they're received?
stats.push_back(req);
}


Then I have another function that occasionally goes through the
stats
vector to check the status of the send. If it's complete, then the function frees both the request and corresponding array in
outbox
.

I've tested this with a small number of messages, and it seems to work, but I'm not sure if it'll ALWAYS work or I'm just being lucky.

Answer

Looks good! If you allocate memory using malloc nobody will mess with it, but you. Since you don't mess with, it'll be safe.

This is a good pattern to interleave computation and communication by making use of more memory. If you want to put a limit on the memory usage you could impose a maximum length for the vector outbox and start using blocking sends when that length is reached.

Just to clarify: Your data is not "safer", because you push it into the vector outbox, it would be safe even without doing that. You push it into the vector to be able to free it later.