HoosierPhysics HoosierPhysics - 9 days ago 6
C Question

Simple MPI_Gather test with memcpy error

I am learning MPI, and trying to create examples of some of the functions. I've gotten several to work, but I am having issues with MPI_Gather. I had a much more complex fitting test, but I trimmed it down to the most simple code. I am still, however, getting the following error:

root@master:/home/sgeadmin# mpirun ./expfitTest5
Assertion failed in file src/mpid/ch3/src/ch3u_request.c at line 584: FALSE
memcpy argument memory ranges overlap, dst_=0x1187e30 src_=0x1187e40 len_=400

internal ABORT - process 0


I am running one master instance and two node instances through AWS EC2. I have all the appropriate libraries installed, as I've gotten other MPI examples to work. My program is:

int main()
{
int world_size, world_rank;
int nFits = 100;
double arrCount[100];
double *rBuf = NULL;

MPI_Init(NULL,NULL);
MPI_Comm_size(MPI_COMM_WORLD, &world_size);
MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);

assert(world_size!=1);
int nElements = nFits/(world_size-1);

if(world_rank>0){
for(int k = 0; k < nElements; k++)
{
arrCount[k] = k;
}}

MPI_Barrier(MPI_COMM_WORLD);

if(world_rank==0)
{
rBuf = (double*) malloc( nFits*sizeof(double));
}

MPI_Gather(arrCount, nElements, MPI_DOUBLE, rBuf, nElements, MPI_DOUBLE, 0, MPI_COMM_WORLD);

if(world_rank==0){
for(int i = 0; i < nFits; i++)
{
cout<<rBuf[i]<<"\n";
}}

MPI_Finalize();

exit(0);
}


Is there something I am not understanding in malloc or MPI_Gather? I've compared my code to other samples, and can't find any differences.

Answer

The root process in a gather operation does participate in the operation. I.e. it sends data to it's own receive buffer. That also means you must allocate memory for it's part in the receive buffer.

Now you could use MPI_Gatherv and specify a recvcounts[0]/sendcount at root of 0 to follow your example closely. But usually you would prefer to write an MPI application in a way that the root participates equally in the operation, i.e. int nElements = nFits/world_size.

Comments