Code works without MPI but gets stuck in parallel mode

Palabos General
#1

hi fellow LBers,

So far I used my own methods to get my structures into the palabos lattice, but since they weren’t made for MPI the code crashed sometimes.

Now I changed my input files to look exactly like in the documentation.
So it’s now a simple textfile with only 4 lines:

  1. nx
  2. ny
  3. nz
  4. data (0 1 0 1 1 1 0 0 …)

and the data ends up in an MultiScalarField3D.

Now the structure(s) can be placed inside the lattice with this class:


template <typename T>
class Test3DStructure : public DomainFunctional3D {
  private:
    MultiScalarField3D<double> *test;
    plint nx, ny, nz;
    plint cx0, cy0, cz0;

  public:
    Test3DStructure(MultiScalarField3D<double> *bla, plint cx0_, plint cy0_, plint cz0_, plint nx_, plint ny_, plint nz_);
    virtual bool operator() (plint iX, plint iY, plint iZ) const {
      if (iX >= cx0 and iX <= cx0 + nx -1 and
	iY >= cy0 and iY <= cy0 + ny -1 and
	iZ >= cz0 and iZ <= cz0 + nz - 1) {
	return test->get(iX-cx0, iY-cy0, iZ-cz0);
      }
      else return false;
    }

    virtual Test3DStructure<T>* clone() const {
      return new Test3DStructure<T>(*this);
    }
};

template <typename T>
Test3DStructure<T>::Test3DStructure(MultiScalarField3D<double> *bla, plint cx0_, plint cy0_, plint cz0_, plint nx_, plint ny_, plint nz_)
{
cx0 = cx0_;
cy0 = cy0_;
cz0 = cz0_;
nx = nx_;
ny = ny_;
nz = nz_;
test = bla;
}

so in the main programm the command to place a structure is something like:


defineDynamics(lattice, lattice.getBoundingBox(), new Test3DStructure<T>(test, 20, 20, 0, nx, ny, nz), new BounceBack<T,DESCRIPTOR>);

And now comes the weird part:

I compile with the MPI flags set true.
If I now run the program on a single core, everything seems to be fine (I terminated the calculation after some pictures since it takes forever).

But when i use the mpirun, the programm gets stuck at the defineDynamics-command.

I have absolutely no clue why, any hints are welcome.

#2

I now added mpi::bCast to the constructor, but that still doesn’t fix it.

Edit 2:

The code always stops at the same point, inside the overloaded operator(), but why is still beyond me :frowning:

#3

In case anybody else stumbles upon the same problem, it seems that using ScalarField3D instead of the MultiScalarField3D seems to fix all the problems.