I have a Multi-grid application with a moving boundary. The motion of the boundary is computed separately in a non parallel framework, which consumes quite a lot of memory (Every MPI process has to have the whole geometry in memory).
I am now trying to integrate this within the Palabos framework. My issue is that if a point is trying to move out of an atomic block (easy to detect using “contained” function) then I have to re-instantiate the immersed surface completely. Instead, I think it would be more efficient to perform a point to point communication (and thus send the point from a block to an other). Also, I need communications to compute the motion of my boundary, I do it with an AllReduce but this involves exchanging much bigger data arrays than necessary.
In general I am not comfortable (yet) with the idea of using point to point communications within AtomicBlocks especially in a multi grid solution (DataProcessors are integrated in the finestLevel).
I understand that Palabos has some wrappers in MpiManager.h but I am not sure what should or should not be used in my case.
Any tip, or piece of advice would be great !
Thanks a lot