From ad056a114a447eb2cafb9ee319878688f0e5cba2 Mon Sep 17 00:00:00 2001 From: ageay Date: Thu, 31 Mar 2011 10:38:06 +0000 Subject: [PATCH] Some docs. --- src/ParaMEDMEM/OverlapDEC.cxx | 93 +++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) diff --git a/src/ParaMEDMEM/OverlapDEC.cxx b/src/ParaMEDMEM/OverlapDEC.cxx index 27df459b7..f01889489 100644 --- a/src/ParaMEDMEM/OverlapDEC.cxx +++ b/src/ParaMEDMEM/OverlapDEC.cxx @@ -23,7 +23,100 @@ #include "MPIProcessorGroup.hxx" #include "OverlapElementLocator.hxx" #include "OverlapInterpolationMatrix.hxx" +/*! + \defgroup overlapdec OverlapDEC + The OverlapDEC enables the \ref InterpKerRemapGlobal "conservative remapping" of fields between two parallel codes. This remapping is based on the computation of intersection volumes on a \b same \b processor \b group. On this processor group is defined a field template defined on A and an another on B. The computation is possible for 3D meshes, 2D meshes, 3D-surface meshes, 1D meshes and 2D-curve meshes. Dimensions must be similar for code A and code B. + The main difference with \ref interpkerneldec is that this DEC manage 2 field-templates on each process in processor group A and B called in code source and target. + Furthermore all process in process group cooperates in global interpolation matrix computation. In this sense InterpKernelDEC is a specialization of OverlapDEC. + \section ParaMEDMEMOverlapDECAlgorithmDescription Algorithm Description + + Let's consider the following use case that is ran in ParaMEDMEMTest_OverlapDEC.cxx to describes the different steps in computation. Processor group contains 3 processors. + \anchor ParaMEDMEMOverlapDECImgTest1 + \image html OverlapDEC1.png "Example showing the use case to explain the different steps." + + \subsection ParaMEDMEMOverlapDECAlgoStep1 Step 1 : Bounding box exchange and global interaction between procs computation. + + In order to reduce as fas as possible number of exchange among processors every procs computes a bounding box for A and B. Then a AllToAll is performed so that + every procs can compute the \b global interactions between procs. + This computation leads every procs to compute the same global TODO list expressed as a list of pair. A pair (x,y) means that proc \b x fieldtemplate A can interacts (because bounding boxes interacts) with fieltemplate B of proc \b y. + In the \ref ParaMEDMEMOverlapDECImgTest1 "example above" this computation leads to the following a \b global TODO list : + + \b (0,0),(0,1),(1,0),(1,2),(2,0),(2,1),(2,2) + + Here pair (0,2) does not appear because bounding box of fieldtemplateA of proc#2 does not interact with proc#0 of fieldtemplate B. + + Stage performed by ParaMEDMEM::OverlapElementLocator::computeBoundingBoxes. + + \subsection ParaMEDMEMOverlapDECAlgoStep2 Step 2 : Computation of local TODO list + + Starting from global interaction previously computed in \ref ParaMEDMEMOverlapDECAlgoStep1 "Step 1", each proc computes the TODO list per proc. + The following rules is chosen : a pair (x,y) can be treated by either proc #x or proc #y, in order to reduce the amount of data transfert accross + procs. The algorithm chosen for loadbalancing is the following : Each proc as an empty \b local TODO list at the beginning. The for each pair (k,m) in + \b global TODO list if proc#k has less temporary local list than proc#m pair (k,m) is added to temparary local TODO list of proc#k. + If proc#m has less temporary local TODO list than proc#k pair (k,m) is added to temparary local TODO list of proc#m. + If proc#k and proc#m have the same amount of temporary local TODO list pair (k,m) is added to temparary local TODO list of proc#k. + + In the \ref ParaMEDMEMOverlapDECImgTest1 "example above" this computation leads to the following a local TODO list : + + - proc#0 : (0,0) + - proc#1 : (0,1),(1,0) + - proc#2 : (1,2),(2,0),(2,1),(2,2) + + Stage performed by ParaMEDMEM::OverlapElementLocator::computeBoundingBoxes too. + + The algorithm described here is not perfect for this use case. It will be enhanced soon, I hope. + + At this stage each proc knows precisely its \b local TODO list (at interpolation sense). The \b local TODO list of other procs than local + is kept for future computation. + + \subsection ParaMEDMEMOverlapDECAlgoStep3 Step 3 : Matrix echange between procs + + At this step knows its \b local TODO list, the aim now is to exchange field-templates between procs. Each proc computes knowing TODO list per + proc computed in \ref ParaMEDMEMOverlapDECAlgoStep2 "Step 2" the exchange TODO list : + + In the \ref ParaMEDMEMOverlapDECImgTest1 "example above" the exchange TODO list gives the following results : + + Sending TODO list per proc : + + - proc #0 : Send fieldtemplate A to Proc#1, Send fieldtemplate B to Proc#1, Send fieldtemplate B to Proc#2 + - Proc #1 : Send fieldtemplate A to Proc#2, Send fieldtemplate B to Proc#2 + - Proc #2 : No send. + + Receiving TODO list per proc : + + - proc #0 : No receiving + - proc #1 : receiving fieldtemplate A from Proc#0, receiving fieldtemplate B from Proc#0 + - proc #2 : receiving fieldtemplate B from Proc#0, receiving fieldtemplate A from Proc#1, receiving fieldtemplate B from Proc#1 + + To avoid as far as possible big amount of exchange between procs only relevant part of mesh. For a proc#k sending fieldtemplate A to fieldtemplate B + of proc #m. In this case proc#k computes part of mesh A in boundingbox B of proc#m. It implies that the corresponding cellIds or nodeIds of + corresponding part are sent to proc #m too. + + This step is performed in ParaMEDMEM::OverlapElementLocator::exchangeMeshes method. + + \subsection ParaMEDMEMOverlapDECAlgoStep4 Step 4 : The interpolation matrix computation + + After mesh exchange in \ref ParaMEDMEMOverlapDECAlgoStep3 "Step3" each proc has all information to perform its \b local TODO list computed in + \ref ParaMEDMEMOverlapDECAlgoStep2 "Step2". This step is so potentially CPU costly. That's why the \b local TODO list per proc is expected to + be as well balanced as possible. + + The interpolation is performed as \ref ParaMEDMEM::MEDCouplingRemapper "Remapper" does. + + This operation is performed by OverlapInterpolationMatrix::addContribution method. + + \subsection ParaMEDMEMOverlapDECAlgoStep5 Step 5 : Global matrix construction. + + After having performed the TODO list at the end of \ref ParaMEDMEMOverlapDECAlgoStep4 "Step4" it is needed to assemble the final matrix. + + The final aim is to have a distributed matrix \f$ M_k \f$ on each proc#k. In order to reduce data exchange during matrix product process. + \f$ M_k \f$ is built using sizeof(Proc group) \c std::vector< std::map >. + + For a proc#k, it is necessary to fetch info of all matrix built in \ref ParaMEDMEMOverlapDECAlgoStep4 "Step4" where the first element in pair + is equal to k. + + The method in charge to perform this is : ParaMEDMEM::OverlapMapping::prepare. +*/ namespace ParaMEDMEM { OverlapDEC::OverlapDEC(const std::set& procIds, const MPI_Comm& world_comm):_own_group(true),_interpolation_matrix(0), -- 2.39.2