From: André Ribes Date: Wed, 26 Jan 2011 14:26:18 +0000 (+0100) Subject: Nouvelle version de la documentation X-Git-Tag: V6_4_0rc3~97 X-Git-Url: http://git.salome-platform.org/gitweb/?a=commitdiff_plain;h=4960341c4abeee8012ba366c9322906948deb098;p=modules%2Fadao.git Nouvelle version de la documentation --- diff --git a/doc/advanced.rst b/doc/advanced.rst new file mode 100644 index 0000000..5f48310 --- /dev/null +++ b/doc/advanced.rst @@ -0,0 +1,64 @@ +================================================================================ +Advanced usage of ADAO +================================================================================ + +This section presents advanced methods to use the ADAO module, how to get more +information, or how to use it without the graphical user interface (GUI). + +Exporting an ADAO command file (JDC) to YACS in console mode +-------------------------------------------------------------------------------- + +An export command can use the Python file generated by the editor used to build +the ADAO command file (JDC). If the ADAO command file is named "Study1.comm", +then a file named "Study1.py" can be found in the same directory. The complete +procedure is the following: + +#. using the SALOME application including ADAO module, launch SALOME with ``./runAppli -k`` +#. initialise the command line session with: ``./runSession`` +#. execute the export command: ``python ${ADAO_ROOT_DIR}/bin/salome/AdaoYacsSchemaCreator.py `` +#. standard output comes on console, successive executions can be done +#. stop SALOME: ``shutdownSalome.py`` or ``killSalome.py`` +#. exit from the session: ``CTRL+D`` + +Be careful, if the output YACS xml scheme already exists, this command replace +it without asking the user. The command accepts files with or without path +specifications. + +It is not necessary to launch and shut down SALOME each time if the application +is already running. + +Running an ADAO calculation scheme in YACS in console mode +-------------------------------------------------------------------------------- + +This section describes how to execute in console mode a YACS calculation scheme, +obtained using the ADAO "Export to YACS" function. It uses the standard YACS +console mode, wich will be briefly recall here (see YACS documentation for more +information). + +The way to do that is as follows: + +#. using the SALOME application including ADAO module, launch SALOME with ``./runAppli -k`` +#. initialise the command line session with: ``./runSession`` +#. change to the YACS calculation scheme directory to be executed +#. execute the YACS supervisor: ``driver `` +#. standard output comes on console, successive executions can be done +#. stop SALOME: ``shutdownSalome.py`` or ``killSalome.py`` +#. exit from the session: ``CTRL+D`` + +It is not necessary to launch and shut down SALOME each time if the application +is already running. + +Getting more information when running an assimilation calculation +-------------------------------------------------------------------------------- + +--TODO-- + +(Use the logging functionality of the library) + + +Complex time or step treatments during an assimilation calculation +-------------------------------------------------------------------------------- + +--TODO-- + +(Scheduling input data) diff --git a/doc/conf.py b/doc/conf.py index a2cfb7e..dc6a04c 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -38,7 +38,7 @@ master_doc = 'index' # General information about the project. project = u'ADAO' -copyright = u'2010, EDF R&D, Jean-Philippe Argaud, André Ribes' +copyright = u'2008-2011, EDF R&D, J.-P. Argaud, A. Ribes' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the diff --git a/doc/examples.rst b/doc/examples.rst new file mode 100644 index 0000000..e040bd6 --- /dev/null +++ b/doc/examples.rst @@ -0,0 +1,288 @@ +================================================================================ +Examples on using the ADAO module +================================================================================ + +This section presents some examples on using the ADAO module in SALOME. + +Building a simple estimation case with explicit data definition +-------------------------------------------------------------------------------- + +This simple example is a demonstration one, and describes how to set a BLUE +estimation framework in order to get *weighted least square estimated state* of +a system from an observation of the state and from an *a priori* knowledge (or +background) of this state. In other words, we look for the weighted middle +between the observation and the background vectors. All the numerical values of +this example are arbitrary. + +Experimental set up ++++++++++++++++++++ + +We choose to operate in a 3-dimensional space. 3D is chosen in order to restrict +the size of numerical object to explicitly enter by the user, but the problem is +not dependant of the dimension and can be set in dimension 1000... The observed +state is of value 1 in each direction, so: + + ``Yo = [1 1 1]`` + +The background state, wich represent some *a priori* knowledge or a +regularization, is of value of 0 in each direction, which is: + + ``Xb = [0 0 0]`` + +Data assimilation requires information on errors covariances for observation and +background variables. We choose here to have uncorrelated errors (that is, +diagonal matrices) and to have the same variance of 1 for all variables (that +is, identity matrices. We get: + + ``B = R = [1 0 0 ; 0 1 0 ; 0 0 1]`` + +Last, we need an observation operator to convert the background value in the +space of observation value. Here, because the space dimensions are the same, we +can choose the identity as the observation operator: + + ``H = [1 0 0 ; 0 1 0 ; 0 0 1]`` + +With such choices, the Best Linear Unbiased Estimator (BLUE) will be the +average vector between ``Yo`` and ``Xb``, named the *analysis* and denoted by +``Xa``: + + ``Xa = [0.5 0.5 0.5]`` + +As en extension of this example, one can change the variances for ``B`` or ``R`` +independantly, and the analysis will move to ``Yo`` or ``Xb`` in inverse +proportion of the variances in ``B`` and ``R``. It is also equivalent to serach +for the analysis thought a BLUE algorithm or a 3DVAR one. + +Using the GUI to build the ADAO case +++++++++++++++++++++++++++++++++++++ + +First, you have to activate the ADAO module by choosing the appropriate module +button or menu of SALOME, and you will see: + + .. _adao_activate: + .. image:: images/adao_activate.png + :align: center + .. centered:: + **Activating the module ADAO in SALOME** + +.. |eficas_new| image:: images/eficas_new.png + :align: middle + +Choose the "*New*" button in this window. You will directly get the EFICAS +interface for variables definition, along with the "*Object browser*". You can +then click on the "*New*" button |eficas_new| to create a new ADAO case, and you +will see: + + .. _adao_viewer: + .. image:: images/adao_viewer.png + :align: center + :width: 100% + .. centered:: + **The EFICAS viewer for cases definition in module ADAO** + +Then fill in the variables to build the ADAO case by using the experimental set +up described above. All the technical information given above will be directly +inserted in the ADAO case definition, by using the *String* type for all the +variables. When the case definition is ready, save it to a "*JDC (\*.comm)*" +native file somewhere in your path. Remember that other files will be also +created near this first one, so it is better to make a specific directory for +your case, and to save the file inside. The name of the file will appear in the +"*Object browser*" window, under the "*ADAO*" menu. The final case definition +looks like this: + + .. _adao_jdcexample01: + .. image:: images/adao_jdcexample01.png + :align: center + :width: 100% + .. centered:: + **Definition of the experimental set up chosen for the ADAO case** + +To go further, we need now to generate the YACS scheme from the ADAO case +definition. In order to do that, right click on the name of the file case in the +"*Object browser*" window, and choose the "*Export to YACS*" submenu as below: + + .. _adao_exporttoyacs: + .. image:: images/adao_exporttoyacs.png + :align: center + :scale: 75% + .. centered:: + **"*Export to YACS*" submenu to generate the YACS scheme from the ADAO case** + +This command will generate the YACS scheme, activate YACS module in SALOME, and +open the new scheme in the YACS module GUI [#]. After reordering the nodes by +using the "*arrange local node*" submenu of the YACS graphical view of the +scheme, you get the following representation of the generated ADAO scheme: + + .. _yacs_generatedscheme: + .. image:: images/yacs_generatedscheme.png + :align: center + :width: 100% + .. centered:: + **YACS generated scheme from the ADAO case** + +After that point, all the modifications, executions and post-processing of the +data assimilation scheme will be done in YACS. In order to check the result in a +simple way, we create here a new YACS node by using the "*inline script node*" +submenu of the YACS graphical view, and we name it "*PostProcessing*". + +This script will retrieve the data assimilation analysis from the +"*algoResults*" output port of the computation bloc (which gives access to a +SALOME Python Object), and will print it on the standard output. + +To obtain this, the inline script node need to have an input port of type +"*pyobj*" named "*results*", that have to be linked graphically to the +"*algoResults*" output port of the computation bloc. Then the code to fill in +the script node is:: + + Xa = results.ADD.get("Analysis").valueserie(-1) + + print + print "Analysis =",Xa + print + +The augmented YACS scheme can be saved (overwriting the generated scheme if the +simple "*Save*" command or button is used, or with a new name). Then, +classically in YACS, it have to be prepared for run, and then executed. After +completion, the printing on standard output is available in the "*YACS Container +Log*", obtained throught the right click menu of the "*proc*" window in the YACS +scheme as shown below: + + .. _yacs_containerlog: + .. image:: images/yacs_containerlog.png + :align: center + :width: 100% + .. centered:: + **YACS menu for Container Log, and dialog window showing the log** + +We verify that the result is correct by checking that the log contains the +following line:: + + Analysis = [0.5, 0.5, 0.5] + +as shown in the image above. + +As a simple extension of this example, one can notice that the same problem +solved with a 3DVAR algorithm gives the same result. This algorithm can be +chosen at the ADAO case building step, before entering in YACS step. The +ADAO 3DVAR case will look completly similar to the BLUE algoritmic case, as +shown by the following figure: + + .. _adao_jdcexample01: + .. image:: images/adao_jdcexample02.png + :align: center + :width: 100% + .. centered:: + **Defining an ADAO 3DVAR case looks completly similar to a BLUE case** + +There is only one keyword changing, with "*ThreeDVAR*" instead of "*Blue*". + +Building a simple estimation case with external data definition by functions +-------------------------------------------------------------------------------- + +It is useful to get parts or all of the data from external definition, using +Python script files to provide access to the data. As an example, we build here +an ADAO case representing the same experimental set up as in the above example +`Building a simple estimation case with explicit data definition`_, but using +data form a single one external Python script file. + +First, we write the following script file, using conventional names for the +desired variables. Here, all the input variables are defined in the script, but +the user can choose to split the file in several ones, or to mix explicit data +definition in the ADAO GUI and implicit data definition by external files. This +script looks like:: + + #-*-coding:iso-8859-1-*- + import numpy + # + # Definition of the Background as a vector + # ---------------------------------------- + Background = [0, 0, 0] + # + # Definition of the Observation as a vector + # ----------------------------------------- + Observation = "1 1 1" + # + # Definition of the Background Error covariance as a matrix + # --------------------------------------------------------- + BackgroundError = numpy.array([[1., 0., 0.], [0., 1., 0.], [0., 0., 1.]]) + # + # Definition of the Observation Error covariance as a matrix + # ---------------------------------------------------------- + ObservationError = numpy.matrix("1 0 0 ; 0 1 0 ; 0 0 1") + # + # Definition of the Observation Operator as a matrix + # -------------------------------------------------- + ObservationOperator = numpy.identity(3) + +The names of the Python variables above are mandatory, in order to define the +right variables, but the Python script can be bigger and define classes, +functions, etc. with other names. It shows different ways to define arrays and +matrices, using list, string (as in Numpy or Octave), Numpy array type or Numpy +matrix type, and Numpy special functions. All of these syntaxes are valid. + +After saving this script somewhere in your path (named here +"*test003_ADAO_example_scripts.py*" for the example), we use the GUI to build +the ADAO case. The procedure to fill in the case is similar except that, instead +of selecting the "*String*" option for the "*FROM*" keyword, we select the +"*Script*" one. This leads to a "*SCRIPT_DATA/SCRIPT_FILE*" entry in the tree, +allowing to choose a file as: + + .. _adao_scriptentry01: + .. image:: images/adao_scriptentry01.png + :align: center + :width: 100% + .. centered:: + **Defining an input value using an external script file** + +Other steps and results are exactly the same as in the `Building a simple +estimation case with explicit data definition`_ previous example. + +In fact, this script methodology allows to retrieve data from inline or previous +calculations, from static files, from database or from stream, all of them +outside of SALOME. It allows also to modify easily some input data, for example +debug purpose or for repetitive process. **But be careful, it is not a "safe" +process, in the sense that erroneous data, or errors in calculations, can be +directly injected into the YACS scheme execution.** + +Adding parameters to control the data assimilation algorithm +-------------------------------------------------------------------------------- + +One can add some optional parameters to control the data assimilation algorithm +calculation. This is done by using the "*AlgorithmParameters*" keyword in the +definition of the ADAO case, which is an option of the algorithm. This keyword +requires a Python dictionary containing some key/value pairs. + +For example, with a 3DVAR algorithm, the possible keys are "*Minimizer*", +"*MaximumNumberOfSteps*", "*Bounds*". Bounds can be given by a list of list of +pairs of lower/upper bounds for each variable, with possibly ``None`` every time +there is no bound. If no bounds at all are required on the control variables, +then one can choose the "BFGS" or "CG" minimisation algorithm for the 3DVAR +algorithm. + +This dictionary has to be defined in an external Python script file, using the +mandatory variable name "*AlgorithmParameters*" for the dictionary. All the keys +inside the dictionary are optional, they all have default values, and can exist +without being used. For example:: + + #-*-coding:iso-8859-1-*- + # + AlgorithmParameters = { + "Minimizer" : "CG", # Possible choice : "LBFGSB", "TNC", "CG", "BFGS" + "MaximumNumberOfSteps" : 10, + } + +Then the script can be added to the ADAO case, in a file entry describing the +"*AlgorithmParameters*" keyword, as follows: + + .. _adao_scriptentry02: + .. image:: images/adao_scriptentry02.png + :align: center + :width: 100% + .. centered:: + **Adding parameters to control the algorithm** + +Other steps and results are exactly the same as in the `Building a simple +estimation case with explicit data definition`_ previous example. + +.. [#] For more information on YACS, see the the *YACS User Guide* available in the main "*Help*" menu of SALOME GUI. + diff --git a/doc/images/adao_activate.png b/doc/images/adao_activate.png new file mode 100644 index 0000000..442ced6 Binary files /dev/null and b/doc/images/adao_activate.png differ diff --git a/doc/images/adao_exporttoyacs.png b/doc/images/adao_exporttoyacs.png new file mode 100644 index 0000000..77552f9 Binary files /dev/null and b/doc/images/adao_exporttoyacs.png differ diff --git a/doc/images/adao_jdcexample01.png b/doc/images/adao_jdcexample01.png new file mode 100644 index 0000000..75a367f Binary files /dev/null and b/doc/images/adao_jdcexample01.png differ diff --git a/doc/images/adao_jdcexample02.png b/doc/images/adao_jdcexample02.png new file mode 100644 index 0000000..7e3ddcc Binary files /dev/null and b/doc/images/adao_jdcexample02.png differ diff --git a/doc/images/adao_scriptentry01.png b/doc/images/adao_scriptentry01.png new file mode 100644 index 0000000..18056f5 Binary files /dev/null and b/doc/images/adao_scriptentry01.png differ diff --git a/doc/images/adao_scriptentry02.png b/doc/images/adao_scriptentry02.png new file mode 100644 index 0000000..ca295d3 Binary files /dev/null and b/doc/images/adao_scriptentry02.png differ diff --git a/doc/images/adao_viewer.png b/doc/images/adao_viewer.png new file mode 100644 index 0000000..12824b4 Binary files /dev/null and b/doc/images/adao_viewer.png differ diff --git a/doc/images/eficas_new.png b/doc/images/eficas_new.png new file mode 100644 index 0000000..f0a4f42 Binary files /dev/null and b/doc/images/eficas_new.png differ diff --git a/doc/images/eficas_save.png b/doc/images/eficas_save.png new file mode 100644 index 0000000..8d09d2a Binary files /dev/null and b/doc/images/eficas_save.png differ diff --git a/doc/images/yacs_containerlog.png b/doc/images/yacs_containerlog.png new file mode 100644 index 0000000..ccf0cc6 Binary files /dev/null and b/doc/images/yacs_containerlog.png differ diff --git a/doc/images/yacs_generatedscheme.png b/doc/images/yacs_generatedscheme.png new file mode 100644 index 0000000..a14dbb9 Binary files /dev/null and b/doc/images/yacs_generatedscheme.png differ diff --git a/doc/index.rst b/doc/index.rst index bd3bbbe..6816059 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -1,18 +1,49 @@ -ADAO documentation -================== +================================================================================ +ADAO module documentation +================================================================================ -Contents: +The ADAO module provides **data assimilation** features [#]_ in SALOME context. +It is based on an extensible generic underlying data assimilation library, and +on usage on other SALOME modules, namely YACS and EFICAS. + +Briefly stated, Data Assimilation is a methodological framework to compute the +optimal estimate of the inaccesible true value of a system state over time. It +uses information coming from experimental measurements or observations, and from +numerical *a priori* models, including information about their errors. Parts of +the framework are also known as *parameter estimation*, *inverse problems*, +*bayesian estimation*, *optimal interpolation*, etc. + +The documentation of this module is divided in 4 parts, the first one being an +introduction. The second part describes how to use the module ADAO. The third +part focuses on advanced usages of the module, how to get more information, or +how to use it without the graphical user interface (GUI). The last part gives +examples on ADAO usage. + +Users interested in quick use of the module can jump to the last part, but a +valuable use of the module requires to read and come back regularly the second +part. + +In all this documentation, we use standard notations of data assimilation. +Vectors are written horizontally or vertically without difference. Matrices are +written either normally, either with a condensed notation, consisting in the +use of a "``;``" to separate the rows in a continuous line. .. toctree:: - :maxdepth: 2 + :maxdepth: 2 - intro + intro + using + advanced + examples Indices and tables -================== +================================================================================ * :ref:`genindex` -* :ref:`modindex` * :ref:`search` +.. _ECMWF: http://www.ecmwf.int/newsevents/training/rcourse_notes/DATA_ASSIMILATION/ASSIM_CONCEPTS/Assim_concepts.html +.. _Wikipedia/Data_assimilation: http://en.wikipedia.org/wiki/Data_assimilation + +.. [#] If you are not familiar with data assimilation concepts in general, an introductory on-line training course can be found at ECMWF_, along with other materials coming from geosciences applications. See also for example `Wikipedia/Data_assimilation`_. Note that data assimilation is not restricted to meteorology or geosciences, but is widely used in other scientific domains. diff --git a/doc/intro.rst b/doc/intro.rst index 24bbf18..4b88fee 100644 --- a/doc/intro.rst +++ b/doc/intro.rst @@ -1,4 +1,9 @@ -Introduction à ADAO -+++++++++++++++++++ +================================================================================ +Introduction to ADAO +================================================================================ -todo +The aim of ADAO module is to help using *data assimilation* methodology in +conjunction with other calculation modules in SALOME. + + +--TODO-- diff --git a/doc/ressources/ADAO.png b/doc/ressources/ADAO.png new file mode 100644 index 0000000..ff778d9 Binary files /dev/null and b/doc/ressources/ADAO.png differ diff --git a/doc/ressources/ADAO_small.png b/doc/ressources/ADAO_small.png new file mode 100644 index 0000000..0f7bbbd Binary files /dev/null and b/doc/ressources/ADAO_small.png differ diff --git a/doc/using.rst b/doc/using.rst new file mode 100644 index 0000000..9d8c4a3 --- /dev/null +++ b/doc/using.rst @@ -0,0 +1,133 @@ +================================================================================ +Using the module ADAO +================================================================================ + +This section presents the usage of the ADAO module in SALOME. + +Logical procedure to build an ADAO test case +-------------------------------------------------------------------------------- + +The construction of an ADAO case follows a simple approach to define the set of +input data, either static or dynamic data, and then generates a complete block +diagram used in Y. Many variations exist for the definition of input data, but +the logical sequence remains unchanged. + +First of all, the user is considered to know the input data needed to set up the +data assimilation study. These data can be in SALOME or not. + +**Basically, the procedure involves the following steps:** + +#. **Activate the ADAO module and use the editor GUI,** +#. **Build and modify the ADAO case and save it,** +#. **Export the ADAO case as a YACS scheme,** +#. **Modify and supplement the YACS scheme and save it,** +#. **Execute the YACS case and obtain the results.** + +Each step will be detailed later in the next section. + +Detailed procedure to build an ADAO test case +-------------------------------------------------------------------------------- + +Activate the ADAO module and use the editor GUI +++++++++++++++++++++++++++++++++++++++++ + +As always for a module, it has to be activated by choosing the appropriate +module button (or menu) in the toolbar of SALOME. A popup appears, allowing to +choose between creating a new study, or opening an already existing one: + + .. _adao_activate: + .. image:: images/adao_activate.png + :align: center + .. centered:: + **Activating the module ADAO in SALOME** + +.. |eficas_new| image:: images/eficas_new.png + :align: middle +.. |eficas_save| image:: images/eficas_save.png + :align: middle + +Choosing the "*New*" button, an embedded case editor EFICAS [#]_ will be opened, +along with the standard "*Object browser*". You can then click on the "*New*" +button |eficas_new| (or choose the "*New*" entry in the "*File*" menu) to create +a new ADAO case, and you will see: + + .. _adao_viewer: + .. image:: images/adao_viewer.png + :align: center + :width: 100% + .. centered:: + **The EFICAS editor for cases definition in module ADAO** + +It is a good habit to save the ADAO case now, by pushing the "*Save*" button +|eficas_save| or by choosing the "*Save/Save as*" entry in the "*File*" menu. +You will be prompted for a location in your file tree and a name, that will be +completed by a "*.comm*" extension used for JDC EFICAS files. + +Build and modify the ADAO case and save it +++++++++++++++++++++++++++++++++++++++++ + +To build a case using EFICAS, you have to go through a series of steps for +selecting a keyword and then filling in its value. The structured editor +indicates hierarchical types, values or keywords allowed. Incomplete or +incorrect keywords are identified by a visual error red flag. + + + + + + +At the end, you have to save your ADAO case. + +Export the ADAO case as a YACS scheme +++++++++++++++++++++++++++++++++++++++++ + + + +Modify and supplement the YACS scheme and save it +++++++++++++++++++++++++++++++++++++++++ + + +Execute the YACS case and obtain the results +++++++++++++++++++++++++++++++++++++++++ + + + + + + +Reference description of the commands and keywords available throught the GUI +-------------------------------------------------------------------------------- + +--TODO-- + + +ASSIM_STUDY + +String + +Script + +Vector + +Matrix + +Function + +Dict + +Background + +BackgroundError + +Observation + +ObservationError + +ObservationOperator + +AlgorithmParameters + +Algorithm : "ThreeDVAR", "Blue", "EnsembleBlue", "Kalman"... + +.. [#] For more information on EFICAS, see the the *EFICAS User Guide* available in the main "*Help*" menu of SALOME GUI. +