3 ================================================================================
5 ================================================================================
7 .. |eficas_new| image:: images/eficas_new.png
9 .. |eficas_save| image:: images/eficas_save.png
11 .. |eficas_yacs| image:: images/eficas_yacs.png
14 This section presents the usage of the ADAO module in SALOME. It is complemented
15 by advanced usage procedures the section :ref:`section_advanced`, and by
16 examples in the section :ref:`section_examples`.
18 Logical procedure to build an ADAO test case
19 --------------------------------------------
21 The construction of an ADAO case follows a simple approach to define the set of
22 input data, either static or dynamic data, and then generates a complete block
23 diagram used in YACS. Many variations exist for the definition of input data,
24 but the logical sequence remains unchanged.
26 First of all, the user is considered to know the input data needed to set up the
27 data assimilation study. These data can be in SALOME or not.
29 **Basically, the procedure of using ADAO involves the following steps:**
31 #. **Activate the ADAO module and use the editor GUI,**
32 #. **Build and modify the ADAO case and save it,**
33 #. **Export the ADAO case as a YACS scheme,**
34 #. **Modify and supplement the YACS scheme and save it,**
35 #. **Execute the YACS case and obtain the results.**
37 Each step will be detailed in the next section.
39 Detailed procedure to build an ADAO test case
40 ---------------------------------------------
42 Activate the ADAO module and use the editor GUI
43 +++++++++++++++++++++++++++++++++++++++++++++++
45 As always for a module, it has to be activated by choosing the appropriate
46 module button (or menu) in the toolbar of SALOME. If there is no study loaded, a
47 popup appears, allowing to choose between creating a new study, or opening an
51 .. image:: images/adao_activate.png
54 **Activating the module ADAO in SALOME**
56 Choosing the "*New*" button, an embedded case editor EFICAS [#]_ will be opened,
57 along with the standard "*Object browser*". You can then click on the "*New*"
58 button |eficas_new| (or choose the "*New*" entry in the "*ADAO*" main menu) to
59 create a new ADAO case, and you will see:
62 .. image:: images/adao_viewer.png
66 **The EFICAS editor for cases definition in module ADAO**
68 It is a good habit to save the ADAO case now, by pushing the "*Save*" button
69 |eficas_save| or by choosing the "*Save/Save as*" entry in the "*ADAO*" menu.
70 You will be prompted for a location in your file tree and a name, that will be
71 completed by a "*.comm*" extension used for JDC EFICAS files.
73 Build and modify the ADAO case and save it
74 ++++++++++++++++++++++++++++++++++++++++++
76 To build a case using EFICAS, you have to go through a series of steps, by
77 selecting a keyword and then filling in its value.
79 The structured editor indicates hierarchical types, values or keywords allowed.
80 Incomplete or incorrect keywords are identified by a visual error red flag.
81 Possible values are indicated for keywords defined with a limited list of
82 values, and adapted entries are given for the other keywords. All the mandatory
83 command or keyword are already present, and optionnal commands can be added.
85 A new case is set up with the minimal list of commands. No mandatory command can
86 be suppressed, but others can be added as allowed keywords for an
87 "*ASSIMILATION_STUDY*" command. As an example, one can add an
88 "*AlgorithmParameters*" keyword, as described in the last part of the section
89 :ref:`section_examples`.
91 At the end, when all fields or keywords have been correctly defined, each line
92 of the commands tree must have a green flag. This indicates that the whole case
93 is valid and completed.
95 .. _adao_jdcexample00:
96 .. image:: images/adao_jdcexample01.png
100 **Example of a valid ADAO case**
102 Finally, you have to save your ADAO case by pushing the "*Save*" button
103 |eficas_save| or by choosing the "*Save/Save as*" entry in the "*ADAO*" menu.
105 Export the ADAO case as a YACS scheme
106 +++++++++++++++++++++++++++++++++++++
108 When the ADAO case is completed, you have to export it as a YACS scheme [#]_ in
109 order to execute the data assimilation calculation. This can be easily done by
110 using the "*Export to YACS*" button |eficas_yacs|, or equivalently choose the
111 "*Export to YACS*" entry in the "*ADAO*" main menu, or in the contextual case
112 menu in the object browser.
114 .. _adao_exporttoyacs01:
115 .. image:: images/adao_exporttoyacs.png
119 **"Export to YACS" sub-menu to generate the YACS scheme from the ADAO case**
121 This will lead to automatically generate an XML file for the YACS scheme, and
122 open YACS module on this file. The YACS file will be stored in the same
123 directory and with the same name as the ADAO saved case, only changing its
124 extension from "*.comm*" to "*.xml*". *Be careful, if the name already exist, it
125 will overwrite it without prompting for replacing the file*. In the same time,
126 an intermediary python file is also stored in the same place, with a "*.py*"
127 extension replacing the "*.comm*" one [#]_.
129 Modify and supplement the YACS scheme and save it
130 +++++++++++++++++++++++++++++++++++++++++++++++++
132 .. index:: single: Analysis
134 When the YACS scheme is generated and opened in SALOME through the YACS module
135 GUI, you can modify or supplement the scheme like any YACS scheme. It is
136 recommended to save the modified scheme with a new name, in order to preserve in
137 the case you re-export to YACS the ADAO case.
139 The main supplement needed in the YACS scheme is a postprocessing step. The
140 evaluation of the results has to be done in the physical context of the
141 simulation used by the data assimilation procedure.
143 The YACS scheme has an "*algoResults*" output port of the computation bloc,
144 which gives access to a "*pyobj*" containing all the results. These results can
145 be obtained by retrieving the named variables stored along the calculation. The
146 main is the "*Analysis*" one, that can be obtained by the python command (for
147 example in an in-line script node)::
149 Analysis = results.ADD.get("Analysis").valueserie(-1)
151 This is a complex object, similar to a list of values calculated at each step of
152 data assimilation calculation. In order to get the last data assimilation
153 analysis, one can use::
155 Xa = results.ADD.get("Analysis").valueserie(-1)
157 This ``Xa`` is a vector of values, that represents the solution of the data
158 assimilation evaluation problem, noted as :math:`\mathbf{x}^a` in the section
159 :ref:`section_theory`.
161 Such command can be used to print results, or to convert these ones to
162 structures that can be used in the native or external SALOME postprocessing. A
163 simple example is given in the section :ref:`section_examples`.
165 Execute the YACS case and obtain the results
166 ++++++++++++++++++++++++++++++++++++++++++++
168 .. index:: single: Analysis
169 .. index:: single: Innovation
170 .. index:: single: APosterioriCovariance
171 .. index:: single: OMB
172 .. index:: single: BMA
173 .. index:: single: OMA
174 .. index:: single: CostFunctionJ
175 .. index:: single: CostFunctionJo
176 .. index:: single: CostFunctionJb
178 The YACS scheme is now complete and can be executed. Parametrisation and
179 execution of a YACS case is fully compliant with the standard way to deal with a
180 YACS scheme, and is described in the *YACS module User's Guide*.
182 Results can be obtained, through the "*algoResults*" output port, using YACS
183 nodes to retrieve all the informations in the "*pyobj*" object, to transform
184 them, to convert them, to save part of them, etc.
186 The data assimilation results and complementary calculations can be retrieved
187 using the "*get*" method af the "*algoResults.ADD*" object. This method pick the
188 different output variables identified by their name. Indicating in parenthesis
189 their availability as automatic (for every algorithm) or optional (depending on
190 the algorithm), and their notation coming from section :ref:`section_theory`,
191 the main available output variables are the following:
193 #. "Analysis" (automatic): the control state evaluated by the data assimilation
194 procedure, noted as :math:`\mathbf{x}^a`.
195 #. "Innovation" (automatic): the difference between the observations and the
196 control state transformed by the observation operator, noted as
197 :math:`\mathbf{y}^o - \mathbf{H}\mathbf{x}^b`.
198 #. "APosterioriCovariance" (optional): the covariance matrix of the *a
199 posteriori* analysis errors, noted as :math:`\mathbf{A}`.
200 #. "OMB" (optional): the difference between the observations and the
201 background, similar to the innovation.
202 #. "BMA" (optional): the difference between the background and the analysis,
203 noted as :math:`\mathbf{x}^b - \mathbf{x}^a`.
204 #. "OMA" (optional): the difference between the observations and the analysis,
205 noted as :math:`\mathbf{y}^o - \mathbf{H}\mathbf{x}^a`.
206 #. "CostFunctionJ" (optional): the minimisation function, noted as :math:`J`.
207 #. "CostFunctionJo" (optional): the observation part of the minimisation
208 function, noted as :math:`J^o`.
209 #. "CostFunctionJb" (optional): the background part of the minimisation
210 function, noted as :math:`J^b`.
212 Input variables are also available as output in order to gather all the
213 information at the end of the procedure.
215 All the variables are list of typed values, each item of the list
216 corresponding to the value of the variable at a time step or an iteration step
217 in the data assimilation optimization procedure. The variable value at a given
218 "*i*" step can be obtained by the method "*valueserie(i)*". The last one
219 (consisting in the solution of the evaluation problem) can be obtained using the
220 step "*-1*" as in a standard list.
222 Reference description of the commands and keywords available through the GUI
223 -----------------------------------------------------------------------------
225 Each command or keyword to be defined through the ADAO GUI has some properties.
226 The first property is to be a required command, an optional command or a keyword
227 describing a type of input. The second property is to be an "open" variable with
228 a fixed type but with any value allowed by the type, or a "restricted" variable,
229 limited to some specified values. The mathematical notations used afterwards are
230 explained in the section :ref:`section_theory`.
232 List of possible input types
233 ++++++++++++++++++++++++++++
235 .. index:: single: Dict
236 .. index:: single: Function
237 .. index:: single: Matrix
238 .. index:: single: String
239 .. index:: single: Script
240 .. index:: single: Vector
242 The different type-style commands are:
245 *Type of an input*. This indicates a variable that has to be filled by a
246 dictionary, usually given as a script.
249 *Type of an input*. This indicates a variable that has to be filled by a
250 function, usually given as a script.
253 *Type of an input*. This indicates a variable that has to be filled by a
254 matrix, usually given either as a string or as a script.
257 *Type of an input*. This indicates a string, such as a name or a literal
258 representation of a matrix or vector, such as "1 2 ; 3 4".
261 *Type of an input*. This indicates a script given as an external file.
264 *Type of an input*. This indicates a variable that has to be filled by a
265 vector, usually given either as a string or as a script.
270 .. index:: single: ASSIMILATION_STUDY
271 .. index:: single: Algorithm
272 .. index:: single: AlgorithmParameters
273 .. index:: single: Background
274 .. index:: single: BackgroundError
275 .. index:: single: Debug
276 .. index:: single: InputVariables
277 .. index:: single: Observation
278 .. index:: single: ObservationError
279 .. index:: single: ObservationOperator
280 .. index:: single: Observers
281 .. index:: single: OutputVariables
282 .. index:: single: Study_name
283 .. index:: single: Study_repertory
284 .. index:: single: UserDataInit
285 .. index:: single: UserPostAnalysis
287 The different commands are the following:
290 *Required command*. This is the general command describing an ADAO case. It
291 hierarchicaly contains all the other commands.
294 *Required command*. This is a string to indicates the data assimilation
295 algorithm chosen. The choices are limited and available through the GUI.
296 There exists for example: "3DVAR", "Blue"... See below the list of
297 algorithms and associated parameters.
299 :AlgorithmParameters:
300 *Optional command*. This command allows to add some optional parameters to
301 control the data assimilation algorithm calculation. It is defined as a
302 "*Dict*" type object. See below the list of algorithms and associated
306 *Required command*. This indicates the backgroud vector used for data
307 assimilation, previously noted as :math:`\mathbf{x}^b`. It is defined as a
308 "*Vector*" type object, that is, given either as a string or as a script.
311 *Required command*. This indicates the backgroud error covariance matrix,
312 previously noted as :math:`\mathbf{B}`.It is defined as a "*Matrix*" type
313 object, that is, given either as a string or as a script.
316 *Required command*. This let choose the level of trace and intermediary
317 debug informations. The choices are limited between 0 (for False) and 1 (for
318 True) and available through the GUI.
321 *Optional command*. This command allows to indicates the name and size of
322 physical variables that are bundled together in the control vector. This
323 information is dedicated to data processed inside of data assimilation
327 *Required command*. This indicates the observation vector used for data
328 assimilation, previously noted as :math:`\mathbf{y}^o`. It is defined as a
329 "*Vector*" type object, that is, given either as a string or as a script.
332 *Required command*. This indicates the observation error covariance matrix,
333 previously noted as :math:`\mathbf{R}`.It is defined as a "*Matrix*" type
334 object, that is, given either as a string or as a script.
336 :ObservationOperator:
337 *Required command*. This indicates the observation operator, previously
338 noted :math:`H`, which transforms the input parameters :math:`\mathbf{x}`
339 to results :math:`\mathbf{y}` to be compared to observations
340 :math:`\mathbf{y}^o`.
343 *Optional command*. This command allows to set internal observers, that are
344 functions linked with a particular variable, which will be executed each
345 time this variable is modified. It is a convenient way to monitor interest
346 variables during the data assimilation process, by printing or plotting it,
350 *Optional command*. This command allows to indicates the name and size of
351 physical variables that are bundled together in the output observation
352 vector. This information is dedicated to data processed inside of data
353 assimilation algorithm.
356 *Required command*. This is an open string to describe the study by a name
360 *Optional command*. If available, this repertory is used to find all the
361 script files that can be used to define some other commands by scripts.
364 *Optional command*. This commands allows to initialise some parameters or
365 data automatically before data assimilation algorithm processing.
368 *Optional command*. This commands allows to process some parameters or data
369 automatically after data assimilation algorithm processing. It is defined as
370 a script or a string, allowing to put simple code directly inside the ADAO
373 .. _subsection_algo_options:
375 List of possible options for the algorithms
376 +++++++++++++++++++++++++++++++++++++++++++
378 .. index:: single: Blue
379 .. index:: single: LinearLeastSquares
380 .. index:: single: 3DVAR
381 .. index:: single: NonLinearLeastSquares
382 .. index:: single: EnsembleBlue
383 .. index:: single: QuantileRegression
385 .. index:: single: AlgorithmParameters
386 .. index:: single: Minimizer
387 .. index:: single: Bounds
388 .. index:: single: MaximumNumberOfSteps
389 .. index:: single: CalculateAPosterioriCovariance
390 .. index:: single: CostDecrementTolerance
391 .. index:: single: ProjectedGradientTolerance
392 .. index:: single: GradientNormTolerance
393 .. index:: single: SetSeed
394 .. index:: single: Quantile
396 Each algorithm can be controled using some generic or specific options given
397 throught the "*AlgorithmParameters*" optional command, as follows::
399 AlgorithmParameters = {
401 "MaximumNumberOfSteps" : 10,
404 This section describes the available options by algorithm. If an option is
405 specified for an algorithm that doesn't support it, the option is simply left
410 :CalculateAPosterioriCovariance:
411 This boolean key allows to enable the calculation and the storage of the
412 covariance matrix of a posteriori anlysis errors. Be careful, this is a
413 numericaly costly step. The default is "False".
415 :"LinearLeastSquares":
421 This key allows to choose the optimization minimizer. The default choice
422 is "LBFGSB", and the possible ones are "LBFGSB" (nonlinear constrained
423 minimizer, see [Byrd95] and [Zhu97]), "TNC" (nonlinear constrained
424 minimizer), "CG" (nonlinear unconstrained minimizer), "BFGS" (nonlinear
425 unconstrained minimizer), "NCG" (Newton CG minimizer).
428 This key allows to define upper and lower bounds for every control
429 variable being optimized. Bounds can be given by a list of list of pairs
430 of lower/upper bounds for each variable, with possibly ``None`` every time
431 there is no bound. The bounds can always be specified, but they are taken
432 into account only by the constrained minimizers.
434 :MaximumNumberOfSteps:
435 This key indicates the maximum number of iterations allowed for iterative
436 optimization. The default is 15000, which very similar to no limit on
437 iterations. It is then recommended to adapt this parameter to the needs on
438 real problems. For some minimizers, the effective stopping step can be
439 slightly different due to algorihtm internal control requirements.
441 :CalculateAPosterioriCovariance:
442 This boolean key allows to enable the calculation and the storage of the
443 covariance matrix of a posteriori anlysis errors. Be careful, this is a
444 numericaly costly step. The default is "False".
446 :CostDecrementTolerance:
447 This key indicates a limit value, leading to stop successfully the
448 iterative optimization process when the cost function decreases less than
449 this tolerance at the last step. The default is 10e-7, and it is
450 recommended to adapt it the needs on real problems.
452 :ProjectedGradientTolerance:
453 This key indicates a limit value, leading to stop successfully the
454 iterative optimization process when all the components of the projected
455 gradient are under this limit. It is only used for constrained algorithms.
456 The default is -1, that is the internal default of each algorithm, and it
457 is not recommended to change it.
459 :GradientNormTolerance:
460 This key indicates a limit value, leading to stop successfully the
461 iterative optimization process when the norm of the gradient is under this
462 limit. It is only used for non-constrained algorithms. The default is
463 10e-5 and it is not recommended to change it.
465 :"NonLinearLeastSquares":
468 This key allows to choose the optimization minimizer. The default choice
469 is "LBFGSB", and the possible ones are "LBFGSB" (nonlinear constrained
470 minimizer, see [Byrd95] and [Zhu97]), "TNC" (nonlinear constrained
471 minimizer), "CG" (nonlinear unconstrained minimizer), "BFGS" (nonlinear
472 unconstrained minimizer), "NCG" (Newton CG minimizer).
475 This key allows to define upper and lower bounds for every control
476 variable being optimized. Bounds can be given by a list of list of pairs
477 of lower/upper bounds for each variable, with possibly ``None`` every time
478 there is no bound. The bounds can always be specified, but they are taken
479 into account only by the constrained minimizers.
481 :MaximumNumberOfSteps:
482 This key indicates the maximum number of iterations allowed for iterative
483 optimization. The default is 15000, which very similar to no limit on
484 iterations. It is then recommended to adapt this parameter to the needs on
485 real problems. For some minimizers, the effective stopping step can be
486 slightly different due to algorihtm internal control requirements.
488 :CostDecrementTolerance:
489 This key indicates a limit value, leading to stop successfully the
490 iterative optimization process when the cost function decreases less than
491 this tolerance at the last step. The default is 10e-7, and it is
492 recommended to adapt it the needs on real problems.
494 :ProjectedGradientTolerance:
495 This key indicates a limit value, leading to stop successfully the
496 iterative optimization process when all the components of the projected
497 gradient are under this limit. It is only used for constrained algorithms.
498 The default is -1, that is the internal default of each algorithm, and it
499 is not recommended to change it.
501 :GradientNormTolerance:
502 This key indicates a limit value, leading to stop successfully the
503 iterative optimization process when the norm of the gradient is under this
504 limit. It is only used for non-constrained algorithms. The default is
505 10e-5 and it is not recommended to change it.
510 This key allow to give an integer in order to fix the seed of the random
511 generator used to generate the ensemble. A convenient value is for example
512 1000. By default, the seed is left uninitialized, and so use the default
513 initialization from the computer.
515 :"QuantileRegression":
518 This key allows to define the real value of the desired quantile, between
519 0 and 1. The default is 0.5, corresponding to the median.
522 This key allows to choose the optimization minimizer. The default choice
523 and only available choice is "MMQR" (Majorize-Minimize for Quantile
526 :MaximumNumberOfSteps:
527 This key indicates the maximum number of iterations allowed for iterative
528 optimization. The default is 15000, which very similar to no limit on
529 iterations. It is then recommended to adapt this parameter to the needs on
532 :CostDecrementTolerance:
533 This key indicates a limit value, leading to stop successfully the
534 iterative optimization process when the cost function or the surrogate
535 decreases less than this tolerance at the last step. The default is 10e-6,
536 and it is recommended to adapt it the needs on real problems.
538 Examples of using these commands are available in the section
539 :ref:`section_examples` and in example files installed with ADAO module.
541 .. [#] For more information on EFICAS, see the *EFICAS module* available in SALOME GUI.
543 .. [#] For more information on YACS, see the *YACS module User's Guide* available in the main "*Help*" menu of SALOME GUI.
545 .. [#] This intermediary python file can be safely removed after YACS export, but can also be used as described in the section :ref:`section_advanced`.