2 Copyright (C) 2008-2015 EDF R&D
4 This file is part of SALOME ADAO module.
6 This library is free software; you can redistribute it and/or
7 modify it under the terms of the GNU Lesser General Public
8 License as published by the Free Software Foundation; either
9 version 2.1 of the License, or (at your option) any later version.
11 This library is distributed in the hope that it will be useful,
12 but WITHOUT ANY WARRANTY; without even the implied warranty of
13 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
14 Lesser General Public License for more details.
16 You should have received a copy of the GNU Lesser General Public
17 License along with this library; if not, write to the Free Software
18 Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
20 See http://www.salome-platform.org/ or email : webmaster.salome@opencascade.com
22 Author: Jean-Philippe Argaud, jean-philippe.argaud@edf.fr, EDF R&D
26 ================================================================================
27 **[DocU]** Advanced usage of the ADAO module
28 ================================================================================
30 This section presents advanced methods to use the ADAO module, how to get more
31 information during calculation, or how to use it without the graphical user
32 interface (GUI). It requires to know how to find files or commands included
33 inside the whole SALOME installation. All the names to be replaced by user are
34 indicated by the syntax ``<...>``.
36 Converting and executing an ADAO command file (JDC) using a Shell script
37 ------------------------------------------------------------------------
39 It is possible to convert and execute an ADAO command file (JDC, or ".comm/.py"
40 files pair, which resides in ``<ADAO JDC file directory>``) automatically by
41 using a template Shell script containing all the required steps. If the SALOME
42 main launching command , named ``salome``, is not available in a classical
43 terminal, the user has to know where are the main SALOME launching files, and in
44 particular the ``salome`` one. The directory in which this script resides is
45 symbolically named ``<SALOME main installation dir>`` and has to be replaced by
46 the good one in the Shell file template.
48 When an ADAO command file is build by the ADAO graphical editor and saved, if it
49 is named for example "AdaoStudy1.comm", then a companion file named
50 "AdaoStudy1.py" is automatically created in the same directory. It is named
51 ``<ADAO Python file>`` in the template, and it is converted to YACS as an
52 ``<ADAO YACS xml scheme>`` as a ".xml" file named "AdaoStudy1.xml". After that,
53 this last one can be executed in console mode using the standard YACS console
54 command (see YACS documentation for more information).
56 In all launching command Shell files examples, we choose to start and stop the
57 SALOME application server in the same script. It is not mandatory, but it is
58 useful to avoid stalling SALOME sessions.
60 The simplest example consist in only launching the given YACS sheme, which was
61 previously generated by the user in the graphical interface. In this case, after
62 having replaced the strings between ``<...>`` symbols, one needs only to save
63 the following Shell script::
66 USERDIR="<ADAO JDC file directory>"
67 SALOMEDIR="<SALOME main installation directory>"
68 $SALOMEDIR/salome start -k -t
69 $SALOMEDIR/salome shell -- "driver $USERDIR/<ADAO YACS xml scheme>"
70 $SALOMEDIR/salome shell killSalome.py
72 It is then required to change it to be in executable mode.
74 A more complete example consist in launching execution of a YACS sheme given by
75 the user, having previously verified its availability. For that, replacing the
76 text ``<SALOME main installation directory>``, one needs only to save the
77 following Shell script::
82 echo -e "\nUsage: $0 <ADAO YACS xml scheme>\n"
87 if (test ! -e $USERFILE)
89 echo -e "\nError : the XML file named $USERFILE does not exist.\n"
92 SALOMEDIR="<SALOME main installation directory>"
93 $SALOMEDIR/salome start -k -t
94 $SALOMEDIR/salome shell -- "driver $USERFILE"
95 $SALOMEDIR/salome shell killSalome.py
98 An another example consist in adding the conversion of the ADAO command file
99 (JDC, or ".comm/.py" files pair) in an associated YACS scheme (".xml" file). At
100 the end of the script, one choose also to remove the ``<ADAO YACS xml scheme>``
101 because it is a generated file. For that, after having carefully replaced the
102 text ``<SALOME main installation directory>``, one needs only to save the
103 following Shell script::
108 echo -e "\nUsage: $0 <ADAO .comm/.py case>\n"
112 F=`basename -s .comm $1`
113 F=`basename -s .py $F`
116 if (test ! -e $USERFILE.py)
118 echo -e "\nError : the PY file named $USERFILE.py does not exist.\n"
121 SALOMEDIR="<SALOME main installation directory>"
122 $SALOMEDIR/salome start -k -t
123 $SALOMEDIR/salome shell -- "python $SALOMEDIR/bin/salome/AdaoYacsSchemaCreator.py $USERFILE.py $USERFILE.xml"
124 $SALOMEDIR/salome shell -- "driver $USERFILE.xml"
125 $SALOMEDIR/salome shell killSalome.py
129 In all cases, the standard output and errors come in the launching terminal.
131 Running an ADAO calculation scheme in YACS using a TUI user mode
132 ----------------------------------------------------------------
134 This section describes how to execute in TUI (Text User Interface) YACS mode a
135 YACS calculation scheme, obtained in the graphical interface by using the ADAO
136 "Export to YACS" function. It uses the standard YACS TUI mode, which is briefly
137 recalled here (see YACS documentation for more information) through a simple
138 example. As described in documentation, a XML scheme can be loaded in a Python.
139 We give here a whole sequence of command lines to test the validity of the
140 scheme before executing it, adding some initial supplementary ones to explicitly
141 load the types catalog to avoid weird difficulties::
146 SALOMERuntime.RuntimeSALOME_setRuntime()
148 r = pilot.getRuntime()
149 xmlLoader = loader.YACSLoader()
150 xmlLoader.registerProcCataLoader()
152 catalogAd = r.loadCatalog("proc", "<ADAO YACS xml scheme>")
155 r.addCatalog(catalogAd)
158 p = xmlLoader.load("<ADAO YACS xml scheme>")
160 print "IO exception:",ex
162 logger = p.getLogger("parser")
163 if not logger.isEmpty():
164 print "The imported file has errors :"
165 print logger.getStr()
168 print "The schema is not valid and can not be executed"
169 print p.getErrorReport()
171 info=pilot.LinkInfo(pilot.LinkInfo.ALL_DONT_STOP)
172 p.checkConsistency(info)
173 if info.areWarningsOrErrors():
174 print "The schema is not consistent and can not be executed"
175 print info.getGlobalRepr()
177 e = pilot.ExecutorSwig()
179 if p.getEffectiveState() != pilot.DONE:
180 print p.getErrorReport()
182 This method allows for example to edit the YACS XML scheme in TUI, or to gather
183 results for further use.
185 .. _section_advanced_observer:
187 Getting information on special variables during the ADAO calculation in YACS
188 -----------------------------------------------------------------------------
190 .. index:: single: Observer
191 .. index:: single: Observer Template
193 Some special internal optimization variables, used during calculations, can be
194 monitored during the ADAO calculation in YACS. These variables can be printed,
195 plotted, saved, etc. This can be done using "*observer*", that are scripts,
196 each associated with one variable. In order to use this feature, the user has to
197 build scripts using as standard inputs (e.g. available in the namespace) the
198 variables ``var`` and ``info``. The variable ``var`` is to be used in the same
199 way as for the final ADD object, that is as a list/tuple object.
201 Some templates are available when editing the ADAO case in graphical editor.
202 These simple scripts can be customized by the user, either at the embedded
203 edition stage, or at the edition stage before execution, to improve the tuning
204 of the ADAO calculation.
206 As an example, here is one very simple script (similar to the "*ValuePrinter*"
207 template) used to print the value of one monitored variable::
209 print " --->",info," Value =",var[-1]
211 Stored in a Python file, this script can be associated to each variable
212 available in the "*SELECTION*" keyword of the "*Observers*" command:
213 "*Analysis*", "*CurrentState*", "*CostFunction*"... The current value of the
214 variable will be printed at each step of the optimization or assimilation
215 algorithm. The "*observer*" can embed plotting capabilities, storage, complex
216 printing, statistical treatment, etc.
218 Getting more information when running a calculation
219 ---------------------------------------------------
221 .. index:: single: Logging
223 When running a calculation, useful data and messages are logged. There are two
224 ways to obtain theses information.
226 The first one, and the preferred way, is to use the built-in variable "*Debug*"
227 available in every ADAO case. It is available through the edition GUI of the
228 module. Setting it to "*1*" will send messages in the log window of the YACS
231 The second one consist in using the "*logging*" native module of Python (see the
232 Python documentation http://docs.python.org/library/logging.html for more
233 information on this module). Everywhere in the YACS scheme, mainly through the
234 scripts entries, the user can set the logging level in accordance to the needs
235 of detailed information. The different logging levels are: "*DEBUG*", "*INFO*",
236 "*WARNING*", "*ERROR*", "*CRITICAL*". All the information flagged with a
237 certain level will be printed for whatever activated level above this particular
238 one (included). The easiest way is to change the log level by using the
239 following Python lines::
242 logging.getLogger().setLevel(logging.DEBUG)
244 The standard logging module default level is "*WARNING*", the default level in
245 the ADAO module is "*INFO*".
247 It is also recommended to include some logging or debug mechanisms in the
248 simulation code, and use them in conjunction with the two previous methods. But
249 be careful not to store too big variables because it cost time, whatever logging
250 level is chosen (that is, even if these variables are not printed).
252 .. _subsection_ref_parallel_df:
254 Accelerating numerical derivatives calculations by using a parallel mode
255 ------------------------------------------------------------------------
257 .. index:: single: EnableMultiProcessing
259 When setting an operator, as described in
260 :ref:`section_ref_operator_requirements`, the user can choose a functional form
261 "*ScriptWithOneFunction*". This form explicitly leads to approximate the tangent
262 and adjoint operators by a finite differences calculation. It requires several
263 calls to the direct operator (user defined function), at least as many times as
264 the dimension of the state vector. This are these calls that can potentially be
265 executed in parallel.
267 Under some conditions, it is then possible to accelerate the numerical
268 derivatives calculations by using a parallel mode for the finite differences
269 approximation. When setting up an ADAO case, it is done by adding the optional
270 sub-command "*EnableMultiProcessing*", set to "1", for the
271 "*SCRIPTWITHONEFUNCTION*" command in the operator definition. The parallel mode
272 will only use local resources (both multi-cores or multi-processors) of the
273 computer on which SALOME is running, requiring as many resources as available.
274 By default, this parallel mode is disabled ("*EnableMultiProcessing=0*").
276 The main conditions to perform parallel calculations come from the user defined
277 function, that represents the direct operator. This function has at least to be
278 "thread safe" to be executed in Python parallel environment (notions out of
279 scope of this paragraph). It is not obvious to give general rules, so it's
280 recommended, for the user who enable this internal parallelism, to carefully
281 verify his function and the obtained results.
283 From a user point of view, some conditions, that have to be met to set up
284 parallel calculations for tangent and the adjoint operators approximations, are
287 #. The dimension of the state vector is more than 2 or 3.
288 #. Unitary calculation of user defined direct function "last for long time", that is, more than few minutes.
289 #. The user defined direct function doesn't already use parallelism (or parallel execution is disabled in the user calculation).
290 #. The user defined direct function avoids read/write access to common resources, mainly stored data, output files or memory capacities.
291 #. The "*observer*" added by the user avoid read/write access to common resources, such as files or memory.
293 If these conditions are satisfied, the user can choose to enable the internal
294 parallelism for the numerical derivative calculations. Despite the simplicity of
295 activating, by setting one variable only, the user is urged to verify the
296 results of its calculations. One must at least doing them one time with
297 parallelism enabled, and an another time with parallelism disabled, to compare
298 the results. If it does fail somewhere, you have to know that this parallel
299 scheme is working for complex codes, like *Code_Aster* in *SalomeMeca*
300 [SalomeMeca]_ for example. So, if it does not work in your case, check your
301 operator function before and during enabling parallelism...
305 in case of doubt, it is recommended NOT TO ACTIVATE this parallelism.
307 It is also recalled that one have to choose the type "*multi*" for the default
308 container in order to launch the scheme, to allow a really parallel execution.
310 Switching from a version of ADAO to a newer one
311 -----------------------------------------------
313 .. index:: single: Version
315 The ADAO module and its ".comm" case files are identified by versions, with
316 "Major", "Minor" and "Revision" characteristics. A particular version is
317 numbered as "Major.Minor.Revision", with strong link with the numbering of the
320 Each version "Major.Minor.Revision" of the ADAO module can read ADAO case files
321 of the previous minor version "Major.Minor-1.*". In general, it can also read
322 ADAO case files of all the previous minor versions for one major branch, but it
323 is not guaranteed for all the commands or keywords. In general also, an ADAO
324 case file for one version can not be read by a previous minor or major version
327 Switching from 7.5 to 7.6
328 +++++++++++++++++++++++++
330 There is no known incompatibility for the ADAO case files. The upgrade procedure
331 is to read the old ADAO case file with the new SALOME/ADAO module, and save it
332 with a new name. This procedure proceed automatically to the required
333 modifications of the storage tree of the ADAO case file.
335 Switching from 7.4 to 7.5
336 +++++++++++++++++++++++++
338 There is no known incompatibility for the ADAO case files. The upgrade procedure
339 is to read the old ADAO case file with the new SALOME/ADAO module, and save it
342 Switching from 7.3 to 7.4
343 +++++++++++++++++++++++++
345 There is no known incompatibility for the ADAO case files. The upgrade procedure
346 is to read the old ADAO case file with the new SALOME/ADAO module, and save it
349 Switching from 7.2 to 7.3
350 +++++++++++++++++++++++++
352 There is no known incompatibility for the ADAO case files. The upgrade procedure
353 is to read the old ADAO case file with the new SALOME/ADAO module, and save it
356 Switching from 6.6 to 7.2
357 +++++++++++++++++++++++++
359 There is no known incompatibility for the ADAO case files. The upgrade procedure
360 is to read the old ADAO case file with the new SALOME/ADAO module, and save it
363 There is one incompatibility introduced for the post-processing or observer
364 script files. The old syntax to call a result object, such as the "*Analysis*"
365 one (in a script provided through the "*UserPostAnalysis*" keyword), was for
368 Analysis = ADD.get("Analysis").valueserie(-1)
369 Analysis = ADD.get("Analysis").valueserie()
371 The new syntax is entirely similar to the (classical) one of a list or tuple
374 Analysis = ADD.get("Analysis")[-1]
375 Analysis = ADD.get("Analysis")[:]
377 The post-processing scripts has to be modified.
379 Switching from 6.5 to 6.6
380 +++++++++++++++++++++++++
382 There is no known incompatibility for the ADAO case file. The upgrade procedure
383 is to read the old ADAO case file with the new SALOME/ADAO module, and save it
386 There is one incompatibility introduced for the naming of operators used to for
387 the observation operator. The new mandatory names are "*DirectOperator*",
388 "*TangentOperator*" and "*AdjointOperator*", as described in the last subsection
389 of the chapter :ref:`section_reference`. The operator scripts has to be
392 Switching from 6.4 to 6.5
393 +++++++++++++++++++++++++
395 There is no known incompatibility for the ADAO case file or the accompanying
396 scripts. The upgrade procedure is to read the old ADAO case file with the new
397 SALOME/ADAO module, and save it with a new name.
399 Switching from 6.3 to 6.4
400 +++++++++++++++++++++++++
402 There is no known incompatibility for the ADAO case file or the accompanying
403 scripts. The upgrade procedure is to read the old ADAO case file with the new
404 SALOME/ADAO module, and save it with a new name.