From: Jean-Philippe ARGAUD Date: Wed, 8 Sep 2021 05:57:32 +0000 (+0200) Subject: Documentation and reporting improvements X-Git-Tag: V9_8_0a1~17 X-Git-Url: http://git.salome-platform.org/gitweb/?a=commitdiff_plain;h=44f76651d9b8e83d196d710205de80900f206472;p=modules%2Fadao.git Documentation and reporting improvements --- diff --git a/doc/en/images/schema_temporel_KF.png b/doc/en/images/schema_temporel_KF.png index a994fd4..dcfa447 100644 Binary files a/doc/en/images/schema_temporel_KF.png and b/doc/en/images/schema_temporel_KF.png differ diff --git a/doc/en/ref_algorithm_ExtendedKalmanFilter.rst b/doc/en/ref_algorithm_ExtendedKalmanFilter.rst index cc45fe9..900b3fa 100644 --- a/doc/en/ref_algorithm_ExtendedKalmanFilter.rst +++ b/doc/en/ref_algorithm_ExtendedKalmanFilter.rst @@ -35,8 +35,8 @@ extended Kalman Filter, using a non-linear calculation of the state and the incremental evolution (process). Conceptually, we can represent the temporal pattern of action of the evolution -operator for this algorithm in the following way, with **x** the state and -**P** the state error covariance : +and observation operators in this algorithm in the following way, with **x** +the state and **P** the state error covariance : .. _schema_temporel_KF: .. image:: images/schema_temporel_KF.png diff --git a/doc/en/ref_algorithm_KalmanFilter.rst b/doc/en/ref_algorithm_KalmanFilter.rst index f9c5f90..13bff48 100644 --- a/doc/en/ref_algorithm_KalmanFilter.rst +++ b/doc/en/ref_algorithm_KalmanFilter.rst @@ -39,8 +39,8 @@ cases. One can verify the linearity of the operators with the help of the :ref:`section_ref_algorithm_LinearityTest`. Conceptually, we can represent the temporal pattern of action of the evolution -operator for this algorithm in the following way, with **x** the state and -**P** the state error covariance : +and observation operators in this algorithm in the following way, with **x** +the state and **P** the state error covariance : .. _schema_temporel_KF: .. image:: images/schema_temporel_KF.png diff --git a/doc/en/tui.rst b/doc/en/tui.rst index 70d78c8..5125c9b 100644 --- a/doc/en/tui.rst +++ b/doc/en/tui.rst @@ -627,6 +627,29 @@ with these Python external case operations. one the commands establishing the current calculation case. Some formats are only available as input or as output. +In addition, simple information about the case study as defined by the user can +be obtained by using the Python "*print*" command directly on the case, at any +stage during its design. For example:: + + from numpy import array, matrix + from adao import adaoBuilder + case = adaoBuilder.New() + case.set( 'AlgorithmParameters', Algorithm='3DVAR' ) + case.set( 'Background', Vector=[0, 1, 2] ) + print(case) + +which result is here:: + + ================================================================================ + ADAO Study report + ================================================================================ + + - AlgorithmParameters command has been set with values: + Algorithm='3DVAR' + + - Background command has been set with values: + Vector=[0, 1, 2] + .. _subsection_tui_advanced: More advanced examples of ADAO TUI calculation case diff --git a/doc/en/tutorials_in_python.rst b/doc/en/tutorials_in_python.rst index 6a268ec..1adb80b 100644 --- a/doc/en/tutorials_in_python.rst +++ b/doc/en/tutorials_in_python.rst @@ -29,11 +29,11 @@ This section presents some examples on using the ADAO module in Python. The first one shows how to build a very simple data assimilation case defining -explicitly all the required input data through the textual user interface -(TUI). The second one shows, on the same case, how to define input data using -external sources through scripts. We describe here always Python scripts -because they can be directly inserted in script definitions of Python -interface, but external files can use other languages. +explicitly all the required input data through the textual user interface (TUI) +described in :ref:`section_tui`. The second one shows, on the same case, how to +define input data using external sources through scripts. We describe here +always Python scripts because they can be directly inserted in script +definitions of Python interface, but external files can use other languages. These examples are intentionally described in the same way than for the :ref:`section_tutorials_in_salome` because they are similar to the ones that diff --git a/doc/fr/images/schema_temporel_KF.png b/doc/fr/images/schema_temporel_KF.png index bd27dde..f1a18b5 100644 Binary files a/doc/fr/images/schema_temporel_KF.png and b/doc/fr/images/schema_temporel_KF.png differ diff --git a/doc/fr/ref_algorithm_ExtendedKalmanFilter.rst b/doc/fr/ref_algorithm_ExtendedKalmanFilter.rst index 58ab954..e680d49 100644 --- a/doc/fr/ref_algorithm_ExtendedKalmanFilter.rst +++ b/doc/fr/ref_algorithm_ExtendedKalmanFilter.rst @@ -34,9 +34,9 @@ Cet algorithme réalise une estimation de l'état d'un système dynamique par un filtre de Kalman étendu, utilisant un calcul non linéaire de l'état et de l'évolution incrémentale (processus). -Conceptuellement, on peut représenter le schéma temporel d'action de -l'opérateur d'évolution de cet algorithme de la manière suivante, avec **x** -l'état et **P** la covariance d'erreur d'état : +Conceptuellement, on peut représenter le schéma temporel d'action des +opérateurs d'évolution et d'observation dans cet algorithme de la manière +suivante, avec **x** l'état et **P** la covariance d'erreur d'état : .. _schema_temporel_KF: .. image:: images/schema_temporel_KF.png diff --git a/doc/fr/ref_algorithm_KalmanFilter.rst b/doc/fr/ref_algorithm_KalmanFilter.rst index 2c464c6..f96dd4c 100644 --- a/doc/fr/ref_algorithm_KalmanFilter.rst +++ b/doc/fr/ref_algorithm_KalmanFilter.rst @@ -38,9 +38,9 @@ incrémentale (processus) linéaires, même s'il fonctionne parfois dans les cas "faiblement" non-linéaire. On peut vérifier la linéarité de l'opérateur d'observation à l'aide de l':ref:`section_ref_algorithm_LinearityTest`. -Conceptuellement, on peut représenter le schéma temporel d'action de -l'opérateur d'évolution de cet algorithme de la manière suivante, avec **x** -l'état et **P** la covariance d'erreur d'état : +Conceptuellement, on peut représenter le schéma temporel d'action des +opérateurs d'évolution et d'observation dans cet algorithme de la manière +suivante, avec **x** l'état et **P** la covariance d'erreur d'état : .. _schema_temporel_KF: .. image:: images/schema_temporel_KF.png diff --git a/doc/fr/tui.rst b/doc/fr/tui.rst index 5b6bd78..c74dcaf 100644 --- a/doc/fr/tui.rst +++ b/doc/fr/tui.rst @@ -654,6 +654,29 @@ externes au cas. autre les commandes établissant le cas de calcul en cours. Certains formats ne sont disponibles qu'en entrée ou qu'en sortie. +De plus, on peut obtenir une information simple sur le cas d'étude tel que +défini par l'utilisateur en utilisant directement la commande "*print*" de Python +sur le cas, à toute étape lors de sa construction. Par exemple:: + + from numpy import array, matrix + from adao import adaoBuilder + case = adaoBuilder.New() + case.set( 'AlgorithmParameters', Algorithm='3DVAR' ) + case.set( 'Background', Vector=[0, 1, 2] ) + print(case) + +dont le résultat est ici:: + + ================================================================================ + ADAO Study report + ================================================================================ + + - AlgorithmParameters command has been set with values: + Algorithm='3DVAR' + + - Background command has been set with values: + Vector=[0, 1, 2] + .. _subsection_tui_advanced: Exemples plus avancés de cas de calcul TUI ADAO diff --git a/doc/fr/tutorials_in_python.rst b/doc/fr/tutorials_in_python.rst index d641a34..3e08b11 100644 --- a/doc/fr/tutorials_in_python.rst +++ b/doc/fr/tutorials_in_python.rst @@ -30,11 +30,12 @@ Cette section présente quelques exemples d'utilisation du module ADAO en Python. Le premier montre comment construire un cas simple d'assimilation de données définissant explicitement toutes les données d'entrée requises à -travers l'interface utilisateur textuelle (TUI). Le second montre, sur le même -cas, comment définir les données d'entrée à partir de sources externes à -travers des scripts. On présente ici toujours des scripts Python car ils sont -directement insérables dans les définitions de script de l'interface Python, -mais les fichiers externes peuvent utiliser d'autres langages. +travers l'interface utilisateur textuelle (TUI) décrite en partie +:ref:`section_tui`. Le second montre, sur le même cas, comment définir les +données d'entrée à partir de sources externes à travers des scripts. On +présente ici toujours des scripts Python car ils sont directement insérables +dans les définitions de script de l'interface Python, mais les fichiers +externes peuvent utiliser d'autres langages. Ces exemples sont intentionnellement décrits de manière semblables aux :ref:`section_tutorials_in_salome` car ils sont similaires à ceux que l'on peut diff --git a/src/daComposant/daCore/Aidsm.py b/src/daComposant/daCore/Aidsm.py index a981db1..679587e 100644 --- a/src/daComposant/daCore/Aidsm.py +++ b/src/daComposant/daCore/Aidsm.py @@ -21,7 +21,7 @@ # Author: Jean-Philippe Argaud, jean-philippe.argaud@edf.fr, EDF R&D """ - Normalized interface for ADAO scripting (generic API) + Normalized interface for ADAO scripting (generic API). """ __author__ = "Jean-Philippe ARGAUD" __all__ = ["Aidsm"] @@ -816,6 +816,11 @@ class Aidsm(object): "Clarifie la visibilité des méthodes" return ['set', 'get', 'execute', 'dump', 'load', '__doc__', '__init__', '__module__'] + def __str__(self): + "Représentation pour impression (mais pas repr)" + msg = self.dump(None, "SimpleReportInPlainTxt") + return msg + def prepare_to_pickle(self): "Retire les variables non pickelisables, avec recopie efficace" if self.__adaoObject['AlgorithmParameters'] is not None: diff --git a/src/daComposant/daCore/BasicObjects.py b/src/daComposant/daCore/BasicObjects.py index e227aa7..bb0bec8 100644 --- a/src/daComposant/daCore/BasicObjects.py +++ b/src/daComposant/daCore/BasicObjects.py @@ -2161,7 +2161,7 @@ class Covariance(object): # ============================================================================== class Observer2Func(object): """ - Creation d'une fonction d'observateur a partir de son texte + Création d'une fonction d'observateur a partir de son texte """ def __init__(self, corps=""): self.__corps = corps @@ -2175,7 +2175,7 @@ class Observer2Func(object): # ============================================================================== class CaseLogger(object): """ - Conservation des commandes de creation d'un cas + Conservation des commandes de création d'un cas """ def __init__(self, __name="", __objname="case", __addViewers=None, __addLoaders=None): self.__name = str(__name) @@ -2186,6 +2186,9 @@ class CaseLogger(object): "TUI" :Interfaces._TUIViewer, "SCD" :Interfaces._SCDViewer, "YACS":Interfaces._YACSViewer, + "SimpleReportInRst":Interfaces._SimpleReportInRstViewer, + "SimpleReportInHtml":Interfaces._SimpleReportInHtmlViewer, + "SimpleReportInPlainTxt":Interfaces._SimpleReportInPlainTxtViewer, } self.__loaders = { "TUI" :Interfaces._TUIViewer, diff --git a/src/daComposant/daCore/Interfaces.py b/src/daComposant/daCore/Interfaces.py index c67e3e2..0b6fbf8 100644 --- a/src/daComposant/daCore/Interfaces.py +++ b/src/daComposant/daCore/Interfaces.py @@ -35,6 +35,7 @@ import copy from daCore import Persistence from daCore import PlatformInfo from daCore import Templates +from daCore import Reporting # ============================================================================== class GenericCaseViewer(object): @@ -610,6 +611,76 @@ class _YACSViewer(GenericCaseViewer): __fid.close() return __text +# ============================================================================== +class _ReportViewer(GenericCaseViewer): + """ + Partie commune de restitution simple + """ + def __init__(self, __name="", __objname="case", __content=None, __object=None): + "Initialisation et enregistrement de l'entete" + GenericCaseViewer.__init__(self, __name, __objname, __content, __object) + self._r = Reporting.ReportStorage() + self._r.clear() + if self._name == "": + self._r.append("ADAO Study report", "title") + else: + self._r.append(str(self._name), "title") + if self._content is not None: + for command in self._content: + self._append(*command) + def _append(self, __command=None, __keys=None, __local=None, __pre=None, __switchoff=False): + "Transformation d'une commande individuelle en un enregistrement" + if __command is not None and __keys is not None and __local is not None: + if __command in ("set","get") and "Concept" in __keys: __command = __local["Concept"] + __text = "" + __text += "%s command has been set"%str(__command.replace("set","")) + __ktext = "" + for k in __keys: + if k not in __local: continue + __v = __local[k] + if __v is None: continue + if k == "Checked" and not __v: continue + if k == "Stored" and not __v: continue + if k == "ColMajor" and not __v: continue + if k == "InputFunctionAsMulti" and not __v: continue + if k == "nextStep" and not __v: continue + if k == "AvoidRC" and __v: continue + if k == "noDetails": continue + if k == "Concept": continue + if k == "self": continue + if isinstance(__v,Persistence.Persistence): __v = __v.values() + numpy.set_printoptions(precision=15,threshold=1000000,linewidth=1000*15) + __ktext += "\n %s=%s, "%(k,repr(__v)) + numpy.set_printoptions(precision=8,threshold=1000,linewidth=75) + if len(__ktext) > 0: + __text += " with values:" + __ktext + __text = __text.rstrip(", ") + self._r.append(__text, "uli") + def _finalize(self, __upa=None): + "Enregistrement du final" + raise NotImplementedError() + +class _SimpleReportInRstViewer(_ReportViewer): + """ + Restitution simple en RST + """ + def _finalize(self, __upa=None): + self._lineSerie.append(Reporting.ReportViewInRst(self._r).__str__()) + +class _SimpleReportInHtmlViewer(_ReportViewer): + """ + Restitution simple en HTML + """ + def _finalize(self, __upa=None): + self._lineSerie.append(Reporting.ReportViewInHtml(self._r).__str__()) + +class _SimpleReportInPlainTxtViewer(_ReportViewer): + """ + Restitution simple en TXT + """ + def _finalize(self, __upa=None): + self._lineSerie.append(Reporting.ReportViewInPlainTxt(self._r).__str__()) + # ============================================================================== class ImportFromScript(object): """ diff --git a/src/daComposant/daCore/PlatformInfo.py b/src/daComposant/daCore/PlatformInfo.py index 950e0bb..9101abe 100644 --- a/src/daComposant/daCore/PlatformInfo.py +++ b/src/daComposant/daCore/PlatformInfo.py @@ -21,7 +21,7 @@ # Author: Jean-Philippe Argaud, jean-philippe.argaud@edf.fr, EDF R&D """ - Informations sur le code et la plateforme, et mise à jour des chemins + Informations sur le code et la plateforme, et mise à jour des chemins. La classe "PlatformInfo" permet de récupérer les informations générales sur le code et la plateforme sous forme de strings, ou d'afficher directement diff --git a/src/daComposant/daCore/Reporting.py b/src/daComposant/daCore/Reporting.py new file mode 100644 index 0000000..96c1cd7 --- /dev/null +++ b/src/daComposant/daCore/Reporting.py @@ -0,0 +1,300 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2008-2021 EDF R&D +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +# +# See http://www.salome-platform.org/ or email : webmaster.salome@opencascade.com +# +# Author: Jean-Philippe Argaud, jean-philippe.argaud@edf.fr, EDF R&D + +""" + Gestion simple de rapports structurés créés par l'utilisateur. +""" +__author__ = "Jean-Philippe ARGAUD" +__all__ = [] + +import os.path + +# ============================================================================== +# Classes de services non utilisateur + +class _ReportPartM__(object): + """ + Store and retrieve the data for C: internal class + """ + def __init__(self, part="default"): + self.__part = str(part) + self.__styles = [] + self.__content = [] + + def append(self, content, style="p", position=-1): + if position == -1: + self.__styles.append(style) + self.__content.append(content) + else: + self.__styles.insert(position, style) + self.__content.insert(position, content) + return 0 + + def get_styles(self): + return self.__styles + + def get_content(self): + return self.__content + +class _ReportM__(object): + """ + Store and retrieve the data for C: internal class + """ + def __init__(self, part='default'): + self.__document = {} + self.__document[part] = _ReportPartM__(part) + + def append(self, content, style="p", position=-1, part='default'): + if part not in self.__document: + self.__document[part] = _ReportPartM__(part) + self.__document[part].append(content, style, position) + return 0 + + def get_styles(self): + op = list(self.__document.keys()) ; op.sort() + return [self.__document[k].get_styles() for k in op] + + def get_content(self): + op = list(self.__document.keys()) ; op.sort() + return [self.__document[k].get_content() for k in op] + + def clear(self): + self.__init__() + +class __ReportC__(object): + """ + Get user commands, update M and V: user intertace to create the report + """ + m = _ReportM__() + + def append(self, content="", style="p", position=-1, part="default"): + return self.m.append(content, style, position, part) + + def retrieve(self): + st = self.m.get_styles() + ct = self.m.get_content() + return st, ct + + def clear(self): + self.m.clear() + +class __ReportV__(object): + """ + Interact with user and C: template for reports + """ + + default_filename="report.txt" + + def __init__(self, c): + self.c = c + + def save(self, filename=None): + if filename is None: + filename = self.default_filename + _filename = os.path.abspath(filename) + # + h = self.get() + fid = open(_filename, 'w') + fid.write(h) + fid.close() + return filename, _filename + + def retrieve(self): + return self.c.retrieve() + + def __str__(self): + return self.get() + + def close(self): + del self.c + return 0 + +# ============================================================================== +# Classes d'interface utilisateur : ReportStorage, ReportViewIn* +# Tags de structure : (title, h1, h2, h3, p, uli, oli, , ) + +ReportStorage = __ReportC__ + +class ReportViewInHtml(__ReportV__): + """ + Report in HTML + """ + + default_filename="report.html" + tags = { + "oli":"li", + "uli":"li", + } + + def get(self): + st, ct = self.retrieve() + inuLi, inoLi = False, False + pg = "\n" + pg += "\nReport in HTML" + pg += "\n\n" + for k,ps in enumerate(st): + pc = ct[k] + try: + ii = ps.index("title") + title = pc[ii] + pg += "%s\n%s\n%s"%('

',title,'

') + except Exception: + pass + for i,s in enumerate(ps): + c = pc[i] + if s == "uli" and not inuLi: + pg += "\n" + inuLi = False + elif s != "oli" and inoLi: + pg += "\n" + inoLi = False + elif s == "title": + continue + for t in self.tags: + if s == t: s = self.tags[t] + pg += "\n<%s>%s"%(s,c,s) + pg += "\n\n" + return pg + +class ReportViewInRst(__ReportV__): + """ + Report in RST + """ + + default_filename="report.rst" + tags = { + "p":["\n\n",""], + "uli":["\n - ",""], + "oli":["\n #. ",""], + } + titles = { + # "title":["=","="], + "h1":["","-"], + "h2":["","+"], + "h3":["","*"], + } + translation = { + "":"**", + "":"*", + "":"**", + "":"*", + } + + def get(self): + st, ct = self.retrieve() + inuLi, inoLi = False, False + pg = "" + for k,ps in enumerate(st): + pc = ct[k] + try: + ii = ps.index("title") + title = pc[ii] + pg += "%s\n%s\n%s"%("="*80,title,"="*80) + except Exception: + pass + for i,s in enumerate(ps): + c = pc[i] + if s == "uli" and not inuLi: + pg += "\n" + inuLi = True + elif s == "oli" and not inoLi: + pg += "\n" + inoLi = True + elif s != "uli" and inuLi: + pg += "\n" + inuLi = False + elif s != "oli" and inoLi: + pg += "\n" + inoLi = False + for t in self.translation: + c = c.replace(t,self.translation[t]) + if s in self.titles.keys(): + pg += "\n%s\n%s\n%s"%(self.titles[s][0]*len(c),c,self.titles[s][1]*len(c)) + elif s in self.tags.keys(): + pg += "%s%s%s"%(self.tags[s][0],c,self.tags[s][1]) + pg += "\n" + return pg + +class ReportViewInPlainTxt(__ReportV__): + """ + Report in plain TXT + """ + + default_filename="report.txt" + tags = { + "p":["\n",""], + "uli":["\n - ",""], + "oli":["\n - ",""], + } + titles = { + # "title":["=","="], + "h1":["",""], + "h2":["",""], + "h3":["",""], + } + translation = { + "":"", + "":"", + "":"", + "":"", + } + + def get(self): + st, ct = self.retrieve() + inuLi, inoLi = False, False + pg = "" + for k,ps in enumerate(st): + pc = ct[k] + try: + ii = ps.index("title") + title = pc[ii] + pg += "%s\n%s\n%s"%("="*80,title,"="*80) + except Exception: + pass + for i,s in enumerate(ps): + c = pc[i] + if s == "uli" and not inuLi: + inuLi = True + elif s == "oli" and not inoLi: + inoLi = True + elif s != "uli" and inuLi: + inuLi = False + elif s != "oli" and inoLi: + inoLi = False + for t in self.translation: + c = c.replace(t,self.translation[t]) + if s in self.titles.keys(): + pg += "\n%s\n%s\n%s"%(self.titles[s][0]*len(c),c,self.titles[s][1]*len(c)) + elif s in self.tags.keys(): + pg += "\n%s%s%s"%(self.tags[s][0],c,self.tags[s][1]) + pg += "\n" + return pg + +# ============================================================================== +if __name__ == "__main__": + print('\n AUTODIAGNOSTIC\n') diff --git a/src/daComposant/daCore/Templates.py b/src/daComposant/daCore/Templates.py index cb3f0a2..7f9e9f5 100644 --- a/src/daComposant/daCore/Templates.py +++ b/src/daComposant/daCore/Templates.py @@ -21,7 +21,7 @@ # Author: Jean-Philippe Argaud, jean-philippe.argaud@edf.fr, EDF R&D """ - Modèles généraux pour les observers, le post-processing + Modèles généraux pour les observers, le post-processing. """ __author__ = "Jean-Philippe ARGAUD" __all__ = ["ObserverTemplates"]