X-Git-Url: http://git.salome-platform.org/gitweb/?a=blobdiff_plain;f=doc%2Fsrc%2Fconfiguration.rst;h=c3c6b9639344f4e9aebe95adf9fb4b82235ba95a;hb=8de57afec087347ead374b04e056c72118a61f71;hp=19fed19f785d4dfafb1c4b574e2d9477b1180cd9;hpb=a2622c0a85347bf7bc146d165631b8fa0e6e31cf;p=tools%2Fsat.git

diff --git a/doc/src/configuration.rst b/doc/src/configuration.rst
index 19fed19..c3c6b96 100644
--- a/doc/src/configuration.rst
+++ b/doc/src/configuration.rst
@@ -2,67 +2,417 @@
 Configuration
 *************
 
-*salomeTools* uses files with **.pyconf** extension to store its configuration parameters.
-These pyconf configuration files are provided by the salomeTool projects that are set by sat init command.
+Introduction
+============
 
-When executing a command, sat will load several configuration files in a specific order.
-When all the files are loaded a *config* object is created.
-Then, this object is passed to all command scripts.
+For the configuration, SAT uses a python module called *config*, which aims to offer more power and flexibility for the configuration of python programs.
+This module was slightly adapted for SAT, and renamed Pyconf.
+(see `config module <http://www.red-dove.com/config-doc/>`_ for a complete description of the module, the associated syntax, the documentation).
 
+*salomeTools* uses files with **.pyconf** extension to store the configuration parameters.
+These *.pyconf* are parsed by SAT, and merged into a global configuration, which is passed to the sat commands and used by them.
 
-Syntax
-======
-The configuration files use a python-like structure format 
-(see `config module <http://www.red-dove.com/config-doc/>`_ for a complete description).
 
-* **{}** define a dictionary,
-* **[]** define a list,
-* **@** can be used to include a file,
-* **$prefix** reference to another parameter (ex: ``$PRODUCT.name``),
-* **#** comments.
+Configuration projects
+======================
 
-.. note:: in this documentation a reference to a configuration parameter will be noted ``XXX.YYY``.
+By default SAT is provided with no configuration at all, except is own internal one.
+The configuration is brought by SAT projects : usually a git base containing all the configuration files of a project (*.pyconf* files).
+For Salome platform, the SAT project is called SAT_SALOME and can be dowloaded from salome Tuleap forge.
+SAT projects are loaded in sat with the sat init command:
 
-Description
-===========
+.. code-block:: bash
 
-.. _VARS-Section:
+    # get salome platform SAT configuration project (SAT_SALOME), and load it into SAT
+    git clone SAT_SALOME
+    SAT/sat init --add_project $(pwd)/SAT_SALOME/salome.pyconf  
 
-VARS section
--------------
-| This section is dynamically created by salomeTools at run time.
-| It contains information about the environment: date, time, OS, architecture etc. 
+SAT_SALOME project provides all configuration files for salome applications, and for the products that are used in these applications.
 
-::
 
-    # to get the current setting
-    sat config --value VARS
+Application configuration
+=========================
+
+The configuration files of applications contain the required information for SAT to build the application.
+They are usually located in the application directory of the project:
+
+.. code-block:: bash
+
+    # list applications provided by SAT_SALOME project
+    ls SAT_SALOME/applications
+        MEDCOUPLING-9.4.0.pyconf             SALOME-7.8.0.pyconf       SALOME-8.5.0.pyconf          SALOME-9.4.0.pyconf
+
+The application configuration file defines the APPLICATION sections. The content of this section can be displayed with *sat config* command:
+
+.. code-block:: bash
+
+    sat config SALOME-9.4.0 -v APPLICATION
+
+SAT users that need to create new application files for their own purpose usually copy an existing configuration file and adapt it to their application.
+Let's discribe the content of an application pyconf file. We take in the following examples the file *SAT_SALOME/applications/SALOME-9.4.0.pyconf*.
+
+
+Global variables and flags
+--------------------------
+
+At the beginning of the APPLICATION sections, global variables and flags are defined:  
+
+  * **name** : the name of the application (mandatory)
+  * **workdir** : the directory in which the application is produced (mandatory)
+  * **tag** : the default tag to use for the git bases
+  * **dev** : activate the dev mode. in dev mode git bases are checked out only one time, to avoid risks of removing developments.
+  * **verbose** : activate verbosity in the compilation
+  * **debug** : activate debug mode in the compilation, i.e -g option
+  * **python3** : 'yes/no' tell sat that the application uses python3 
+  * **base** : 'yes/no/name' to set up the use of a SAT base
+
+.. code-block:: bash
+
+    APPLICATION :
+    {
+        name : 'SALOME-9.4.0'
+        workdir : $LOCAL.workdir + $VARS.sep + $APPLICATION.name + '-' + $VARS.dist
+        tag : 'V9_4_BR'
+        debug : 'no'
+        dev : 'no'
+        base : 'no'
+        python3 : 'yes'
+        ...
+
+Please note the workdir variable is defined in the above example with references to other sections defined in other configurations files (i.e. $LOCAL and $VARS).
+It's a useful Pyconf functionality).
+Most of the global variables are optionnal, except name and workdir.
+
+Environment subsection
+----------------------
+
+This subsection allows defining environment variables at the application level (most of the time the environment is set by the products configuration).
+
+.. code-block:: bash
 
-APPLICATION section
+    APPLICATION :
+    {
+    ...
+        environ :
+        {
+            build : {CONFIGURATION_ROOT_DIR : $workdir + $VARS.sep + "SOURCES" + $VARS.sep + "CONFIGURATION"}
+            launch : {PYTHONIOENCODING:"UTF_8"}
+            SALOME_trace : "local" # local/file:.../with_logger
+            SALOME_MODULES : "SHAPER,GEOM,SMESH,PARAVIS,YACS,JOBMANAGER"  # specify the first modules to display in gui
+        }
+    }
+
+In the example above CONFIGURATION_ROOT_DIR variable will be set only at compile time (usage of *build* key), while PYTHONIOENCODING will be set only at run-time (use of *launch* key).
+variables SALOME_trace and SALOME_MODULES are set both at compile time and run time.
+
+
+products subsection
+-------------------
+
+This subsection will specify which products are included in the application.
+For each product, it is possible to specify in a dictionnary:
+
+  * **tag** : the tag to use for the product
+  * **dev** : activate the dev mode.
+  * **verbose** : activate verbosity in the compilation
+  * **debug** : activate debug mode
+
+If this flags are not specified, SAT takes the default application flag.
+In the following example, SAT uses the the default tag V9_4_BR for products SHAPER, KERNEL and MEDCOUPLING.
+For LIBBATCH it uses the tag V2_4_2.
+KERNEL is compile in debug and verbose mode.
+
+.. code-block:: bash
+
+    APPLICATION :
+    {
+    ...
+        tag : 'V9_4_BR'
+    ...
+        products :
+        {
+        'SHAPER'
+        'LIBBATCH' : {tag :'V2_4_2'}
+        'KERNEL' : {debug:'yes', verbose:'yes'}
+        'MEDCOUPLING'
+        ...
+
+
+properties
+----------
+
+Properties are used by SAT to define some general rules or policies.
+They can be defined in the application configuration with the properties subsection:
+
+.. code-block:: bash
+
+    APPLICATION :
+    {
+    ...
+        properties :
+        {
+            mesa_launcher_in_package : "yes"
+            repo_dev : "yes"
+            pip : 'yes'
+            pip_install_dir : 'python'
+        }
+
+In this example the following properties are used:
+
+ * **mesa_launcher_in_package** : ask to put a mesa launcher in the packages produced by sat package commans
+ * **repo_dev** : use the development git base (for salome, the tuleap forge)
+ * **pip** : ask to use pip to get python products
+ * **pip_install_dir** : install pip products in python installation directory (not in separate directories)
+
+
+Products configuration
+======================
+
+The configuration files of products contain the required information for SAT to build each product.
+They are usually located in the product directory of the project. SAT_SALOME supports a lot of products:
+
+.. code-block:: bash
+
+    ls SAT_SALOME/products/
+    ADAO_INTERFACE.pyconf  COREFLOWS_PROFILE.pyconf  GHS3DPLUGIN.pyconf         JOBMANAGER.pyconf       omniORB.pyconf       Python.pyconf                    Sphinx.pyconf
+    ADAO_MODULE.pyconf     COREFLOWS.pyconf          GHS3DPRLPLUGIN.pyconf      KERNEL.pyconf           omniORBpy.pyconf     pytz.pyconf                      sphinx_rtd_theme.pyconf
+    ADAO.pyconf            cppunit.pyconf            gl2ps.pyconf               kiwisolver.pyconf       opencv.pyconf        qt.pyconf                        subprocess32.pyconf
+    alabaster.pyconf       cycler.pyconf             glu.pyconf                 lapack.pyconf           openmpi.pyconf       qwt.pyconf                       swig.pyconf
+    ALAMOS_PROFILE.pyconf  Cython.pyconf             GMSHPLUGIN.pyconf          lata.pyconf             ospray.pyconf        requests.pyconf                  tbb.pyconf
+    ALAMOS.pyconf          dateutil.pyconf           gmsh.pyconf                LIBBATCH.pyconf         packaging.pyconf     SALOME_FORMATION_PROFILE.pyconf  tcl.pyconf
+    Babel.pyconf           distribute.pyconf         graphviz.pyconf            libpng.pyconf           ParaViewData.pyconf  SALOME_PROFILE.pyconf            TECHOBJ_ROOT.pyconf
+    BLSURFPLUGIN.pyconf    DOCUMENTATION.pyconf      GUI.pyconf                 libxml2.pyconf          ParaView.pyconf      SALOME.pyconf                    tk.pyconf
+    boost.pyconf           docutils.pyconf           hdf5.pyconf                llvm.pyconf             PARAVIS.pyconf       SAMPLES.pyconf                   Togl.pyconf
+    bsd_xdr.pyconf         doxygen.pyconf            HELLO.pyconf               markupsafe.pyconf       ParMetis.pyconf      scipy.pyconf                     TRIOCFD_IHM.pyconf
+    CALCULATOR.pyconf      EFICAS.pyconf             HEXABLOCKPLUGIN.pyconf     matplotlib.pyconf       patches              scons.pyconf                     TrioCFD.pyconf
+    CAS.pyconf             EFICAS_TOOLS.pyconf       HEXABLOCK.pyconf           MEDCOUPLING.pyconf      petsc.pyconf         scotch.pyconf                    TRUST.pyconf
+    CDMATH.pyconf          eigen.pyconf              HexoticPLUGIN.pyconf       medfile.pyconf          planegcs.pyconf      setuptools.pyconf                typing.pyconf
+    CEATESTBASE.pyconf     embree.pyconf             Hexotic.pyconf             med_pre_windows.pyconf  pockets.pyconf       SHAPER.pyconf                    uranie_win.pyconf
+    certifi.pyconf         env_scripts               homard_bin.pyconf          MED.pyconf              pthreads.pyconf      sip.pyconf                       urllib3.pyconf
+    cgns.pyconf            expat.pyconf              homard_pre_windows.pyconf  mesa.pyconf             PY2CPP.pyconf        six.pyconf                       VISU.pyconf
+    chardet.pyconf         f2c.pyconf                HOMARD.pyconf              MeshGems.pyconf         PYCALCULATOR.pyconf  SMESH.pyconf                     vtk.pyconf
+    click.pyconf           FIELDS.pyconf             HXX2SALOME.pyconf          metis.pyconf            Pygments.pyconf      snowballstemmer.pyconf           XDATA.pyconf
+    cmake.pyconf           freeimage.pyconf          HYBRIDPLUGIN.pyconf        NETGENPLUGIN.pyconf     pyhamcrest.pyconf    solvespace.pyconf                YACSGEN.pyconf
+    colorama.pyconf        freetype.pyconf           idna.pyconf                netgen.pyconf           PYHELLO.pyconf       sphinxcontrib_napoleon.pyconf    YACS.pyconf
+    compil_scripts         ftgl.pyconf               imagesize.pyconf           nlopt.pyconf            pyparsing.pyconf     sphinxcontrib.pyconf             zlib.pyconf
+    COMPONENT.pyconf       functools32.pyconf        ispc.pyconf                numpy.pyconf            PyQt.pyconf          sphinxcontrib_websupport.pyconf
+    CONFIGURATION.pyconf   GEOM.pyconf               Jinja2.pyconf              omniNotify.pyconf       pyreadline.pyconf    sphinxintl.pyconf
+
+
+Available product configuration flags
+-------------------------------------
+
+* **name** : the name of the product 
+* **build_source** : the method to use when getting the sources, possible choices are script/cmake/autotools. If "script" is chosen, a compilation script should be provided with compil_script key
+* **compil_script** : to specify a compilation script (in conjonction with build_source set to "script"). The programation langage is bash under linux, and bat under windows.  
+* **get_source** : the mode to get the sources, possible choices are archive/git/svn/cvs
+* **depend** : to give SAT the dependencies of the product
+* **patches** : provides a list of patches, if required
+* **source_dir** : where SAT copies the source
+* **build_dir** : where SAT builds the product
+* **install_dir** : where SAT installs the product
+
+The following example is the configuration of boost product:
+
+.. code-block:: bash
+
+    default :
+    {
+        name : "boost"
+        build_source : "script"
+        compil_script :  $name + $VARS.scriptExtension
+        get_source : "archive"
+        environ :
+        {
+           env_script : $name + ".py"
+        }
+        depend : ['Python' ]
+        opt_depend : ['openmpi' ]
+        patches : [ ]
+        source_dir : $APPLICATION.workdir + $VARS.sep + 'SOURCES' + $VARS.sep + $name
+        build_dir : $APPLICATION.workdir + $VARS.sep + 'BUILD' + $VARS.sep + $name
+        install_dir : 'base'
+        properties :
+        {
+            single_install_dir : "yes"
+            incremental : "yes"
+        }
+    }
+
+
+Product properties
 ------------------
-| This section is defined in the application pyconf file.
-| It contains instructions on how to build a version of SALOME (list of products and versions, compilation options, etc.)
 
-:: 
+Properties are also associated to products.
+It is possible to list all the properties with the command *./sat config SALOME-9.4.0 --show_properties**
 
-    # to get the current setting
-    sat config SALOME-xx --value APPLICATION
+Here are some properties frequently used:
 
-PRODUCTS section
----------------------
-| This section contains all the information required to build the products contained in the application.
-| It is build from the products configuration files.
+* **single_install_dir* : the product can be installed in a common directory 
+* **compile_time* : the product is used only at compile time (ex : swig)
+* **pip* : the product is managed by pip
+* **not_in_package** : the product will not be put in packages
+* **is_SALOME_module** : te product is a SALOME module
+* **is_distene** : the product requires a DISTENE licence
 
-:: 
+The product properties allow SAT doing specific choices according to the property.
+They also allow users filtering products when calling commands.
+For example it is possible to compile only SALOME modules with the command:
 
-    # to get the current setting
-    sat config SALOME-xx --value PRODUCT
+.. code-block:: bash
+
+    # just recompile SALOME modules, not other products
+    ./sat compile SALOME-9.4.0 --properties is_SALOME_module:yes --clean_all
+
+
+Product environment
+-------------------
+
+The product environment is declared in a subsection called environment.
+It is used by sat at compile time the set up the environment for the compilation of all the products depending upon it.
+It is also used at run tim to set up the application environment.
+
+Two mecanisms are offered to define the environment.
+The first one is similar to the one used in the application configuration : inside the environ section, we declare variables or paths.
+A variable appended or prepended by an underscore is treated as a path, to which we prepend or append the valued according to the position of the underscore.
+In the above example, the value *<install_dir/share/salome/ressources/salome* is prepended to the path SalomeAppConfig.
+
+.. code-block:: bash
+
+    environ :
+    {
+        _SalomeAppConfig : $install_dir + $VARS.sep + "share" + $VARS.sep + "salome" + $VARS.sep + "resources" + $VARS.sep + "salome"
+    }
+
+
+But the most common way is to use a environment script, which specify the environment by using an API provided by sat: 
+
+.. code-block:: bash
+
+    # use script qt.py to set up qt environement
+    environ :
+    {
+       env_script : "qt.py"
+    }
+
+As an example, the environment script for qt is:
+
+.. code-block:: python
+
+    #!/usr/bin/env python
+    #-*- coding:utf-8 -*-
 
+    import os.path
+    import platform
+
+    def set_env(env, prereq_dir, version):
+        env.set('QTDIR', prereq_dir)
+
+        version_maj = version.split('.')
+        if version_maj[0] == '5':
+            env.set('QT5_ROOT_DIR', prereq_dir)
+            env.prepend('QT_PLUGIN_PATH', os.path.join(prereq_dir, 'plugins'))
+            env.prepend('QT_QPA_PLATFORM_PLUGIN_PATH', os.path.join(prereq_dir, 'plugins'))
+            pass
+        else:
+            env.set('QT4_ROOT_DIR', prereq_dir)
+            pass
+
+        env.prepend('PATH', os.path.join(prereq_dir, 'bin'))
+
+        if platform.system() == "Windows" :
+            env.prepend('LIB', os.path.join(prereq_dir, 'lib'))
+            pass
+        else :
+            env.prepend('LD_LIBRARY_PATH', os.path.join(prereq_dir, 'lib'))
+            pass
+
+*env* is the API provided by SAT, prereq_dir is the installation directory, version the product version.
+env.set set a variable, env.prepend and env.append are used to prepend or append values to a path.
+
+The **setenv** function is used to set the environement at compile time and run time.
+It is also possible to use **set_env_build** and **set_env_launch** callback functions to set specific compile or run time environment.
+Finally the functions **set_nativ_env** is used for native products.
+
+
+Product sections
+----------------
+
+The product configuration file may contain several sections.
+In addition to the  "default" section, it is possible to declare other section that will be used for specific versions of the product.
+This allows SAT compiling different version of a products.
+To determine which section should be used, SAT has an algorithm that takes into account the version number.
+Here are some examples of sections that will be taken into account by SAT :
+
+.. code-block:: bash
+
+    # this section will be used for versions between 8.5.0 and 9.2.1
+    _from_8_5_0_to_9_2_1 :
+    {
+        ...
+    }
+
+    # this section will only ve used for 9.3.0 version
+    version_9_3_0 :
+    {
+        ...
+    }
+
+Several version numbering are considered by SAT (not only X.Y.Z)
+For example V9, v9, 9, 9.0.0, 9_0_0, are acepted. 
+
+By default SAT only consider one section : the one determined according to the version number, or the default one.
+But if the **incremental property** is defined in the default section, and is set to "yes", then SAT enters in the **incremental mode** and merges different sections into one,
+by proceeding incremental steps. SAT uses the following algorithm to merge the sections:
+
+#. We take the complete "default" section
+#. if a "default_win" section is defined, we merge it.
+#. If a section name correspond to the version number, we also merge it.
+#. Finally on windows platform if the same section name appended by _win exists, we merge it.
+
+
+Other configuration sections
+============================ 
+
+
+The configuration of SAT is split into height sections : VARS, APPLICATION, PRODUCTS, PROJECTS, PATHS, USER, LOCAL, INTERNAL.
+These sections are feeded by the pyconf files which are loaded by sat: each pyconf file is parsed by SAT and merged into the global configuration.
+One file can reference variables defined in other files. Files are loaded in this order :
+
+* the internal pyconf (declared inside sat)
+* the personal pyconf : *~/.salomeTools/SAT.pyconf*
+* the application pyconf
+* the products pyconf (for all products declared in the application)
+
+In order to check the configuration and the merge done by sat, it is possible to display the resulting height section with the command:
+
+.. code-block:: bash
+
+    # display the content of a configuration section 
+    # (VARS, APPLICATION, PRODUCTS, PROJECTS, PATHS, USER, LOCAL, INTERNAL)
+    SAT/sat config SALOME-9.4.0 -v <section>
+
+Note also that if you don't remember the name of a section it is possible to display section names with the automatic completion functionality.
+
+We have already described two of the sections : APPLICATION and PRODUCTS.
+Let's describe briefly the six others
+
+.. _VARS-Section:
+
+VARS section
+-------------
+| This section is dynamically created by SAT at run time.
+| It contains information about the environment: date, time, OS, architecture etc. 
+
+::
+
+    # to get the current setting
+    sat config --value VARS
 
-.. _USER-Section:
 
 USER section
 --------------
+
 This section is defined by the user configuration file, 
 ``~/.salomeTools/SAT.pyconf``.
 
@@ -88,21 +438,50 @@ The ``USER`` section defines some parameters (not exhaustive):
 Other sections
 --------------
 
-* **PROJECTs** : This section contains the configuration of the projects loaded in salomeTool by sat init --add_project command. 
+* **PROJECTs** : This section contains the configuration of the projects loaded in SAT by *sat init --add_project* command. 
 * **PATHS** : This section contains paths used by saloeTools.
-* **LOCAL** : contains information relative to the local installation of salomeTool.
-* **INTERNAL** : contains internal salomeTool information
+* **LOCAL** : contains information relative to the local installation of SAT.
+* **INTERNAL** : contains internal SAT information
 
-All these sections can be printed with sat config command:
 
-::
+Overwriting the configution
+===========================
+
+At the end of the process, SAT ends up with a complete global configuration resulting from the parsing of all *.pyconf* files.
+It may be interesting to overwrite the configuration.
+SAT offer two overwriting mecanism to answer these two use cases:
+
+#. Be able to conditionaly modify the configuration of an application to take into account specifics and support multi-platform builds
+#. Be able to modify the configuration in the command line, to enable or disable some options at run time
+
+Application overwriting
+-----------------------
+
+At the end of the application configuration, it is possible to define an overwrite section with the keyword **__overwrite__ :**.
+It is followed by a list overwrite sections, that may be conditionnal (use of the keyword **__condition__ :**).
+A classical usage of the application overwriting is the change of a prerequisite version for a given platform (when the default version do not compile).
+/bin/bash: q : commande introuvable
+
+.. code-block:: bash
+
+    __overwrite__ :
+    [
+      {
+       # opencv 3 do not compile on old CO6
+        __condition__ : "VARS.dist in ['CO6']"
+        'APPLICATION.products.opencv' : '2.4.13.5'
+      }
+    ]
+
+
+Command line overwriting
+------------------------
+
+Command line overwriting is triggered by sat **-o** option, followed in double quotes by the parameter to overwrite, the = sign and the value in simple quotes.
+In the following example, we suppose that the application SALOME-9.4.0 has set both flags debug and verbose to "no", and that we want to recompile MEDCOUPLING in debug mode, with cmake verbosity activated. The command to use is:
 
-    # It is possible to use sat completion mode to print available sections.
-    sat config SALOME-xx --value <TAB> <TAB>
-    > APPLICATION.       INTERNAL.          LOCAL.             PATHS. 
-    > PRODUCTS.          PROJECTS.          USER.              VARS.
+.. code-block:: bash
 
-    # get paths used by sat
-    sat config SALOME-xx --value PATHS
+    # recompile MEDCOUPLING in debug mode (-g) and with verbosity
+    ./sat -t -o "APPLICATION.verbose='yes'" -o "APPLICATION.debug='yes'" compile SALOME-9.4.0 -p MEDCOUPLING --clean_all
 
-It is possible to use sat completion mode to print available sections.