From 8de57afec087347ead374b04e056c72118a61f71 Mon Sep 17 00:00:00 2001 From: crouzet Date: Thu, 28 Nov 2019 17:10:35 +0100 Subject: [PATCH] sat #8544 : doc improvement --- doc/rst_prolog.rst | 1 - doc/src/configuration.rst | 486 +++++++++++++++--- doc/src/index.rst | 55 +- doc/src/installation_of_sat.rst | 35 +- doc/src/release_notes/release_notes_5.5.0.rst | 24 +- doc/src/usage_of_sat.rst | 195 ++++--- doc/src/write_command.rst | 5 +- 7 files changed, 642 insertions(+), 159 deletions(-) diff --git a/doc/rst_prolog.rst b/doc/rst_prolog.rst index 55ae5ae..43d943b 100644 --- a/doc/rst_prolog.rst +++ b/doc/rst_prolog.rst @@ -17,7 +17,6 @@ .. _Git: https://git-scm.com .. _GitCheatSheet: https://services.github.com/kit/downloads/github-git-cheat-sheet.pdf .. _SatCheatSheet: ../sat-cheat-sheet.pdf -.. _SatPdf: ./../latex/salomeTools.pdf .. _CLI: https://en.wikipedia.org/wiki/Command-line_interface .. _Tar: https://en.wikipedia.org/wiki/Tar_(computing) .. _OS: https://en.wikipedia.org/wiki/Operating_system diff --git a/doc/src/configuration.rst b/doc/src/configuration.rst index 4f1ec16..c3c6b96 100644 --- a/doc/src/configuration.rst +++ b/doc/src/configuration.rst @@ -2,80 +2,417 @@ Configuration ************* -*salomeTools* uses files with **.pyconf** extension to store its configuration parameters. -These pyconf configuration files are provided by the salomeTool projects that are set by sat init command. +Introduction +============ -When executing a command, sat will load several configuration files in a specific order. -When all the files are loaded a *config* object is created. -Then, this object is passed to all command scripts. +For the configuration, SAT uses a python module called *config*, which aims to offer more power and flexibility for the configuration of python programs. +This module was slightly adapted for SAT, and renamed Pyconf. +(see `config module `_ for a complete description of the module, the associated syntax, the documentation). +*salomeTools* uses files with **.pyconf** extension to store the configuration parameters. +These *.pyconf* are parsed by SAT, and merged into a global configuration, which is passed to the sat commands and used by them. -Syntax -====== -The configuration files use a python-like structure format -(see `config module `_ for a complete description). -* **{}** define a dictionary, -* **[]** define a list, -* **@** can be used to include a file, -* **$prefix** reference to another parameter (ex: ``$PRODUCT.name``), -* **#** comments. +Configuration projects +====================== -.. note:: in this documentation a reference to a configuration parameter will be noted ``XXX.YYY``. +By default SAT is provided with no configuration at all, except is own internal one. +The configuration is brought by SAT projects : usually a git base containing all the configuration files of a project (*.pyconf* files). +For Salome platform, the SAT project is called SAT_SALOME and can be dowloaded from salome Tuleap forge. +SAT projects are loaded in sat with the sat init command: -Description -=========== +.. code-block:: bash -.. _VARS-Section: + # get salome platform SAT configuration project (SAT_SALOME), and load it into SAT + git clone SAT_SALOME + SAT/sat init --add_project $(pwd)/SAT_SALOME/salome.pyconf -VARS section -------------- -| This section is dynamically created by salomeTools at run time. -| It contains information about the environment: date, time, OS, architecture etc. +SAT_SALOME project provides all configuration files for salome applications, and for the products that are used in these applications. -:: - # to get the current setting - sat config --value VARS +Application configuration +========================= + +The configuration files of applications contain the required information for SAT to build the application. +They are usually located in the application directory of the project: + +.. code-block:: bash + + # list applications provided by SAT_SALOME project + ls SAT_SALOME/applications + MEDCOUPLING-9.4.0.pyconf SALOME-7.8.0.pyconf SALOME-8.5.0.pyconf SALOME-9.4.0.pyconf + +The application configuration file defines the APPLICATION sections. The content of this section can be displayed with *sat config* command: + +.. code-block:: bash + + sat config SALOME-9.4.0 -v APPLICATION + +SAT users that need to create new application files for their own purpose usually copy an existing configuration file and adapt it to their application. +Let's discribe the content of an application pyconf file. We take in the following examples the file *SAT_SALOME/applications/SALOME-9.4.0.pyconf*. + + +Global variables and flags +-------------------------- + +At the beginning of the APPLICATION sections, global variables and flags are defined: + + * **name** : the name of the application (mandatory) + * **workdir** : the directory in which the application is produced (mandatory) + * **tag** : the default tag to use for the git bases + * **dev** : activate the dev mode. in dev mode git bases are checked out only one time, to avoid risks of removing developments. + * **verbose** : activate verbosity in the compilation + * **debug** : activate debug mode in the compilation, i.e -g option + * **python3** : 'yes/no' tell sat that the application uses python3 + * **base** : 'yes/no/name' to set up the use of a SAT base + +.. code-block:: bash + + APPLICATION : + { + name : 'SALOME-9.4.0' + workdir : $LOCAL.workdir + $VARS.sep + $APPLICATION.name + '-' + $VARS.dist + tag : 'V9_4_BR' + debug : 'no' + dev : 'no' + base : 'no' + python3 : 'yes' + ... + +Please note the workdir variable is defined in the above example with references to other sections defined in other configurations files (i.e. $LOCAL and $VARS). +It's a useful Pyconf functionality). +Most of the global variables are optionnal, except name and workdir. + +Environment subsection +---------------------- + +This subsection allows defining environment variables at the application level (most of the time the environment is set by the products configuration). -APPLICATION section +.. code-block:: bash + + APPLICATION : + { + ... + environ : + { + build : {CONFIGURATION_ROOT_DIR : $workdir + $VARS.sep + "SOURCES" + $VARS.sep + "CONFIGURATION"} + launch : {PYTHONIOENCODING:"UTF_8"} + SALOME_trace : "local" # local/file:.../with_logger + SALOME_MODULES : "SHAPER,GEOM,SMESH,PARAVIS,YACS,JOBMANAGER" # specify the first modules to display in gui + } + } + +In the example above CONFIGURATION_ROOT_DIR variable will be set only at compile time (usage of *build* key), while PYTHONIOENCODING will be set only at run-time (use of *launch* key). +variables SALOME_trace and SALOME_MODULES are set both at compile time and run time. + + +products subsection +------------------- + +This subsection will specify which products are included in the application. +For each product, it is possible to specify in a dictionnary: + + * **tag** : the tag to use for the product + * **dev** : activate the dev mode. + * **verbose** : activate verbosity in the compilation + * **debug** : activate debug mode + +If this flags are not specified, SAT takes the default application flag. +In the following example, SAT uses the the default tag V9_4_BR for products SHAPER, KERNEL and MEDCOUPLING. +For LIBBATCH it uses the tag V2_4_2. +KERNEL is compile in debug and verbose mode. + +.. code-block:: bash + + APPLICATION : + { + ... + tag : 'V9_4_BR' + ... + products : + { + 'SHAPER' + 'LIBBATCH' : {tag :'V2_4_2'} + 'KERNEL' : {debug:'yes', verbose:'yes'} + 'MEDCOUPLING' + ... + + +properties +---------- + +Properties are used by SAT to define some general rules or policies. +They can be defined in the application configuration with the properties subsection: + +.. code-block:: bash + + APPLICATION : + { + ... + properties : + { + mesa_launcher_in_package : "yes" + repo_dev : "yes" + pip : 'yes' + pip_install_dir : 'python' + } + +In this example the following properties are used: + + * **mesa_launcher_in_package** : ask to put a mesa launcher in the packages produced by sat package commans + * **repo_dev** : use the development git base (for salome, the tuleap forge) + * **pip** : ask to use pip to get python products + * **pip_install_dir** : install pip products in python installation directory (not in separate directories) + + +Products configuration +====================== + +The configuration files of products contain the required information for SAT to build each product. +They are usually located in the product directory of the project. SAT_SALOME supports a lot of products: + +.. code-block:: bash + + ls SAT_SALOME/products/ + ADAO_INTERFACE.pyconf COREFLOWS_PROFILE.pyconf GHS3DPLUGIN.pyconf JOBMANAGER.pyconf omniORB.pyconf Python.pyconf Sphinx.pyconf + ADAO_MODULE.pyconf COREFLOWS.pyconf GHS3DPRLPLUGIN.pyconf KERNEL.pyconf omniORBpy.pyconf pytz.pyconf sphinx_rtd_theme.pyconf + ADAO.pyconf cppunit.pyconf gl2ps.pyconf kiwisolver.pyconf opencv.pyconf qt.pyconf subprocess32.pyconf + alabaster.pyconf cycler.pyconf glu.pyconf lapack.pyconf openmpi.pyconf qwt.pyconf swig.pyconf + ALAMOS_PROFILE.pyconf Cython.pyconf GMSHPLUGIN.pyconf lata.pyconf ospray.pyconf requests.pyconf tbb.pyconf + ALAMOS.pyconf dateutil.pyconf gmsh.pyconf LIBBATCH.pyconf packaging.pyconf SALOME_FORMATION_PROFILE.pyconf tcl.pyconf + Babel.pyconf distribute.pyconf graphviz.pyconf libpng.pyconf ParaViewData.pyconf SALOME_PROFILE.pyconf TECHOBJ_ROOT.pyconf + BLSURFPLUGIN.pyconf DOCUMENTATION.pyconf GUI.pyconf libxml2.pyconf ParaView.pyconf SALOME.pyconf tk.pyconf + boost.pyconf docutils.pyconf hdf5.pyconf llvm.pyconf PARAVIS.pyconf SAMPLES.pyconf Togl.pyconf + bsd_xdr.pyconf doxygen.pyconf HELLO.pyconf markupsafe.pyconf ParMetis.pyconf scipy.pyconf TRIOCFD_IHM.pyconf + CALCULATOR.pyconf EFICAS.pyconf HEXABLOCKPLUGIN.pyconf matplotlib.pyconf patches scons.pyconf TrioCFD.pyconf + CAS.pyconf EFICAS_TOOLS.pyconf HEXABLOCK.pyconf MEDCOUPLING.pyconf petsc.pyconf scotch.pyconf TRUST.pyconf + CDMATH.pyconf eigen.pyconf HexoticPLUGIN.pyconf medfile.pyconf planegcs.pyconf setuptools.pyconf typing.pyconf + CEATESTBASE.pyconf embree.pyconf Hexotic.pyconf med_pre_windows.pyconf pockets.pyconf SHAPER.pyconf uranie_win.pyconf + certifi.pyconf env_scripts homard_bin.pyconf MED.pyconf pthreads.pyconf sip.pyconf urllib3.pyconf + cgns.pyconf expat.pyconf homard_pre_windows.pyconf mesa.pyconf PY2CPP.pyconf six.pyconf VISU.pyconf + chardet.pyconf f2c.pyconf HOMARD.pyconf MeshGems.pyconf PYCALCULATOR.pyconf SMESH.pyconf vtk.pyconf + click.pyconf FIELDS.pyconf HXX2SALOME.pyconf metis.pyconf Pygments.pyconf snowballstemmer.pyconf XDATA.pyconf + cmake.pyconf freeimage.pyconf HYBRIDPLUGIN.pyconf NETGENPLUGIN.pyconf pyhamcrest.pyconf solvespace.pyconf YACSGEN.pyconf + colorama.pyconf freetype.pyconf idna.pyconf netgen.pyconf PYHELLO.pyconf sphinxcontrib_napoleon.pyconf YACS.pyconf + compil_scripts ftgl.pyconf imagesize.pyconf nlopt.pyconf pyparsing.pyconf sphinxcontrib.pyconf zlib.pyconf + COMPONENT.pyconf functools32.pyconf ispc.pyconf numpy.pyconf PyQt.pyconf sphinxcontrib_websupport.pyconf + CONFIGURATION.pyconf GEOM.pyconf Jinja2.pyconf omniNotify.pyconf pyreadline.pyconf sphinxintl.pyconf + + +Available product configuration flags +------------------------------------- + +* **name** : the name of the product +* **build_source** : the method to use when getting the sources, possible choices are script/cmake/autotools. If "script" is chosen, a compilation script should be provided with compil_script key +* **compil_script** : to specify a compilation script (in conjonction with build_source set to "script"). The programation langage is bash under linux, and bat under windows. +* **get_source** : the mode to get the sources, possible choices are archive/git/svn/cvs +* **depend** : to give SAT the dependencies of the product +* **patches** : provides a list of patches, if required +* **source_dir** : where SAT copies the source +* **build_dir** : where SAT builds the product +* **install_dir** : where SAT installs the product + +The following example is the configuration of boost product: + +.. code-block:: bash + + default : + { + name : "boost" + build_source : "script" + compil_script : $name + $VARS.scriptExtension + get_source : "archive" + environ : + { + env_script : $name + ".py" + } + depend : ['Python' ] + opt_depend : ['openmpi' ] + patches : [ ] + source_dir : $APPLICATION.workdir + $VARS.sep + 'SOURCES' + $VARS.sep + $name + build_dir : $APPLICATION.workdir + $VARS.sep + 'BUILD' + $VARS.sep + $name + install_dir : 'base' + properties : + { + single_install_dir : "yes" + incremental : "yes" + } + } + + +Product properties ------------------ -| This section is defined in the application pyconf file. -| It contains instructions on how to build a version of SALOME (list of products and versions, compilation options, etc.) -:: +Properties are also associated to products. +It is possible to list all the properties with the command *./sat config SALOME-9.4.0 --show_properties** - # to get the current setting - sat config SALOME-xx --value APPLICATION +Here are some properties frequently used: +* **single_install_dir* : the product can be installed in a common directory +* **compile_time* : the product is used only at compile time (ex : swig) +* **pip* : the product is managed by pip +* **not_in_package** : the product will not be put in packages +* **is_SALOME_module** : te product is a SALOME module +* **is_distene** : the product requires a DISTENE licence -Globals +The product properties allow SAT doing specific choices according to the property. +They also allow users filtering products when calling commands. +For example it is possible to compile only SALOME modules with the command: - * name : the name of the application - * workdir : the directory in which the application is produced - * tag : the default tag to use for the git bases - * dev : activate the dev mode. in dev mode git bases are checked out only one time, to avoid risks of removing developments. - * verbose : activate verbosity in the compilation - * debug : activate debug mode in the compilation, i.e -g option - * python3 : 'yes/no' tell sat that the application uses python3 - * base : 'yes/no/name' +.. code-block:: bash + # just recompile SALOME modules, not other products + ./sat compile SALOME-9.4.0 --properties is_SALOME_module:yes --clean_all -PRODUCTS section ---------------------- -| This section contains all the information required to build the products contained in the application. -| It is build from the products configuration files. -:: +Product environment +------------------- - # to get the current setting - sat config SALOME-xx --value PRODUCT +The product environment is declared in a subsection called environment. +It is used by sat at compile time the set up the environment for the compilation of all the products depending upon it. +It is also used at run tim to set up the application environment. + +Two mecanisms are offered to define the environment. +The first one is similar to the one used in the application configuration : inside the environ section, we declare variables or paths. +A variable appended or prepended by an underscore is treated as a path, to which we prepend or append the valued according to the position of the underscore. +In the above example, the value * + +Note also that if you don't remember the name of a section it is possible to display section names with the automatic completion functionality. + +We have already described two of the sections : APPLICATION and PRODUCTS. +Let's describe briefly the six others + +.. _VARS-Section: + +VARS section +------------- +| This section is dynamically created by SAT at run time. +| It contains information about the environment: date, time, OS, architecture etc. + +:: + + # to get the current setting + sat config --value VARS -.. _USER-Section: USER section -------------- + This section is defined by the user configuration file, ``~/.salomeTools/SAT.pyconf``. @@ -101,21 +438,50 @@ The ``USER`` section defines some parameters (not exhaustive): Other sections -------------- -* **PROJECTs** : This section contains the configuration of the projects loaded in salomeTool by sat init --add_project command. +* **PROJECTs** : This section contains the configuration of the projects loaded in SAT by *sat init --add_project* command. * **PATHS** : This section contains paths used by saloeTools. -* **LOCAL** : contains information relative to the local installation of salomeTool. -* **INTERNAL** : contains internal salomeTool information +* **LOCAL** : contains information relative to the local installation of SAT. +* **INTERNAL** : contains internal SAT information -All these sections can be printed with sat config command: -:: +Overwriting the configution +=========================== + +At the end of the process, SAT ends up with a complete global configuration resulting from the parsing of all *.pyconf* files. +It may be interesting to overwrite the configuration. +SAT offer two overwriting mecanism to answer these two use cases: + +#. Be able to conditionaly modify the configuration of an application to take into account specifics and support multi-platform builds +#. Be able to modify the configuration in the command line, to enable or disable some options at run time + +Application overwriting +----------------------- + +At the end of the application configuration, it is possible to define an overwrite section with the keyword **__overwrite__ :**. +It is followed by a list overwrite sections, that may be conditionnal (use of the keyword **__condition__ :**). +A classical usage of the application overwriting is the change of a prerequisite version for a given platform (when the default version do not compile). +/bin/bash: q : commande introuvable + +.. code-block:: bash + + __overwrite__ : + [ + { + # opencv 3 do not compile on old CO6 + __condition__ : "VARS.dist in ['CO6']" + 'APPLICATION.products.opencv' : '2.4.13.5' + } + ] + + +Command line overwriting +------------------------ + +Command line overwriting is triggered by sat **-o** option, followed in double quotes by the parameter to overwrite, the = sign and the value in simple quotes. +In the following example, we suppose that the application SALOME-9.4.0 has set both flags debug and verbose to "no", and that we want to recompile MEDCOUPLING in debug mode, with cmake verbosity activated. The command to use is: - # It is possible to use sat completion mode to print available sections. - sat config SALOME-xx --value - > APPLICATION. INTERNAL. LOCAL. PATHS. - > PRODUCTS. PROJECTS. USER. VARS. +.. code-block:: bash - # get paths used by sat - sat config SALOME-xx --value PATHS + # recompile MEDCOUPLING in debug mode (-g) and with verbosity + ./sat -t -o "APPLICATION.verbose='yes'" -o "APPLICATION.debug='yes'" compile SALOME-9.4.0 -p MEDCOUPLING --clean_all -It is possible to use sat completion mode to print available sections. diff --git a/doc/src/index.rst b/doc/src/index.rst index 694f12e..0d4276a 100644 --- a/doc/src/index.rst +++ b/doc/src/index.rst @@ -10,32 +10,44 @@ Salome Tools ************ -.. image:: images/sat_about.png - :scale: 100 % - :align: center +Welcome +======= + +**SAT** is a tool that makes it easy to build on various linux platforms and windows large software, which rely on a lot of prerequisites. +It was originally created for the maintenance and the packaging of SALOME platform (its name comes from **Sa**\ lome\ **T**\ ools), its usage is now wider. +The following features should be highlighted: + + the **definition** of the application content: which products (prerequisite) are necessary and which version are required + the **configuration** of the application : how to get the source of products, how to compile them, which options to use, etc. The configuration can be conditionnaly overwritten, it allows application developers taking into account platform specifics. + the **preparation** of the complete software: all the required sources with correct versions are retrieved from git/svn/cvs repositories, or from already prepared tarballs. + management of **patches** if some are required to compile on specific platforms (portage) + management of the **environment** to set up at compile time and at runtime + automatic **compilation** of the complete application (the application with all its products). + production of a **launcher** that sets up the environment and starts the application + management of **tests**: both unit and integration tests are managed + **packaging**: creation of binary and/or source packages to distribute the application + **overwriting** the configuration in command line: it allows users setting their own preferences or options + +SAT uses **python**, and many of its strength come from the power and straightforwardness of this tool. +sat is a Command Line Interface (CLI_) based on python langage. It is a suite of commands, which are detailed later in this documentation. +These commands are used to perform the operations on the application. -The **Sa**\ lome\ **T**\ ools (sat) is a suite of commands -that can be used to perform operations on SALOME_. - -For example, sat allows you to compile SALOME's codes -(prerequisites, products) -create application, run tests, create package, etc. - -This utility code is a set of Python_ scripts files. -Find a `pdf version of this documentation `_ +Documentation +============= +.. toctree:: + :maxdepth: 2 -Quick start -=========== + Installation of SAT + Using SAT + Configuration .. toctree:: :maxdepth: 1 - Installation of salomeTools - Usage of salomeTools - Configuration + Extending SAT with additional commands List of Commands ================ @@ -57,15 +69,6 @@ List of Commands init environ -Developer documentation -======================= - -.. toctree:: - :maxdepth: 1 - - Add a command - - Release Notes ============= diff --git a/doc/src/installation_of_sat.rst b/doc/src/installation_of_sat.rst index 5bd988b..ba7a3bc 100644 --- a/doc/src/installation_of_sat.rst +++ b/doc/src/installation_of_sat.rst @@ -2,7 +2,40 @@ Installation ************ -**sat** is provided either embedded into a salome package, or as a standalone package. +**sat** is provided either embedded into a salome package, or as a standalone package. He can also be retrieved from the git repositories. + + +From git bases +-------------- + +**sat** git bases are hosted by the `salome platftorm Tuleap forge `_ . Therefore you first has to get an account to this forge. +To get started, one has to download sat, and at last one sat project (usually SAT_SALOME project, which contains all the configuration required to build SALOME and its prerequisites). The following script get sat and SAT_SALOME project from git repos: :: + + # get sat + BASE_SAT=https://codev-tuleap.cea.fr/plugins/git/spns/SAT.git + BASE_PROJET=https://codev-tuleap.cea.fr/plugins/git/spns/SAT_SALOME.git + TAG_SAT=master + TAG_PROJET=master + git clone ${BASE_SAT} + cd SAT + git checkout ${TAG_SAT} + cd .. + + # get sat project SAT_SALOME + git clone ${BASE_PROJET} + cd SAT_SALOME + git checkout ${TAG_PROJET} + cd .. + + # initialisation de sat + + # add SAT_SALOME project to sat, other configurations projects can be added + SAT/sat init --add_project $(pwd)/SAT_SALOME/salome.pyconf + + # record tag and url (not mandatory) + SAT/sat init --VCS $BASE_SAT + SAT/sat init --tag $(git describe --tags) + Embedded sat version -------------------- diff --git a/doc/src/release_notes/release_notes_5.5.0.rst b/doc/src/release_notes/release_notes_5.5.0.rst index f2c4262..a8e6191 100644 --- a/doc/src/release_notes/release_notes_5.5.0.rst +++ b/doc/src/release_notes/release_notes_5.5.0.rst @@ -10,6 +10,8 @@ Release Notes, November 2019 New features and improvments ---------------------------- + + **pip mode for python modules** This new mode was introduced in order to simplify the management of the python products (which number is constantly raising years after years...) @@ -111,6 +113,16 @@ Here is as an example the incremental definition used for boost products. For ve patches : [ "boost-1.49.0.patch" ] } +** Suppression of the global "no_base" flag in application configuration** + +**no_base : "no"** is not interpreted anymore in application pyconf. +One has to use the **base** flag. +The possible values are: + +* **yes** : all the products go into the base +* **no** : no product goes into the base + +The complete usage rule of bases is explaned in the documentation. Change log @@ -123,6 +135,8 @@ most significant changes are listed. +-------------+-----------------------------------------------------------------------------------+ | Artifact | Description | +=============+===================================================================================+ +| spns #8544 | The documentation has been improved! | ++-------------+-----------------------------------------------------------------------------------+ | spns #16894 | clean the temp directory at the end of sat package | +-------------+-----------------------------------------------------------------------------------+ | sat #12965 | optimisation of sat compile : better, simplier and faster algo for dependencies! | @@ -137,7 +151,7 @@ most significant changes are listed. | sat # 8547 | Generalisation of sat launcher command (new option --exe to specify which exe | | | should be launched after setting the environment | +-------------+-----------------------------------------------------------------------------------+ -| sat #17357 | New field in application pyconf "rm_rpoducts" to blacklist some products | +| sat #17357 | New field "rm_products" to blacklist products in overwrite section of appli pyconf| +-------------+-----------------------------------------------------------------------------------+ | sat #17194 | Parametrication of the value of INSTALL and BINARIES directories | | | (in src/internal_config/salomeTools.pyconf) | @@ -146,5 +160,11 @@ most significant changes are listed. +-------------+-----------------------------------------------------------------------------------+ | sat #17359 | New incremental mode for the definition of products | +-------------+-----------------------------------------------------------------------------------+ -| | | +| sat #17766 | The environment of products is now loaded in the order of product dependencies. | +| sat #17848 | To treat correctly dependencies in the environment | ++-------------+-----------------------------------------------------------------------------------+ +| sat #17955 | No unit tests for native products | ++-------------+-----------------------------------------------------------------------------------+ +| | SAT_DEBUG and SAT_VERBOSE environment variables are now available in the | +| | compilation, which can now forward the information and do the job! | +-------------+-----------------------------------------------------------------------------------+ diff --git a/doc/src/usage_of_sat.rst b/doc/src/usage_of_sat.rst index 3bc67e3..0bdb315 100644 --- a/doc/src/usage_of_sat.rst +++ b/doc/src/usage_of_sat.rst @@ -1,50 +1,61 @@ .. include:: ../rst_prolog.rst -************************* -Main usage of SAlomeTools -************************* +********* +Using SAT +********* -Purpose, Command Line Interface -=============================== +Getting started +=============== sat is a Command Line Interface (CLI_) based on python langage. -Its purpose is to cover the maintenance and the production of the salome platform and its applications. +Its purpose is to cover the maintenance and the production of an application which run on several platforms and depend upon a lot of prerequisites. +It is most of the time used interactively from a terminal, but there is also a batch mode that can be used for example in automatic procedures (like jenkins jobs). +sat is used in command line by invoking after its name a sat option (which is non mandatory), then a command name, followed by the arguments of the command (most of the time the name of an application and command options): -Notably: - -* the definition of the applications content (the products, the resources, the options, the environment, the launcher, etc.) -* the description of the products (the environment to set, how to get the sources; how to compilation (which options), , the dependencies, etc). -* the complete preparation and build -* the management of unit or integration tests -* the production of binary or source packages +.. code-block:: bash -It is designed to run on several Linux OS and Windows, with -the possibility to specify platform dependent specifics (with the **__overwrite__** functionality. -It can be used from interactively from a terminal, or in batch mode. + ./sat [generic_options] [command] [application] [command_options] -.. code-block:: bash - sat [generic_options] [command] [application] [command_options] +The main sat options are: +* **-h** : to get the list of available options and commands +* **-o** : to overwrite at runtime a configuration parameter or option +* **-v** : to change the verbosity (default is 3, minimum 0 and maximum 6) +* **-b** : to enter the batch mode and avoid any question (this non interactive mode is useful for automatic procedures like jenkins jobs) +* **-t** : to display the compilation logs in the terminal (otherwise they are logged in files and displayed by the log command +The main sat commands are: + +* **prepare** : to get the sources of the application products (from git repositories or archives) and apply patches if there are any +* **compile** : to build the application (using cmake, automake or shell script) +* **launcher** : to generate a launcher of the application (in the most general case the launcher sets up the run-time environment and start an exe) +* **package** : to build a package of the application (binary and/or source) +* **config** : to display the configuration +* **log** : to display within a web browser the logs of the commands + + Getting help ============ Help option -h -------------- -To get help in terminal mode as simple text, use **--help** or **-h** option: +More details are provided by the help of sat. The help option can ba calle at two levels : the (high) level displays information on how to use sat, the command level displays information on how to use a sat command. .. code-block:: bash - sat -h # or --help : get the list of existing commands and main options - sat compile -h # get the help on the specific command 'compile' + # display sat help + ./sat -h + + # display the help of the compile command + ./sat compile -h Completion mode --------------- -When getting started with sat, the use of the competion mode is usefull. This mode will display by type twice on the **tab key** the available options, command, applications or product available. The completion mode has to be activated by sourcing the file **complete_sat.sh** contained in salomeTool directory: +When getting started with sat, the use of the competion mode is convenient. This mode will display by typing twice on the **tab key** the available options, command, applications or product available. The completion mode has to be activated by sourcing the file **complete_sat.sh** contained in SAT directory: .. code-block:: bash @@ -59,82 +70,132 @@ When getting started with sat, the use of the competion mode is usefull. This mo > --products --show --with_children -Verbose and Debug mode -====================== - -Verbosity ---------- -**sat** has several levels of verbosity. The default value is **3** and correspond to the impression of the main information on what has been done. -A verbosity of **0** correspond to no impression at all, while on the opposite a verbosity of **6** prints a lot of information. +Build from scratch an application +================================= -Change verbosity level (default is 3). +This is the main use case : build from scratch an application. .. code-block:: bash - sat -v0 prepare SALOME-9.3.0 -p GEOM # prepare GEOM product in silent mode - sat -v6 compile SALOME-9.3.0 -p GEOM # compile GEOM with full verbosity + # get the list of available applications in your context + # the result depends upon the projects that have been loaded in sat. + ./sat config -l + > ... + > SALOME-8.5.0 + > SALOME-9.3.0 + > SALOME-9.4.0 + + # get all sources of SALOME-9.4.0 application + ./sat prepare SALOME-9.4.0 + + # compile all products (prerequisites and modules of SALOME-9.4.0) + ./sat compile SALOME-9.4.0 + + # if a compilation error occured, you can access the compilation logs with: + ./sat log SALOME-9.4.0 -Debug mode -g -------------- + # create a SALOME launcher, displays its path. + ./sat launcher SALOME-9.4.0 + > Generating launcher for SALOME-9.4.0 : + > .../SALOME-9.4.0-CO7/salome -This mode is used by developers to see more traces and *stack* if an exception is raised. + # start salome platform + .../SALOME-9.4.0-CO7/salome + # create a binary package to install salome on other computers + ./sat package SALOME-9.4.0 -b -Building a SALOME application -============================= -Get the list of available applications --------------------------------------- +All the build is done in the *application directory*, which is parameted by the sat configuration variable *$APPLICATION.workdir*. In the above example t corresponds to *.../SALOME-9.4.0-CO7*. +**sat** can only build application provided by the projects that have been loaded in sat with *sat init* command. These available applications are lister by *sat config -l* command. -To get the list of the current available applications in your context: + +Partial recompilation of a packaged application +=============================================== + +Getting all the sources and compile everything is a long process. +The following use case has proven to be convenient for fast usage. +It consists to get the application through a sat package containing the binaries and sat. +This allows using directly the application (the binary part). +And later, if required, it is possible to add a module, or modify and recompile one. .. code-block:: bash - sat config -l + # untar a sat package containing binaries (for CentOS7) and sources + tar xfz SALOME-9.4.0-CO7-SRC.tar.gz + + # start salome + SALOME-9.4.0-CO7-SRC/salome + + # copy binaries in INSTALL directory, do required substitution to enable recompilation + ./install_bin.sh + + # get sources of module we want to recompile + salomeTools/sat prepare SALOME-9.4.0 -p SHAPER,SMESH + + # do some modifications and recompile both modules + salomeTools/sat compile SALOME-9.4.0 -p SHAPER,SMESH --clean_all -The result depends upon the projects that have been loaded in sat. +This use case is documented in the README file of the package +Using SAT bases +=============== -Prepare sources of an application ---------------------------------- +Users or developers that have to build several applications, which share common products, may want to mutualise the compilation of the common products. +The notion of SAT base follow this obective. It allows sharing the installation of products between several applications. -To prepare (get) *all* the sources of an application (*SALOME_xx* for example): +Location +-------- + +By default the SAT base is located in the parent directory of sat (the directory containing sat directory) and is called BASE. +This default can be changed by the user with sat init command : .. code-block:: bash - # get all sources - sat prepare SALOME_xx + # change the location of SAT base directory + ./sat init -b + +Which products go into the base +------------------------------- - # get (git) sources of SALOME modules - sat prepare SALOME_xx --properties is_SALOME_module:yes +The application developper has the possibility to declare that a products will go by default in the base. +He uses for that the keyword 'base' in the install_dir key within the product configuration file (products pyconf) : *install_dir : 'base'* +It is done usually fir products that are considered as prerequisites. -| The sources are usually copied in directory *$APPLICATION.workdir + $VARS.sep + 'SOURCES'* +At this stage, all products with install_dir set to 'base' will be installed in SAT base directory. -Compile an application ----------------------- +Application configuration +------------------------- -To compile an application +The default behaviour of products can be modified in the application configuration, with the base flag. +Like other application flags (debug, verbose, dev) the base flag can be used for a selection of products, or globally for all products. .. code-block:: bash - # compile all prerequisites/products - sat compile SALOME_xx - - # compile only three products (KERNEL, GUI and SHAPER), if not done yet - sat compile SALOME_xx -p KERNEL,GUI,SHAPER + # declare in application configuration that SMESH and YACS are installed in base + products : + { + ... + SMESH : {base : "yes"} + YACS : {base : "yes"} + ... + } + + # declare with a global application flag that all products are installed in base + base : "yes" + - # compile only two products, unconditionaly - sat compile SALOME_xx -p KERNEL,GUI --clean_all +Mutualisation of products +------------------------- - # (re)compile only salome modules - sat compile SALOME_xx --properties is_SALOME_module:yes --clean_all +Products that go in base and have the same configuration will be shared by different application (it's the objective). +SAT does check the configuration to prevent of an application using a product in base with a non compatible configuration. +To check the compatibility, SAT stores in a file *sat-config-.pyconf* the configuration. +In a next build (for example in an other application), SAT checks if the new configuration corresponds to what is described in *sat-config-.pyconf*. +If it corresponds, the previous build is used in base, otherwise a new build is done, and stored in a new directory called *config-*. -| The products are usually build in the directory -| *$APPLICATION.workdir + $VARS.sep + 'BUILD'* -| -| The products are usually installed in the directory -| *$APPLICATION.workdir + $VARS.sep + 'INSTALL'* +.. warning:: Please note that only the dependencies between products are considered for the checking. If the compilation options changed, it will not be tracked (for example the use of debug mode with -g option will not produce a second configuraion, it will overwrite the previous build done in production mode) diff --git a/doc/src/write_command.rst b/doc/src/write_command.rst index 9a68de8..0a9f7b9 100644 --- a/doc/src/write_command.rst +++ b/doc/src/write_command.rst @@ -2,8 +2,9 @@ .. include:: ../rst_prolog.rst -Add a user custom command -*************************** +************************************** +Extending SAT with additional commands +************************************** Introduction ============ -- 2.30.2