1 /*! \page engine Engine
3 \section toc Table of contents
9 \section engine_intro Introduction
11 The engine is in charge to :
18 graphs of execution independently from the context (\ref engine_runtime) the graph is destined to run.
20 \section basic_concepts Basic concepts in YACS::ENGINE
22 The understanding of YACS::ENGINE implementation needs a good knowledge of the basic concepts (\subpage engineConcepts):
29 - \ref engine_placement
31 \section engine_executor Executor
33 The executor is in charge to run a graph of execution. Executor is
34 TOTALLY independant from Node and Port implementation. The only APIs seen
35 from YACS::ENGINE::Executor are YACS::ENGINE::Scheduler and YACS::ENGINE::Task. So, from the Executor point of
36 view, a graph of execution is a scheduler scheduling tasks.
37 The responsability of executor is to launch, concurrently or not, tasks selected by
39 notify to tasks and to the scheduler what it is going to do and what happend
40 during tasks' execution.
41 There are several launching mode in executor. Here the common modes :
42 - launch tasks until scheduler declares that all is finished.
43 - launch tasks until a given task is upon to be launched.
44 - launch tasks one by one. (step by step)
48 /*! \page engineConcepts Engine concepts
50 \section engine_node Nodes
52 A Node is an entity performing a treatement or job using ingoing data
53 given in ingoing \ref engine_ports
54 provided by other Nodes or 'manually' set and providing itself data in outgoing \ref engine_ports. A
55 Node is eventually put into a scope (see \ref engine_hierarchy) in
56 which all it's links with other Nodes can be performed. The most
57 little scope if it exists of a Node is referenced by YACS::ENGINE::Node::_father.
59 There are 2 types of Node :
61 - Node performing a job NOT splitable into several simpler jobs. In
62 this case job is called task. This type of Node can be dowcasted into
63 YACS::ENGINE::ElementaryNode. That's why, YACS::ENGINE::ElementaryNode inherits from YACS::ENGINE::Task
64 and YACS::ENGINE::Node.
66 - Node performing job splittable into several jobs. This type of
67 Node can be dowcasted into YACS::ENGINE::ComposedNode. As this specific
68 type of Node is composed of several Nodes it is in charge of schedule
69 them. That's why, YACS::ENGINE::ComposedNode inherits from YACS::ENGINE::Scheduler and YACS::ENGINE::Node.
71 \subsection engine_hierarchy Node hierarchy
73 - It has been called hierarchy 'MyHierarchy' of a node 'MyNode' the biggest tree which each node of
74 this tree is an instance of ComposedNode and each leaves are instances
75 of ElementaryNode. One of these leaves/nodes is 'MyNode'. The links between leaves/nodes nodes/nodes are
76 established by the fatherness stored in each instance of Node
77 (YACS::ENGINE::Node::_father).
79 this tree only linked down (with _father equal to 0) and having no
80 father is called \b RootNode of 'MyHierarchy'.
81 - A \b level \b of \b hierarchy of 'MyHierarchy' is the set of node/leaf having
83 - A \b scope of a the hierarchy 'MyHierarchy' is a subtree of
84 'MyHierarchy' tree. A scope is represented by the head node of this subtree.
86 \section engine_ports Ports
88 A YACS::ENGINE::Port is an interface of a YACS::ENGINE::Node from the
89 outside. Ports can have several semantics.
91 - \ref engine_control_flow
92 - \ref engine_data_flow
93 - \ref engine_data_stream
95 \subsection engine_control_flow Control flow
97 The semantic of this port is to put constraints on the sequence of
98 execution to synchronize nodes, on THE SAME
99 LEVEL of \ref engine_hierarchy between them.
101 \subsection engine_data_flow Data
103 Instances of these type of ports inherits from YACS::ENGINE::DataFlowPort.
104 This type of port represents data exchange protocol performed
105 physically by implementation in \ref
106 engine_runtime at the end of execution an instance of an ElementaryNode. So contrary to \ref
107 engine_data_stream, this data exchange protocol is performed following
108 \ref engine_control_flow synchronization.
110 \subsection engine_data_stream DataStream
112 Instances of these type of ports inherits from
113 YACS::ENGINE::DataStreamPort. DataStream ports are ports for data
114 exchange NOT synchronized by \ref engine_control_flow.
116 \section engine_links Links
118 A link in YACS does not lie onto a C++ object. A link exists in
119 YACS::ENGINE model only like
120 a sorted pair (YACS::ENGINE::OutPort, YACS::ENGINE::InPort).
121 The semantic of elements of pair can be different (\ref engine_data_flow
122 or \ref engine_data_stream ). This pair
123 is stored in both YACS::ENGINE::OutPort side and YACS::ENGINE::InPort
124 side. The storage YACS::ENGINE::InPort side is only for
125 edition facility (Node, Port suppression), but at execution time, only links info
126 YACS::ENGINE::OutPort side is used.
128 A link is constructed by the call to
129 YACS::ENGINE::ComposedNode::edAddLink method. The instance of
130 YACS::ENGINE::ComposedNode on which apply edAddLink must be so that
131 inPort and OutPort are both in its scope.
133 \subsection engine_links_cpx Point of view of engine at run time
135 It is important to note that a link defined by edAddLink method can
136 hide potentially a serie of elementary links. This is particalary true
137 when 2 dataflow ports inside 2 different loops are linked each other ;
138 in this case \ref engine_data_flow / \ref engine_data_stream
139 conversion is requested.
140 An elementary link is a link in which the semantic of both elements in pair
141 are exactly the same. So the complexity linked to modification of port
142 semantic in a link is managed at edition time so that at run time only
143 elementary links are seen.
145 \section engine_types Data types
147 YACS::ENGINE::TypeCode instances are destined to describe data
148 structure of a piece of data expected by an instance of
149 YACS::ENGINE::DataPort (for type checking at edition : static type
150 checking) or held in YACS::ENGINE::Any instance (for type checking
151 at execution : dynamic type checking).
152 All data exchanged by calculation nodes through input and output
153 data ports have a type.
155 The type is given by the port that holds the data by calling
156 its YACS::ENGINE::DataPort::edGetType() method.
157 All types are instances of the YACS::ENGINE::TypeCode class or one of
158 its derived classes : YACS::ENGINE::TypeCode_objref, YACS::ENGINE::TypeCode_seq.
160 \section engine_context Context
162 A context is a set of libraries allowing directly or indirectly from C++ calls to :
164 - launch on demand, a computation or a job specified uniquely by a
165 string of character and a list of inputs and giving in return a
166 list of data in specific data formats.
167 - handle data of these specific data formats. Each of these data
168 formats overlapping all or part of data overlapped by YACS::ENGINE::Any.
170 Concretely it takes form of set of :
172 - middlewares (CORBA, SOAP, MPI, PVM ...)
173 - high level langage interpreters (as python, perl, shell) callable
175 - plateform (<a href="http://www.salome-platform.org">SALOME</a>)
177 \section engine_runtime Runtime
179 A runtime exists in a given \ref engine_context. A runtime is in charge to :
181 - treat physically the basic execution of elementary tasks in a given context
182 - the traduction of data in this context.
183 - to perform the physical deployment of the execution.
185 The runtime simply appears in engine as an interface
186 that a concrete runtime must implement to be piloted by YACS::ENGINE.
188 \section engine_placement Deployment
190 This the 3rd point of view of a graph of execution, after
191 links between nodes (see \ref engine_links ) and hierarchy in nodes (
192 \ref engine_hierarchy ).
193 Deployment concept is accessible in interface YACS::ENGINE::Task with the
194 concept of YACS::ENGINE::ComponentInstance and YACS::ENGINE::Container.
196 - a ComponentInstance is a common environement \b shared by a set of
197 YACS::ENGINE::ServiceNode. Typically ServiceNodes sharing a same
198 ComponentInstance are sharing a same state taking different form
199 (environement var and/or memory space and/or internal variables...).
200 - a Container is a common process \b shared by several ComponentInstances.
202 A task needed to be placed or deployed
203 at runtime has to return something different from 0 on call to
204 YACS::ENGINE::Task::getComponent(). YACS::ENGINE::ServiceNode is the
205 abstract class representing all executable nodes needed to be
208 It's important to note that the placement (or deployment) of
209 ServiceNodes can be performed at different moment at runtime.
211 - Either it is performed once for all YACS::ENGINE::Container before
212 launching any task. This placement startegy is \b absolutly required for batch
213 mode. This condition is checked by the YACS::ENGINE::Executor by
214 calling YACS::ENGINE::Scheduler::isPlacementPredictableB4Run.
215 - or it is done at the last moment on call of
216 YACS::ENGINE::Load when execution of a ServiceNode is required by
217 Executor on running process.