Search

link to homepage

Institute for Advanced Simulation (IAS)

Navigation and service


Usage guide

Usage of JuFo

JuFo can be built and installed as described in the installation instructions shipped with the source bundle. After a successful installation the binary
/install/Forecast is generated, which is the main application for simulating job schedulers. By calling this binary a short description of JuFo's options is printed.

Mainly, JuFo requires an LML input file and generates an extended LML output file, which contains -- next to the input data -- job attributes holding the estimated job dispatch dates and used compute resources. Thus, a simple call for JuFo is given in the following example:

> /install/Forecast -i input_LML.xml -o output_LML.xml

The input LML file is parsed, a scheduler prediction is run and the output_LML.xml is generated holding the simulation's results. The JuFo source bundle contains an example input LML file at /doc/dataformat/exampleLML.xml. This file documents all configuration parameters used in the scheduler simulation. It can be used as template input file for a new scheduler prediction. JuFo is meant to be integrated with the monitoring application LLview.

LLview can be used as input source providing LML input files for JuFo runs. The first steps of an LML_DA workflow usually generate LML formatted files. A file, which combines status data of running and submitted jobs as well as compute nodes, can be used as input file for a scheduler prediction with JuFo. Afterwards, JuFo's output file can be further processed by successive workflow steps, which ultimately produce input files for LLview used to visualize the generated data.

Integration with LLview

This section outlines how to integrate JuFo into LLview's data gathering process. Before using JuFo, LML_DA needs to be installed as described in the LLview documentation. In addition, JuFo has to be installed on the system running LML_DA. In order to activate the job prediction step, the used workflow description has to be adjusted. The best position for the JuFo steps is right before the step, which converts the LML format into the old LLview data format. This step calls the LML2llview.pl script.

In order to simplify the insertion of the JuFo step, the surrounding steps are also listed in the following example:

<!-- The addcolor step adds unique colors to all
objects in the LML input file-->
<step active="1" id="addcolor"
exec_after="addSchedConfig" type="execute">
<cmd exec="/usr/bin/perl
$instdir/LML_color/LML_color_obj.pl
-colordefs $instdir/LML_color/default.conf
-dbdir $permdir -o $stepoutfile $stepinfile"/>
</step>
<!--
LML_DA does not automatically detect the job
scheduler's internals. They need to be configured
in a configuration file for JuFo. Here, the
configuration file is assumed to be located at
scheduler.xml. Check the
/doc/dataformat/exampleLML.xml as template
for a JuFo configuration.
In this step the JuFo configuration is integrated
into the current workflow LML file.
-->
<step active="1" id="addSchedConfig"
exec_after="addcolor" type="execute">
<cmd exec="$JUFO/TestFileGenerator
-i $stepinfile -o $stepoutfile
-c scheduler.xml"/>
</step>
<!--
This step actually calls JuFo in order to run the
scheduler simulation. The result of the simulation
is stored in $permdir/LML_fc.xml configured with
the -o option.
-->
<step active="1" exec_after="addSchedConfig"
id="jufo" type="execute">
<cmd exec="cp $stepinfile
$tmpdir/datastep_jufoinput.xml"/>
<cmd exec="$JUFO/Forecast -i $stepinfile
-o $permdir/LML_fc.xml &gt;
$tmpdir/jufo.stdout 2&gt; $tmpdir/jufo.stderr"/>
</step>
<!-- This step converts LML files into the XML
format used by the LLview client. -->
<step active="1" exec_after="jufo"
id="convertLLview" type="execute">
<cmd exec="/usr/bin/perl
$instdir/LML2llview/LML2llview.pl
-config='(SMTenabled=8)(CONF_CPUS=16)(CONF_CORES=8)
(CONF_LEVEL_SMALL_JOBS=512)'
-o $permdir/llview_final.xml
$permdir/LML_fc.xml"/>
</step>

Two workflow steps here are related to the JuFo integration: addSchedConfig and jufo. The former reads the scheduler configuration stored in the file scheduler.xml and adds it to the input file. The latter uses the generated input file and triggers a scheduler prediction. The output file contains attributes for each job providing the estimated start and completion date. These attributes are used by the LLview client to render the prediction diagram showing a Gantt chart for the predicted future schedule. An example of that diagram is depicted by the following figure.


Servicemeu

Homepage