Simula Research Laboratory / Center for Resilient Networks and Applications / NorNet
Homepage of Thomas Dreibholz / Thomas Dreibholz's SimProcTC – A Powerful Tool-Chain for Setup, Distributed Processing and Analysis of OMNeT++ Simulations


SimProcTC
A Simulation Processing Tool-Chain for OMNeT++ Simulations
SimProcTC



πŸ“° News


πŸ’‘ Description

In order to efficiently perform simulations using my RSerPool simulation model RSPSIM as well as simulations with the CMT-SCTP model in the INET Framework, I have also developed a model-independent, flexible and powerful tool-chain for the setup, parallel run execution, results aggregation, data analysis and debugging – completely based on Open Source software: SimProcTC (Simulation Processing Tool-Chain). Due to its independence of a specific model, it may also be useful for many more users of OMNeT++. Therefore, I have released this tool-chain as Open Source under GPLv3 license.

A detailed description of SimProcTC can be found in Appendix B of ``Evaluation and Optimisation of Multi-Path Transport using the Stream Control Transmission Protocol´´. Some further information on SimProcTC can also be found in the following references:

The complete BibTeX references in a single file can be found here!

2023

2022

2012

2009

2008

2007


πŸ“¦ Installation

The following items are a step-by-step installation guide for SimProcTC.

Install OMNeT++

Get the latest version of OMNeT++ here and install it under Linux. If you do not have Linux installed already, you may find my Little Ubuntu Linux Installation Guide helpful. This installation guide also provides help on how to install OMNeT++ on an Ubuntu system.
Note that while OMNeT++ also works under Microsoft Windows, my tool-chain has not been tested under this operating system yet. In particular, run distribution using RSerPool will not work under Windows unless you port the RSPLIB RSerPool implementation to Windows.
After installing OMNeT++, make sure that it is working properly.

Install GNU R

Install GNU R. Usually, it will be available for your Linux distribution as installation package. However, if you decide to install it from source, you can download the source here.
Under Ubuntu/Debian Linux, you can download and install GNU R using the following command line:

sudo apt-get install r-base r-base-html r-base-latex r-base-dev r-doc-info r-doc-pdf r-doc-html r-mathlib

After installation, you can start GNU R by:

R --vanilla

You can quit GNU R using Ctrl+D.

Install LIBBZ2

The simulation tool-chain requires LIBBZ2 for compression and decompression of files. In particular, also the developer files (include files) of this library are required to compile the tool-chain. Usually, it will be available for your Linux distribution as installation package. However, if you decide to install it from source, you can download the source here.
Under Ubuntu/Debian Linux, you can download and install LIBBZ2 using the following command line:

sudo apt-get install libbz2-dev

Install the Simulation Tool-Chain

Get the simulation tool-chain package from the Sources Download section and unpack it. Also read the description paper in the docs/ directory; it gives important information on what the tool-chain actually does! The tool-chain archive includes the files of the tool chain as well as a small example simulation. The files have the following purposes:

In order to compile tool-chain and examples, call the following commands in the corresponding example-simulation subdirectory:

opp_makemake -f
make
cd tools
make
cd ..

Make sure that everything compiles successfully. Otherwise, the tool-chain will not work properly! After compilation, you can start the demo simulation by calling:

./example-simulatio

πŸƒ Running the Demo Simulation

The example simulation packaged with SimProcTC simply presents the effects of fragmenting large packets into cells and forwarding them: the delays will significantly reduce at the price of increased overhead. Have a look into scenario.ned to see the parameters of the model:

An example simulation for this model is defined in test1.R: for each parameter of the model, the list "simulationConfigurations" contains a list with the parameter name as first element and its value(s) as further elements. For example, list("sourcePayloadSize", 1000, 2500) means that the parameter "sourcePayloadSize" should be used with the values 1000 bytes and 2500 bytes. For each parameter combination, a separate run will be created. Furthermore, the variable "simulationRuns" specifies how many different seeds should be used. That is, for simulationRuns=3, runs for each parameter combinations are created with 3 different seeds (i.e. tripling the number of runs!).

The actual output of .ini files is realized in simulate-version1.R. Have a look over this file first, it should be quite self-explaining! In the function demoWriteParameterSection(), the actual lines for the parameters above are written for each simulation run. "simCreatorAdditionalActiveVariables" defines for which variables a table row should always be written. For example, if you always use cellHeaderSize=4, the createsummary script would neglect this parameter in the output table. Since it may be useful for your post-processing, you can add it to "simCreatorAdditionalActiveVariables". Note, that "simCreatorWriteParameterSection" is set to "demoWriteParameterSection". In the generic simulation.R script, always the names simCreatorXXX instead of demoXXX are used. In order to be model-independent, it is necessary to set these variables to the actual model-dependent functions in simulate-version1.R! When you adapt the tool-chain to you own model, you only have to create your own simulation-versionX.R script and leave the other scripts unmodified.

The variables "distributionPool" and "distributionProcs" in test1.R are used to control the request distribution. They will be explained later. For now, make sure that distributionProcs is set to 0! This setting means that all runs are processed on the local machine.

Now, in order to perform the simulation defined in test1.R, simply execute test1.R using R:

R --vanilla < test1.R

The script will now create an .ini file for each run and a Makefile containing all runs. Finally, "make" will be called to process the created Makefile. "make" will already be called with the -j parameter corresponding to your number of CPUs/cores, so that it fully utilizes the computation power of your machine. You can observe the progress of the simulation processing by monitoring the log file:

tail -f test1/make.log

You can abort the simulation processing and continue later. Only the run(s) currently in progress are lost and have to be re-processed upon resumption. Already completed runs are saved and no re-processing is necessary.


πŸ“ˆ Plotting the Results

After processing the simulation defined by test1.R, you can plot the results using plot-test1.R:

R --vanilla < plot-test1.R

The results will be written to test1.pdf (the file name will be the simulation output directory + .pdf). You can view it with any PDF reader, e.g. Okular. The plotter settings at the head of plot-test1.R should be almost self-explaining. For "colorMode", you can also use cmBlackAndWhite or cmGreyScale. Setting "plotOwnOutput" to TRUE results in an own output file for each plot (instead of a single PDF file). "plotConfigurations" contains the definitions for each plot, in particular title, output file name for "plotOwnOutput", x- and y-axis ticks, legend position and the results data for each axis given by a template. A set of model-specific templates is already defined in simulate-version1.R, you can add additional ones there or to "plotVariables" in plot-test1.R. See also the paper for more details on templates.


πŸš€ Run Distribution to a Pool of PCs

Make sure that the previous steps (performing simulations and plotting) work. If they are not working properly, the run distribution will also fail! First, it is necessary to install the RSPLIB RSerPool implementation. On a Ubuntu system, RSPLIB can be installed directly using APT:

sudo apt-get install rsplib-registrar rsplib-services rsplib-tools

In case of a need for a manual installation, also see the RSPLIB Handbook here!

One one computer, run CSP monitor to display the status of the other components:

cspmonitor

Note the IP address of this system. The CSP monitor runs on UDP port 2960.

For the other components to be started, define environment variables:

export CSP_SERVER=<IP_OF_CSP_MONITOR>:2960
export CSP_INTERVAL=333

You can put these commands e.g. into ~/.bashrc, so that the variables are available in all new shell instances!

In your network, start at least one RSerPool Pool Registrar (PR):

rspregistrar

With the environment variables above set correctly, the CSP monitor should show the registrar.

Then, start a Scripting Service Pool Element (PE) in another shell.

rspserver -scripting -policy=LeastUsed -ssmaxthreads=4

The parameter -ssmaxthreads specifies the number of parallel sessions; use the number of cores/CPUs in your machine). The output of "rspserver" should look as follows:

Starting service ...
Scripting Server - Version 2.0
==============================

General Parameters:
   Pool Handle             = ScriptingPool
   Reregistration Interval = 30.000s
   Local Addresses         = { all }
   Runtime Limit           = off
   Max Threads             = 4
   Policy Settings
      Policy Type          = LeastUsed
      Load Degradation     = 0.000%
      Load DPF             = 0.000%
      Weight               = 0
      Weight DPF           = 0.000%
Scripting Parameters:
   Keep Temp Dirs          = no
   Verbose Mode            = no
   Transmit Timeout        = 30000 [ms]
   Keep-Alive Interval     = 15000 [ms]
   Keep-Alive Timeout      = 10000 [ms]
   Cache Max Size          = 131072 [KiB]
   Cache Max Entries       = 16
   Cache Directory         =
   Keyring                 =
   Trust DB                =
Registration:
   Identifier              = $249c7176

In particular, take care of the "Identifier" line. This is the ID of the pool element under which it has been registered. If there are error messages saying that registration has failed, etc., have a look into the RSPLIB handbook. Usually, this means a small configuration problem which can be solved easily! It may also be helpful to use Wireshark for debugging network issues; it has dissectors for the RSerPool protocols as well as for CSP and the Scripting Service protocols!

With the environment variables above set correctly, the CSP monitor should show the PE.

Have a look into the script ssdistribute. Ensure that the variable setting for SIMULATION_POOLUSER points to the executable scriptingclient of the RSPLIB package (if installed from the Ubuntu package: /usr/bin/scriptingclient). If scriptingclient is located elsewhere, e.g. src/rsplib-3.3.2/src in your home directory, the line should be:

SIMULATION_POOLUSER=~/src/rsplib-3.3.2/src/scriptingclient

In test1.R, set "distributionProcs" to the maximum number of simultaneous sessions (at least 1; if you later start 5 pool elements with 2 cores each, you should use 10). It is safe to use 1 for the following test. After modifying "distributionProcs", increase "simulationRuns" e.g. by 1. Otherwise, since you have already performed the run of test1.R before, no more runs would be necessary (since their results are already there!). Now, run test1.R again:

R --vanilla < test1.R

Have a look at the output of "rspserver": it should receive jobs and process them. Also, have a look at the log output:

tail -f test1/make.log

When the job distribution is working properly, you can start more pool elements and set up your simulation computation pool. Do not forget to increase "distributionProcs" accordingly!

With the environment variables above set correctly, the CSP monitor should show the status of pool users and pool elements during the simulation processing.

The workload distribution system works as follows:


πŸ”§ Adapting SimProcTC to your own simulation

In order to use SimProcTC with your own model, perform the following tasks:

Before using the RSerPool-based run distribution, first test your simulation on your local machine! This makes finding problems much easier. If everything works, you can continue with run distribution.


πŸ’Ύ Sources Download

SimProcTC is released under the GNU General Public Licence (GPL).

Please use the issue tracker at https://github.com/dreibh/simproctc/issues to report bugs and issues!

Development Version

The Git repository of the SimProcTC sources can be found at https://github.com/dreibh/simproctc:

git clone https://github.com/dreibh/simproctc
cd simproctc

Contributions:

Current Stable Release

The tarball has been signed with my GnuPG key 21412672­518D8B2D­1862EFEF­5CD5D12A­A0877B49. Its authenticity and integrity can be verified by:

gpg --keyserver hkp://keyserver.ubuntu.com --recv-keys 21412672518D8B2D1862EFEF5CD5D12AA0877B49
gpg --verify simproctc-<VERSION>.tar.gz.asc simproctc-<VERSION>.tar.gz

Old Releases

The tarballs have been signed with my GnuPG key 21412672­518D8B2D­1862EFEF­5CD5D12A­A0877B49. Its authenticity and integrity can be verified by:

gpg --keyserver hkp://keyserver.ubuntu.com --recv-keys 21412672518D8B2D1862EFEF5CD5D12AA0877B49
gpg --verify simproctc-<VERSION>.tar.gz.asc simproctc-<VERSION>.tar.gz