You are here: Home Downloads Model reduction benchmarks Submission rules
Document Actions

Submission rules

Oberwolfach benchmark collection contains benchmarks related to model order reduction. Its goal is to supply to the researches from different fields with the dynamic systems in a computer-readable format in order to test new algorithms, software, etc.

The collection contains documents that are supposed to be processed by a human being and datafiles that are supposed to be read automatically by software. As a result, with minor exceptions, the rules concerning documents have a recommendation status, and the rules concerning datafiles are obligatory.

We start the paper with the description of documents, then we present the publication policy, and finally we describe the datafiles.

1. Documents

The collection consists of documents forming a two-level hierarchy. A top-level documents will be referred to as benchmarks. Each benchmark document may have links to several documents referred to as reports. A benchmark and its reports may be written by different authors.

Each document is written according to conventional scientific practice, that is, it describes the matters in such a way that, at least in principle, anyone could reproduce the results presented. The authors should understand that the document may be read by people from quite different disciplines. Hence, abbreviations should be avoided or at least explained and references to the background ideas should be made.

1.1. Benchmark

The goal of a benchmark document is to describe the origin of the dynamic system and its relevance to the application area. It is important to present the mathematical model, the meaning of the inputs and outputs and the desired behavior from the application viewpoint.

A few points to be addressed:

  • The purpose of the model should be explained clearly. (For instance, simulation, iterative system design, feedback control design, ...)
  • Why should the model be reduced at all? (For instance, reducing simulation time, reducing implementation effort in observers, controllers...)
  • What are the QUALITATIVE requirements to the reduced model? What variables are to be approximated well? Is the step response to be approximated or is it the bode plot? What are typical input signals? (Some systems are driven by a step function and nothing else, others are driven by a wide varity of input signals, others are used in closed loop and can cause instability, although beeing stable themselves)
  • What are the QUANTITATIVE requirements to the reduced model? Best would be if the authors of any individual model can suggest some cost functions (performance indices) to be used for the comparison. These can be in time domain, or in frequency domain (including bandwidth), or both.
  • Are there limits of input and state variables known? (application related or generally)? What are the physical limits where the model becomes useless/false? If known a-priori: Out of the technically typical input signals, which one will cause "the most nonlinear" behaviour?

We stress the importance to describe the software employed, as well as its related options.

If the dynamic system is obtained from partial differential equations, then the information about material properties, geometrical data, initial and boundary conditions should be given. The exception to this rule is the case when the original model came from industry. In this case, if trade secrets are tied with the information mentioned, it may be kept hidden.

The authors are encouraged to produce several dynamic models of different dimensions in order to provide an opportunity to apply different software and to research scalability issues. If an author has an interactive page on his/her server to generate benchmarks, a link to this page is welcome.

The dynamic system may be obtained by means of compound matrices, for example, when the second-order system is converted to first-order. In this case, the document should describe such a transformation but in the datafile the original and not the compound matrices should be given. In this way, this will allow us to research other ways of model reduction of the original system.

1.2. Report

A report document may contain:

a) The solution of the original benchmark that contains sample outputs for the usual input signals. Plots and numerical values of time and frequency response. Eigenvalues and eigenvectors, singular values, poles, zeros, etc.

b) Model reduction and its results as compared to the original system.

c) Description of any other related results.

Once more, we stress the importance to describe the software employed as well as its related options.

1.3. Document Format

Any document is considered as a Web-page. As such it should have a main page in HTML and all other objects linked to the main page such as pictures and plots (gif, jpeg), additional documents (pdf, html). In particular, a document can have just a small introductory part written in HTML and the main part as a linked pdf.

The authors are advised to keep the layout simple.

Scripts included in the Web-page should be avoided, or at least they must not be obligatory to view the page.

Numerical data including the original dynamic system and the simulation results should be given in a special format described in Section 3.

2. Publishing Method

A document is submitted to IMTEK in the electronic form (see below) as an archive of all the appropriate files (tar.gz or zip). Then it is placed in a special area and enters a reviewing stage for four weeks. An announcement about a new document is posted to benchmark@elmo.imtek.uni-freiburg.de and is send to reviewers chosen by a chief editor. Depending on the comments, the document is published, rejected or sent to authors to make corrections. The decision is taken by an editorial board.

A published document is never changed or deleted. Rather, a new version of the document is submitted again and it is published as a new document if accepted. In this case, the old document receives a status "expired" but it still remains in the public area.

2.1. Rules for Online Submission

  • Only ZIP or TAR.GZ archives are accepted for the submission.
  • The maximum compressed size for these files is 15 Mb.
  • The archive should contain at least one HTML file, named “index.html”. This file represents the main document file.
  • The archive may only contain files of the following types: *.html, *.htm, *.pdf, *.gif, *.jpg, *.png, *.zip, *.tar.gz
  • After the submission, the files are post-processed:
    File types not specified above are deleted.
    Only the body part of every HTML file is kept.
    All the format/style/css information, like “style=..”, “class=..” are removed from the body part.
  • If you decide to use PDF documents, use the “index.html” to include links to them.
  • There are three states of the submission:
    Submitted: The author and the chief editor receive a notification mail. The submission is only accessible for the chief editor to accept the submission.
    Opened for review: The submission is open for several users to post their comments and reviews. After that the chief editor can accept the paper.
    Accepted: The submission is open for everybody.

3. Datafiles

All the numerical data for the collection can be considered as a list of matrices, a vector being a m x 1 matrix. As a result, first one should follow a naming convention for the matrices, second one should write each matrix in the format described below.

Let us briefly mention the Dynamic System Interchange Format [1]. It is directly based on the Matlab scripting language. The problems are that it does not scale well and that it is not easy to parse it if one needs to use it outside of Matlab.

Below we suggest an alternative solution that should overcome both problems for linear dynamic systems. The development of a scalable data format for time-dependent and nonlinear dynamic systems is considered to be a challenge to be solved later on. Until a new format is developed for time-dependent nonlinear systems, we suggest to use the Dynamic System Interchange Format [1] for this case.

3.1. Naming convention

Below x is a state vector, x1 is its first derivative, and x2 is its second derivative.

For the two cases of a linear dynamic system of the first and second orders, the naming convention should be written as follows

E x1 = A x + Bu
y = C x + D u

M x2 + E x1 + K x = Bu
y = C x + D u

We recommend to use for nonlinear models

M x2 + E x1 + A x = B u + F g(x,u)
y = C x + D f(x, u)

An author can use another notation in the case when the convention above is not appropriate.

3.2. Matrix format for linear systems

A matrix should be written in the the Matrix Market format as described at http://math.nist.gov/MatrixMarket/.

A file with a matrix should be named as


If there is no file for a matrix, it is assumed to be identity for the E matrix and 0 for the D matrix.

All matrix files for a given problem should be compressed in a single zip or tar.gz archive.

4. References

[1] J. Lienemann, B. Salimbahrami, B. Lohmann. Draft for the Dynamic System Interchange Format, http://www.imtek.uni-freiburg.de/simulation/mstkmpkt/models/DSIF-draft.html.

[2] Younès Chahlaoui and Paul Van Dooren. A collection of benchmark examples for model reduction of linear time invariant dynamical systems; SLICOT Working Note 2002-2: February 2002, http://www.win.tue.nl/niconet/NIC2/benchmodred.html.

Personal tools