JarCastingStochastic Performance Logic evaluation engine
Stochastice Performance Logic is a formalism for capturing performance assumptions. It is, for example, possible to capture assumption that newer version of a function bar is faster than the previous version or that library foobar is faster than library barfoo when rendering antialiased text.
This repository contains an implementation for evaluation of the SPL formulas. That is, you would use this library when you have your performance measured (or modeled) and you need to evaluate whether your assumptions are correct.
Requirements
To compile and run the framework, the following software has to be available on your machine.
- Java SDK >= 1.8.0
- Maven
Compilation
Simply execute
$ mvn package
to compile the program. For creating .jar package and add it to local Maven repository run
$ mvn install
Resulting file is target/spl-evaluation-java-1.0.0.jar.
All the generated files (classes, test reports) are stored in the target/ directory.
Usage
There are several command-line options. Quick help is printed on error (for example when no argument is given).
usage: spl-evaluation-java [-c <formulas>] -d <data_directory> [-f
<formula_file>] [-j <benchmarks.jar>] [-p] [-r <mapping_file>]
Evaluate measured data against SPL formulas.
-c,--commandline-formulas <formulas> Read SPL formulas from commandline
arguments.
-d,--data-dir <data_directory> Path to directory with measured
data.
-f,--file-formulas <formula_file> Read SPL formulas from text file.
-j,--jar-formulas <benchmarks.jar> Read SPL formulas from JAR file.
-p,--print-unknown Print unknown revisions and exit.
-r,--revision-mapping <mapping_file> Mapping of formula revisions to
file names.
SPL formulas can be specified in 3 ways - inside benchmarks.jar, in text file or directly on command-line. Formulas are parsed in this order, latter has higher priority and will override previous formulas.
benchmarks.jar- formulas are saved in META-INF/SPLFormulas file formatted asbenchmark_name:formula. This file is generated via corresponding maven plugin (jmh_spl-maven-plugin) from@SPLFormulaannotations.- text file - simple text file formatted as
benchmark_name:formula - command-line arguments - list of values
benchmark_name:formulaseparated by space. Note that enclosing each value into"symbols may be required because of shell argument parsing.
print-unknown option prints versions, that are mentioned in SPL formulas but are not present in measured data or transitively in custom mapping of revisions. No evaluation is performed. Empty rows or rows starting with # are ignored.
revision-mapping is simple text file with : separated values of custom revision and actual measured data revision. Each line contains one mapping, empty rows or rows starting with # are ignored.
Example
$ java -jar spl-evaluation-java.jar -d ./demo-data/jmh -r /tmp/mapping.txt -j ./target/benchmarks.jar -c "cz.stdin.ps.MyBenchmark.testMethod:last < ver2" "cz.stdin.ps.MyBenchmark.methodTwo:ver1 < base"
Tests
The attached unit tests can be compiled and executed by running
$ mvn test
Documentation
... is not very detailed at the moment.
Running mvn site will generate JavaDoc documentation with basic information on how to use this evaluation engine, unit test report and detailed project summary. To see it, open target/site/index.html in your browser.
SPL was originally created for performance unit testing, more information is available at SPL for Java page at Department of Distributed and Dependable Systems (Faculty of Mathematics and Physics, Charles University in Prague).
Demos
Examples of how to use this project as a library are located in src/demo/ directory. It contains separate pom.xml file to build them. Note, that the parent project must be installed in local Maven repository prior building demos (via mvn install command).
There are currently two prepared scenarios, which can be run directly from Maven:
mvn exec:java@regression-testermvn exec:java@sensitivity-comparison