Benchmarks

Benchmarks for LivingDoc

Some components include microbenchmarks that can be used to estimate the performance of parts of the system. The benchmarks are implemented using JMH

Executing the Benchmarks

Automatically

The benchmarks are integrated into the Gradle build and can be run with

$ ./gradlew jmh

This executes all defined benchmarks with some default parameters. Since the benchmarks are executed sequentially, they can run for some time. If you want more control over the benchmarks you can also run them manually.

Manually

Manually executing the benchmarks is a two-step process. First you need to build the benchmarks:

$ ./gradlew jmhJar

Then you can run the JAR file containing the benchmarks for the component you want (e.g. livingdoc-engine):

$ java -jar livingdoc-engine/build/lib/livingdoc-engine-2.0-SNAPSHOT-jmh.jar

The actual path depends on the component and the version you would like to benchmarks. You can pass the -h flag to the above command to get an overview over the available options. You can also select which benchmarks should be run by passing a regular expression to the above invocation.

Profiling a Benchmark

Sometimes it might be necessary to analyze the behavior of the code that is benchmarked. JMH has built-in support for some profilers.

One of these profilers is a simple stack based profiler, stack. To run a benchmark under stack, simply run:

$ java -jar livingdoc-engine/build/lib/livingdoc-engine-2.0-SNAPSHOT-jmh.jar -prof stack

After the benchmark is finished, the hottest code lines will be printed. To get an overview of options for stack, run

$ java -jar livingdoc-engine/build/lib/livingdoc-engine-2.0-SNAPSHOT-jmh.jar -prof stack:help