[Pharo-dev] [ANN] CalipeL: a benchmarking tool for Smalltalk/X and Pharo
jan.vrany at fit.cvut.cz
Fri Oct 23 04:47:41 EDT 2015
After more than 2 years of (time-to-time) development and about
that much time of use, I'd like to announce CalipeL, a tool for
benchmarking and monitoring performance regressions.
The basic ideas that drove the development:
* Benchmarking and (especially) interpreting benchmark results
is always a monkey business. The tool should produce raw numbers,
letting the user to use whichever statistics she need to make up
* Benchmark results should be kept and managed at a single place so
one can view and retrieve all past benchmark results pretty much
the same way as one can view and retrieve past versions of
the software from a source code management tool.
- simple - creating a benchmark is as simple as writing a method
in a class
- flexible - a special set-up and/or warm-up routines could be
specified at benchmark-level as well as set of parameters
to allow fine-grained measurements under different conditions
- batch runner - contains a batch runner allowing one to run
benchmarks from a command line or at CI servers such as Jenkins.
- web - comes with simple web interface to gather and process
benchmark results. However, the web application would deserve
some more work.
http://smalltalkhub.com/#!/~JanVrany/CalipeL-S (read-only export
from the above and Pharo-specific code)
I have been using CalipeL for benchmarking and keeping track of
performance of Smalltalk/X VM, STX:LIBJAVA, a PetitParser compiler
and other code I was working over the time.
Finally, I'd like to thank to Marcel Hlopko for his work on the
web application and Jan Kurs for his comments.
I hope some of you may find it useful. If you have any comments
or questions, do not hesitate and let me know!
More information about the Pharo-dev