About Mercury/ benchmarks

Running the benchmarks

We have used a set of ten small logic programming benchmarks to compare the performance of Mercury to the performance of other logic programming systems. Since the syntax of Mercury is quite close to the syntax of Prolog, we have only one version of each benchmark, which is put through the C preprocessor, with a set of definitions depending on the system being tested. This is how we enable the declarations appropriate for Mercury, Aquarius Prolog, SICStus Prolog or NU-Prolog.

The benchmarks were run on a Sun SPARCserver 1000 with four 50 MHz TI SuperSPARC processors and 256 megabytes of memory running SunOS 5.3 (Solaris 2.3). Each processor is rated at 60.3 SPECint92, and has a 4-way associative 16 Kb I-cache and a 5-way associative 20 Kb D-cache, backed by 1 Mb of unified secondary cache.

The tested systems are SWI-Prolog 1.9.0, NU-Prolog 1.6.4, wamcc 2.21, Quintus Prolog 3.2, SICStus Prolog 2.1, Aquarius Prolog 1.0 and Mercury 0.5-beta.

Benchmarks used

This list contains links to the sources of the benchmarks:

  • cqueens is the queens benchmark after it has been put through the source-to-source transformation of Seki and Furukawa from the Proceedings of the Fourth IEEE Symposium on Logic Programming. (Note: cqueens uses the source code for queens, but defines a macro that selects different versions of some predicates).
  • crypt solves a cryptoarithmetic puzzle.
  • deriv symbolically differentiates four functions of a single variable.
  • nrev reverses a list of 30 elements using the naive algorithm.
  • poly symbolically raises 1+x+y+z to the tenth power.
  • prime finds all primes up to 100.
  • qsort quicksorts a list of 50 integers using difference lists.
  • queens finds all safe placements of 9 queens on a 9x9 chessboard.
  • query finds countries with approximately equal population density.
  • tak is an artificial benchmark, originally written in Lisp; it is heavily recursive and does lots of simple integer arithmetic.

The benchmarks use the following test harness:

Benchmarking results

The benchmark results are available here in three forms:

  • as performance ratios normalised to the speed of SWI-Prolog.
  • as speed percentages normalised to the speed of the fastest system variant on each benchmark.
  • as raw times listed as milliseconds per iteration of the benchmark.

Generated code

This list contains links to the C files generated by the Mercury compiler for the benchmarks: