MLton, Ocaml, SML/NJ benchmarks

Stephen Weeks sweeks@wasabi.epr.com
Tue, 19 Oct 1999 11:03:52 -0700 (PDT)


Here are the running times in seconds of some benchmarks I found at
ftp://ftp.inria.fr/INRIA/Projects/cristal/Xavier.Leroy/benchmarks/

I marked the fastest in each case with an X.

		MLton	Ocaml	SML/NJ
boyer		22.44X	29.79	46.75
fft		28.49	23.08X	43.56
fib		37.19	15.39X	46.78
nucleic		30.48	17.22X	76.69
quicksort	27.80	15.31X	40.80
sieve		29.07X	34.98	58.41
takc		26.23	10.71X	38.85
taku		26.29	10.72X	37.94

SML/NJ is pathetic -- nothing more needs to be said.  Now, comparing
MLton and Ocaml:

* For fib, taku, and takc, I blame MLton's pathetic calling convention.
* boyer, fft and sieve are in the noise
* nucleic and quicksort I have no explanation for.  Quicksort in
  particular is surprising, since it is very loopy (there are only two 
  toplevel functions).  Also, for nucleic, the usual benchmark version 
  that I had been using (see
  http://www.neci.nj.nec.com/PLS/MLton/performance.html), MLton beats
  SML/NJ by a factor of 3.86, much more than what we see here
  (although not quite as good as Ocaml).  I'll have to look into the
  two benchmarks to see what's different.  The only thing that I can
  imagine offhand is that in the usual benchmarks, I went through and 
  turned off all print messages so they would run silently, whereas
  with this set I got from Xavier, I just >/dev/null'ed them.

All in all, though, impressive for Ocaml.

We really need that new backend :-)