Team PLClub ICFP entry -- comparing the performance of OCAML and SML

[Stephen Weeks]

> From what I know, for the floating-point intensive benchmarks, Ocaml
> is faster than SML/NJ mainly because Ocaml directly calls the native,
> C-based math library (which in turn is fine-tuned for the actual
> processor).  SML/NJ, on the other hand, implements all of the math
> library functions (e.g., cos, sin, tan) in ML itself (as part of the
> SML basis). There is easily an order-of-magnitude difference between
> these two implementations.  You could try to measure a few such math
> functions to find out.

I will restrict my discussion to the raytracing benchmark at hand.  I'll also
only deal with John Reppy's modified version of the benchmark and with SML/NJ
110.29 for the rest of this post.

I instrumented the source to count the number of calls to Real.Math functions
(asin, acos, cos, exp, ln, sin, sqrt, tan).  The instrumentation shows that the
dominant function is sqrt.  I then wrote a simple loop that repeatedly performs
a sqrt computation.  For this loop, SML/NJ performs about 460 thousand sqrts per
second.  MLton, which calls the C library, performs about 5.1 million sqrts per
second.  So, you are correct for Real.Math.sqrt -- the SML/NJ implementation is
about 10 times slower than the C math library.  I then computed a rough estimate
for the number of seconds spent by SML/NJ in sqrt for each benchmark by taking
the number of sqrts divided by SML/NJ's rate for sqrts.  Here are the numbers
for each benchmark, along with the original running time measurements.
                                                
              number of    number SML/NJ   Total running time
               calls to        of   sqrt 
              Real.Math     sqrts   time  OCAML  MLton SML/NJ
-----------   ---------  --------  -----  -----  -----  -----
holes            252374    252359    0.6    1.8    3.2    3.9
fov              416558    416545    0.9    1.5    1.8    3.2
intercyl         789548    735426    1.6    1.6    2.1    4.3
snowgoon         470715    362464    0.8    2.9    3.3    5.1
dice            1263757    925063    2.0    3.9    4.9    8.8
golf             193489    130088    0.3    1.5    2.4    3.1
cone-fractal     603638    455931    1.0    3.7    4.3    6.5
large             73500     73487    0.6    4.3    3.0    6.7
pipe            1087559    983314    2.2    5.4    4.6    7.9
chess           1192926   1030668    2.3   16.0   15.5   21.6
fractal         2296066   2229266    4.9   12.2    8.5   45.4