benchmarking
Stephen Weeks
MLton@sourcelight.com
Tue, 3 Jul 2001 15:33:31 -0700
Here are some numbers similar to what's going to go on the performance page,
except that old mlton (20000906) won't be there. But it's nice to see how much
better new MLton is than the old -- uniformly across the board! It's also nice
to see how much better we are than the new Kit (3.9.0) and the new SML/NJ
(110.33), with the exception of a few times when NJ is better.
run time ratio (to MLton 20010708)
old
bench kit mlton mosml nj
barnes-hut 3.5 1.3 12.3 0.8
checksum * 1.7 * 3.0
count-graphs 6.4 1.4 21.1 1.5
fft * 1.8 24.9 1.0
fib 1.0 1.3 4.7 1.1
hamlet * * * 1.4
knuth-bendix 4.7 1.1 12.1 2.1
lexgen 2.4 2.8 5.6 1.6
life 6.9 1.9 19.8 1.0
logic * 1.4 3.4 0.6
mandelbrot 4.4 1.7 33.0 1.4
matrix-multiply 10.7 2.0 37.2 3.4
mlyacc * 1.8 7.6 1.7
mpuz 8.4 2.2 47.7 3.0
nucleic * 1.2 15.6 0.7
peek 6.9 4.1 31.8 2.1
psdes-random 8.4 1.7 * 2.3
ratio-regions 12.0 2.6 46.8 5.1
ray 4.4 1.9 11.4 0.8
raytrace * 1.6 * 2.3
simple 2.3 1.3 12.7 1.4
smith-normal-form * 1.1 * 95.6
tak 1.6 1.6 7.4 1.3
tensor * 1.7 * 5.8
tsp 2.5 1.5 14.5 1.6
vector-concat 9.5 4.1 16.9 5.9
vector-rev 10.1 3.4 32.3 19.2
vliw 2.6 1.6 7.7 1.3
wc-input1 0.0 4.4 * 6.8
wc-scanStream 0.0 2.5 * 1.5
zebra 12.2 1.8 23.6 7.0
zern * 1.4 * 1.5
compile time
bench kit mlton old-mlton mosml nj
barnes-hut 8.2 2.5 5.6 0.8 1.7
checksum * 0.7 1.1 * 0.2
count-graphs 2.6 1.7 3.9 0.2 1.1
fft 2.1 1.5 2.9 0.2 0.9
fib 1.0 0.6 1.0 0.1 0.2
hamlet * 52.8 * * 100.4
knuth-bendix 5.4 2.3 6.3 0.4 2.0
lexgen 10.2 5.8 16.6 0.8 4.8
life 2.9 1.4 2.9 0.2 0.7
logic 6.7 7.5 20.3 0.4 2.0
mandelbrot 1.1 0.7 1.1 0.1 0.2
matrix-multiply 1.2 0.7 1.2 0.1 0.3
mlyacc 60.1 19.6 156.0 7.4 24.2
mpuz 1.3 0.9 1.6 0.1 0.4
nucleic 28.0 4.3 6.6 2.1 2.9
peek 1.0 1.1 2.4 0.1 0.2
psdes-random 1.1 0.7 1.2 * 0.3
ratio-regions 5.2 3.1 6.7 0.4 2.1
ray 3.8 3.6 10.0 0.3 1.1
raytrace * 10.3 28.4 * 6.7
simple 14.5 7.5 39.8 0.9 4.5
smith-normal-form * 8.1 70.5 * 3.5
tak 1.0 0.6 1.0 0.1 0.2
tensor * 3.1 7.7 * 3.5
tsp 2.7 1.8 3.4 0.3 0.8
vector-concat 1.0 0.7 1.2 0.1 0.2
vector-rev 1.0 0.7 1.1 0.1 0.2
vliw 37.2 12.5 64.1 3.0 18.2
wc-input1 1.0 1.7 3.2 0.1 0.3
wc-scanStream 1.1 1.8 3.3 0.1 0.3
zebra 2.9 4.8 16.1 0.1 0.8
zern * 1.1 2.1 * 0.7
size
old
bench kit mlton mlton mosml nj
barnes-hut 179,964 41,194 56,472 94,990 331,768
checksum * 18,121 21,190 * 332,504
count-graphs 109,996 37,305 43,766 84,575 355,376
fft 107,348 31,921 32,730 84,095 332,808
fib 68,700 17,905 20,774 79,878 310,968
hamlet * 979,200 * * 1,263,816
knuth-bendix 115,348 56,410 76,500 88,439 321,504
lexgen 227,444 128,601 168,044 104,883 390,136
life 100,380 34,593 38,062 83,390 305,120
logic 136,180 145,577 151,158 87,252 331,744
mandelbrot 101,660 17,753 19,598 81,341 311,992
matrix-multiply 118,380 18,673 21,958 81,879 338,632
mlyacc 525,580 421,737 595,532 148,286 700,456
mpuz 89,564 22,961 25,334 82,383 320,184
nucleic 234,188 55,033 48,306 207,154 354,288
peek 77,500 26,074 41,324 81,618 311,016
psdes-random 84,636 19,065 22,374 * 313,016
ratio-regions 111,356 54,425 55,174 87,485 335,856
ray 123,420 70,808 100,476 89,860 384,072
raytrace * 178,237 215,473 * 510,056
simple 194,092 166,265 201,080 94,397 641,056
smith-normal-form * 139,153 153,440 * 483,400
tak 68,468 17,817 19,806 79,928 306,872
tensor * 50,521 61,468 * 342,048
tsp 115,732 34,394 42,866 86,140 322,552
vector-concat 77,756 18,569 21,878 80,191 317,128
vector-rev 77,956 18,713 21,582 80,073 317,128
vliw 418,268 260,745 736,036 135,386 618,576
wc-input1 144,636 38,330 49,157 86,900 311,992
wc-scanStream 145,084 40,882 49,501 87,076 313,016
zebra 85,516 106,234 135,996 83,419 310,256
zern * 24,880 25,554 * 326,664