benchmarks
Matthew Fluet
Matthew Fluet <fluet@CS.Cornell.EDU>
Mon, 5 Nov 2001 14:09:56 -0500 (EST)
Here are the results of benchmarking the 20011006 release versus the new
SSA passes. Obviously, a little ways to go, but I just wanted to try out
the SSA passes on another set of programs. All of the "Out of memory"
failures can be attributed to non-tail loops that should be turned into
tail loops. Compile times are a bit high, but I think that's reasonable
since I'm running removeUnused after every pass, which in turn in
introducing lots of extra wrapper blocks that aren't being shrunk.
MLton0 -- mlton-stable
MLton1 -- mlton -drop-pass removeUnused1CPS -drop-pass leafInlineCPS
-drop-pass raiseToJump1CPS -drop-pass contify1CPS -drop-pass
localFlatten1CPS -drop-pass uselessCPS -drop-pass removeUnused2CPS
-drop-pass unusedArgs1CPS -drop-pass simplifyTypesCPS -drop-pass
polyEqualCPS -drop-pass contify2CPS -drop-pass inlineCPS -drop-pass
localFlatten2CPS -drop-pass removeUnused3CPS -drop-pass raiseToJump2CPS
-drop-pass contify3CPS -drop-pass unusedArgs2CPS -drop-pass
introduceLoopsCPS -drop-pass loopInvariantCPS -drop-pass flattenCPS
-drop-pass localFlatten3CPS -drop-pass commonSubexpCPS -drop-pass
commonBlockCPS -drop-pass redundantTestsCPS -drop-pass redundantCPS
-drop-pass unusedArgs3CPS -drop-pass removeUnused4CPS
compile time
benchmark MLton0 MLton1
barnes-hut 2.2 8.0
checksum 0.6 1.1
count-graphs 1.6 3.8
DLXSimulator 3.8 13.0
fft 1.1 3.8
fib 0.6 0.9
hamlet 41.5 105.1
knuth-bendix 2.0 4.2
lexgen 4.7 14.8
life 1.2 2.7
logic 5.5 14.0
mandelbrot 0.6 1.1
matrix-multiply 0.6 1.3
md5 1.5 3.3
merge 0.6 1.1
mlyacc 19.1 44.8
mpuz 0.8 1.7
nucleic 2.9 5.2
peek 0.9 2.6
psdes-random 0.6 1.2
ratio-regions 2.3 8.0
ray 3.0 10.3
raytrace 8.1 24.1
simple 5.8 20.1
smith-normal-form 7.4 16.6
tailfib 0.6 0.9
tak 0.6 0.9
tensor 2.6 9.5
tsp 1.4 3.8
tyan 3.4 12.9
vector-concat 0.6 1.2
vector-rev 0.7 1.2
vliw 11.0 36.1
wc-input1 1.4 4.0
wc-scanStream 1.5 4.6
zebra 8.3 10.4
zern 0.9 2.5
run time
benchmark MLton0 MLton1
barnes-hut 3.9 66.3
checksum 3.2 * -- Out of memory
count-graphs 4.7 74.2
DLXSimulator 15.1 75.6
fft 7.8 143.9
fib 3.4 7.6
hamlet 8.1 89.8
knuth-bendix 6.5 32.2
lexgen 10.5 253.4
life 7.9 13.3
logic 25.7 46.9
mandelbrot 6.7 474.9
matrix-multiply 5.2 141.0
md5 3.3 121.4
merge 49.0 116.9
mlyacc 9.4 102.5
mpuz 4.6 57.9
nucleic 6.8 144.0
peek 3.4 35.1
psdes-random 3.4 * -- Out of memory
ratio-regions 8.2 395.2
ray 3.8 89.7
raytrace 4.6 195.0
simple 6.0 79.1
smith-normal-form 0.9 * -- IntInf/flattening bug
tailfib 16.4 * -- Out of memory
tak 7.9 103.8
tensor 7.1 1824.7
tsp 9.0 165.3
tyan 19.5 153.8
vector-concat 5.7 404.0
vector-rev 4.1 209.8
vliw 6.2 66.7
wc-input1 2.2 55.9
wc-scanStream 3.6 131.3
zebra 2.1 9.1
zern 33.8 1199.9
run time ratio
benchmark MLton1
barnes-hut 16.8
count-graphs 15.6
DLXSimulator 5.0
fft 18.4
fib 2.2
hamlet 11.1
knuth-bendix 4.9
lexgen 24.2
life 1.7
logic 1.8
mandelbrot 71.0
matrix-multiply 27.3
md5 37.0
merge 2.4
mlyacc 10.9
mpuz 12.5
nucleic 21.2
peek 10.2
ratio-regions 48.1
ray 23.4
raytrace 42.8
simple 13.2
tak 13.2
tensor 255.8
tsp 18.3
tyan 7.9
vector-concat 71.2
vector-rev 50.8
vliw 10.8
wc-input1 25.1
wc-scanStream 36.6
zebra 4.4
zern 35.5
size
benchmark MLton0 MLton1
barnes-hut 59,793 278,920
checksum 20,917 39,864
count-graphs 40,461 126,856
DLXSimulator 78,237 419,896
fft 29,441 126,836
fib 20,909 34,520
hamlet 945,328 2,939,955
knuth-bendix 59,710 125,273
lexgen 122,061 462,920
life 38,565 87,896
logic 147,501 436,984
mandelbrot 20,901 40,512
matrix-multiply 21,309 45,528
md5 34,038 113,345
merge 21,885 38,408
mlyacc 409,501 1,388,752
mpuz 26,645 59,040
nucleic 60,653 180,352
peek 28,542 86,841
psdes-random 21,901 43,688
ratio-regions 41,893 268,568
ray 66,688 331,059
raytrace 159,381 791,056
simple 146,913 681,644
smith-normal-form 141,053 346,756
tailfib 20,637 34,888
tak 20,957 35,624
tensor 62,516 323,843
tsp 33,774 128,385
tyan 77,054 436,169
vector-concat 21,557 42,384
vector-rev 21,389 40,928
vliw 261,417 1,193,180
wc-input1 39,222 133,561
wc-scanStream 41,614 155,153
zebra 103,502 253,049
zern 26,504 80,619