[MLton] [fwd] bug report, strange floating point behavior

[Scott Cruzen]

I originally sent this to the mlton-user list after having problems
submitting it, so I thought I'd send it to the main mlton list just to
see if anyone besides Matthew has looked at it.

----- Forwarded message from Matthew Fluet <fluet at tti-c.org> -----

From: Matthew Fluet <fluet at tti-c.org>
To: Matthew Fluet <fluet at tti-c.org>
Cc: mlton-user at mlton.org, Scott Cruzen <sic at lerp.com>
Subject: Re: [MLton-user] bug report, strange floating point behavior
Date: Mon, 19 Feb 2007 10:36:48 -0600
Message-ID: <45D9D220.3000203 at tti-c.org> 
	(sfid-H20070219-083705-1 at spamfilter.osbf.lua)
Organization: Toyota Technological Institute at Chicago
User-Agent: Thunderbird 1.5.0.9 (Windows/20061207)
Sender: mlton-user-bounces at mlton.org

Matthew Fluet wrote:
>>(*
>>works
>>*)
>>fun go (x,y) 100000 = (print (Real.toString x ^ "\n");
>>                       print (Real.toString (x+y) ^ "\n")
>>                      )
>>  | go (x,y) i      = go (x*y/3.0, x*9.0) (i+1)
>>
>>(*
>>fails
>>*)
>>fun go2 (x,y) 100000 = print (Real.toString (x+y) ^ "\n")
>>  | go2 (x,y) i      = go2 (x*y/3.0, x*9.0) (i+1)
>>
>>val _ = go (1.0/3.0, 3.0) 1
>>val _ = go2 (1.0/3.0, 3.0) 1
>>
>>(*
>>I expect the output of the previous code to be:
>>0.333333333333
>>3.33333333333
>>3.33333333333
>>
>>mlton without options produces
>>0.333333333333
>>3.33333333333
>>0
>>
>>with -ieee-fp true or -codegen bytecode, go2 produces the expected result
>>tested with 20051202 and svn mlton as of yesterday
>>*)
>
>I've confirmed the behavior on x86-linux; what is interesting is that I 
>see the expected behavior with
>  -codegen native -ieee-fp
>  -codegen bytecode
>and I see the unexpected behavior with
>  -codegen native
>  -codegen c
>
>That leads me to believe that it could be explained by floating-point 
>rounding at extended 80bit precision, but I don't see exactly why.

On x86-darwin, I see the expected behavior with
  -codegen native -ieee-fp true
  -codegen c
  -codegen bytecode
and I see the unexpected behavior with
  -codegen native

----- End forwarded message -----