[MLton] [fwd] bug report, strange floating point behavior

[Matthew Fluet]

> I originally sent this to the mlton-user list after having problems
> submitting it, so I thought I'd send it to the main mlton list just to
> see if anyone besides Matthew has looked at it.

I'll throw out a couple of more data points.

I see the "good" behavior on the experimental amd64-linux; this uses the
C codegen and on that architecture, gcc defaults to using sse
instructions for floating point math, which performs the calculation at
64-bits.

I also see the "good" behavior on x86-linux with the compiler options:
-codegen c -cc-opt -msse -ccopt -mfpmath=sse
which should work on any modern x86 processor.  This also causes gcc to
use sse instructions for floating point math, hence performs the
calculation at 64-bits.

So, I think the behavior is pretty easily explained by the fact that at 
80-bit precision, the expression (x*y/3.0, x*9.0) doesn't quite turn out 
to be (1/3,3), but rather one of them comes out just a few bits less 
than the required value, which, after many iterations, tends towards 0.
(You can see this more explicitly by changing the cutoff value from 
10000 to 75, where you can see some precision being eroded.)

While it is curious that floating-point register allocation is being 
affected by the code at the loop termination, it isn't totally surprising.


> ----- Forwarded message from Matthew Fluet <fluet at tti-c.org> -----
> 
> From: Matthew Fluet <fluet at tti-c.org>
> To: Matthew Fluet <fluet at tti-c.org>
> Cc: mlton-user at mlton.org, Scott Cruzen <sic at lerp.com>
> Subject: Re: [MLton-user] bug report, strange floating point behavior
> Date: Mon, 19 Feb 2007 10:36:48 -0600
> Message-ID: <45D9D220.3000203 at tti-c.org> 
> 	(sfid-H20070219-083705-1 at spamfilter.osbf.lua)
> Organization: Toyota Technological Institute at Chicago
> User-Agent: Thunderbird 1.5.0.9 (Windows/20061207)
> Sender: mlton-user-bounces at mlton.org
> 
> Matthew Fluet wrote:
>>> (*
>>> works
>>> *)
>>> fun go (x,y) 100000 = (print (Real.toString x ^ "\n");
>>>                       print (Real.toString (x+y) ^ "\n")
>>>                      )
>>>  | go (x,y) i      = go (x*y/3.0, x*9.0) (i+1)
>>>
>>> (*
>>> fails
>>> *)
>>> fun go2 (x,y) 100000 = print (Real.toString (x+y) ^ "\n")
>>>  | go2 (x,y) i      = go2 (x*y/3.0, x*9.0) (i+1)
>>>
>>> val _ = go (1.0/3.0, 3.0) 1
>>> val _ = go2 (1.0/3.0, 3.0) 1
>>>
>>> (*
>>> I expect the output of the previous code to be:
>>> 0.333333333333
>>> 3.33333333333
>>> 3.33333333333
>>>
>>> mlton without options produces
>>> 0.333333333333
>>> 3.33333333333
>>> 0
>>>
>>> with -ieee-fp true or -codegen bytecode, go2 produces the expected result
>>> tested with 20051202 and svn mlton as of yesterday
>>> *)
>> I've confirmed the behavior on x86-linux; what is interesting is that I 
>> see the expected behavior with
>>  -codegen native -ieee-fp
>>  -codegen bytecode
>> and I see the unexpected behavior with
>>  -codegen native
>>  -codegen c
>>
>> That leads me to believe that it could be explained by floating-point 
>> rounding at extended 80bit precision, but I don't see exactly why.
> 
> On x86-darwin, I see the expected behavior with
>   -codegen native -ieee-fp true
>   -codegen c
>   -codegen bytecode
> and I see the unexpected behavior with
>   -codegen native
> 
> ----- End forwarded message -----
> 
> _______________________________________________
> MLton mailing list
> MLton at mlton.org
> http://mlton.org/mailman/listinfo/mlton
> 
> 
>