[MLton-user] minor 32bit vs. 64bit differences in floating-point calculations with large numbers

[Wesley W. Terpstra]

On Thu, Oct 14, 2010 at 4:18 PM, David Hansel
<hansel at reactive-systems.com>wrote:

> For example:
>
> (1E308 * 10.0) / 10.0 = 1E308 when compiled as 32-bit
> (1E308 * 10.0) / 10.0 = inf   when compiled as 64-bit
>

This is documented in the man page. 32-bit MLton uses (by default) the FPU
and 64-bit uses SSE registers. This means that 32-bit MLton has slightly
higher intermediate precision than a "normal" IEEE 754 point registers, but
performs nicely. You can override this behaviour with "-ieee-fp true" which
will force MLton to lose this precision for intermediate calculations.

Note: on i386, the results can also differ depending on when the compiler
can no longer fit the intermediate floating point values in the FPU
registers. YMMV.

If you care to know the precision MLton is using for a given platform, check
regression/real-basic.sml You'll see that MLton behaves the same on every
supported architecture except i386. (but don't depend on this!)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton-user/attachments/20101015/a1c03301/attachment.html