[MLton-user] IEEEReal rounding not working

[Sean McLaughlin]

Hi Matthew,

  Yes, I also came to this conclusion

  val mtenth_lo = (down();(Option.valOf ` Real.fromString "1.0") / (
Option.valOf ` Real.fromString "10.0"))
  val mtenth_hi = (up();(Option.valOf ` R.fromString "1.0") / (Option.valOf`
R.fromString "10.0"))

These do the correct thing.

I'm not sure what the correct behavior should be.  I'd hate to lose
optimizations, as I switched
from O'Caml because the floating point computations were so much faster in
my application.
But it would be nice to have a document somewhere specifying exactly how
rounding modes
are treated in expressions.  For instance, I can imagine an optimization
that executes
Real.fromString at compile time for a string constant, which would kill the
fix.


> BTW, would you expect the following output:

3FB9999999999999
> 3FB999999999999A
> 3FB999999999999A
> 3FB9999999999999


Yes, that looks right.

Thanks,

Sean
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton-user/attachments/20070726/6989bce8/attachment.htm