Real.maxFinite and friends

Matthew Fluet Matthew Fluet <fluet@CS.Cornell.EDU>
Fri, 5 Oct 2001 14:05:19 -0400 (EDT)


> With  regards  to  isNormal and isFinite being made primitives, I would think
> that the more efficient way would be
>     store float to memory
>     load float from memory
>     test float register
> Aren't there floating point instructions for  testing  if  a  floating  point
> register is `normal'?  

Yes, but is there a problem with normality of extended precision numbers?
The problem I'm thinking of, which may not exist in practice, is that I
save minPos (a denormal number in 64bits) to memory, fetch it back, and
test it, but might that be a normal number in 80bits.  I may be totally
wrong.

BTW, I tried setting the precision control field for the FPU, and while
that did change the intermediate results (as inspected by gdb), the final
calculation of minPos always came out to 0.0.

I also realized that with -native-live-stack true, more floats are carried
in the FP stack, so there is additional opportunity for deviation from
IEEE standard.  Although, I'm bothered less by the deviation from IEEE and
more by the fact that (minor) changed in register allocation could lead to
different results.

> As to the problems of different versions of the `same'
> value, some of which have been converted to 64 bit and some of which haven't,
> wouldn't  the  above code also `solve' that problem since now you can use the
> floating point register you just loaded.  (This might require that the  store
> was to where the value `really' lives, if there is such a location.)

Yes, this does solve the problem, and everything has a place where it
really lives.  The real issue is that equational reasoning is all shot to
hell:

val r1 = Real.minPos / 3.0
val r2 = r1
val b1 = Real.isNormal r1
val b2 = Real.==(r1, r2)
val _ = print (...)

This would probably come out o.k., because MLton will rewrite away r2's
with r1's, and then Real.==(r1, r1) will be simplified to true.  But, I
think I can craft an example where the fact that we isNormal'ed one of two
otherwise equal values would cause them to become non-equal.

Anyways, this just tends to make my head hurt.  And since no numerical
analysts have complained, I'm not that concerned.

BTW, Steve, could you see what the MLKit's native backend does with
real7.sml, with and without the hack.