Real.maxFinite bug

Stephen Weeks MLton@sourcelight.com
Tue, 2 Oct 2001 13:30:10 -0700


> It wasn't clear to me from the start of this discussion why exactly
> maxFinite, minPos, and minNormalPos are _computed_ values? 

Because we encountered some portability problems back in June 1999 going from
RedHat 5.2 to 6.0.  At the time, we decided that since there is a simple way to
compute the values, to go ahead and do it.

> It seems that this is
> sensitive to the binary encoding of floating-point numbers;

Yep.

> and if you're
> depending on IEEE, then just construct the value as a pair of words (or a
> packed real) and cast it.

I don't have an objection to this approach.  We should probably keep around the
code somewhere so when we need to compute the constants again for some new arch
(just once, of course) we can do so easily.

> maxFinite doesn't have the exact same issue; 
> the resulting value will be of the form a_n*2^n + a_{n-1}*2^{n-1} + ... +
> a_1*2 + a_0*2^0 + a_{-1}*2^{-1} + ...  where a_i \in {0,1} and there is an
> I < n such that a_i = 1 for i > I and a_i = 0 for I <= I. But this again
> assumes that the binary encoding of the maximum finite number is of this
> form.

True.  We assumed binary, which didn't seem too restrictive.

Anyways, I'm happy with 8 byte constants, as long as Henry is.