Re: [PACTH,libm] hypothl(x) mishandles subnormal numbers.

> On 10 Feb 2021, at 00:15, Steve Kargl <sgk_at_troutmask.apl.washington.edu> wrote:
> 
> On Sat, Feb 06, 2021 at 10:32:33PM +0100, Dimitry Andric wrote:
>> On 6 Feb 2021, at 22:04, Steve Kargl <sgk_at_troutmask.apl.washington.edu> wrote:
>>> 
>>> On Sat, Feb 06, 2021 at 12:39:29PM -0800, Steve Kargl wrote:
>>>> I've long forgotten by freebsd bugzilla password.
>>>> So, if someone would like to submit a bug report,
>>>> here's a test program.
>>>> 
>>> Forgot to include that issue was identified from
>>> a bug report in the OpenLibm bug mailing list.
>>> 
>>> https://github.com/JuliaMath/openlibm/issues/224
>> 
>> I put this in <https://bugs.freebsd.org/253313>. Now the trick is to
>> figure out what is going on in e_hypotl.c... :)
>> 
> 
> This patch fixes the issue.  t1 is used to scale the subnormal
> numbers.  It is generated by scaling the exponent, but that
> only works if t1 is 1 not 0.
> 
> Index: src/e_hypotl.c
> ===================================================================
> --- src/e_hypotl.c	(revision 2342)
> +++ src/e_hypotl.c	(working copy)
> _at__at_ -82,7 +82,7 _at__at_ hypotl(long double x, long double y)
> 	        man_t manh, manl;
> 		GET_LDBL_MAN(manh,manl,b);
> 		if((manh|manl)==0) return a;
> -		t1=0;
> +		t1=1;
> 		SET_HIGH_WORD(t1,ESW(MAX_EXP-2));	/* t1=2^(MAX_EXP-2) */
> 		b *= t1;
> 		a *= t1;
> 

Ah, having looked at the glibc code, I had concluded something similar
in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253313#c2, but
using INSERT_LDBL80_WORDS(t1,ESW(MAX_EXP-2),0x8000000000000000).

Your solution is a lot simpler though. :) Note that to make it work, I
also needed to insert a volatile into the INSERT_LDBL80_WORDS() macro.

There are more places where this is apparently needed, due to the way
recent clang versions tend to over-optimize floating point code at
compile time. And specifically for the case where one union field is
written, and then another field read, sometimes leading to unexpected
results...

In any case, I will apply this soon, and also add a test case. Thanks!

-Dimitry