Bug#560026: [PATCH] yorick test2.i fails with "mathlib signals error" on ia64

[Stephan Schreiber]

found 560026 2.2.02+dfsg-6
notfound 560026 2.1.05 2.1.05+dfsg+cvs20091202-1
# these versions are over; I think, nobody wants to patch them anymore
tags 560026 - help
tags 560026 + patch
thanks



The problem is in the source code file yorick/std0.c, line 1355, in  
the function expLoop():

static void expLoop(double *dst, double *src, long n)
{
   long i;
   for (i=0 ; i<n ; i++) {
     dst[i]= exp(src[i]);
     if (errno) { if (errno==ERANGE && !dst[i]) errno=0; else break; }
   }
}


The function calculates the mathematical exp function for each item of  
the src array.
One item is -744.668 (IEEE754 representation C08745590D710CC1) when  
the test runs. The exp() function sets errno to 34 (ERANGE) upon that  
and returns the IEEE754 representation 0000000000000001. This is  
around 4.94066E-324, a denormalized value; means underflow.

As you see above, the idea of the expLoop() code is to commute a  
result to 0 if exp() returns an underflow upon (large) negative  
exponents.

Unfortunately it doesn't work for the example since the check for a  
result of 0 - the
!dst[i]
evaluates to 0 (false).
Since errno isn't 0 after that, the yorick interpreter stops the  
execution of the script and reports the seen error message:
ERROR (escout) mathlib function signals error
   LINE: 251  FILE: /usr/lib/yorick/i/test2.i
yorick: quitting on error in batch mode

Obviously the floating point arithmetic doesn't behave in the same way  
on the different archs in 'subnormal' cases.
I assume, the exp() function returns denormalized values on ia64 but  
round them down to 0 on other archs.

My proposal is to the check the arg of exp() rather than the result in  
order to determine whether it is an underflow or not:
(Note that the pointer dst may be equal to the src pointer.)

static void expLoop(double *dst, double *src, long n)
{
   long i;
   for (i=0 ; i<n ; i++) {
     double e = src[i];
     dst[i]= exp(e);
     if (errno) { if (errno==ERANGE && e<0.0) errno=0; else break; }
   }
}

The test does no longer fail, and can be enabled on ia64.

Stephan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix-exp-underflow.patch
Type: application/octet-stream
Size: 931 bytes
Desc: fix-exp-underflow.patch
URL: <http://lists.alioth.debian.org/pipermail/debian-science-maintainers/attachments/20130106/c919eb76/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: enable-test-on-ia64.patch
Type: application/octet-stream
Size: 726 bytes
Desc: enable-test-on-ia64.patch
URL: <http://lists.alioth.debian.org/pipermail/debian-science-maintainers/attachments/20130106/c919eb76/attachment-0001.obj>