[Pkg-openssl-devel] Bug#339532: libssl0.9.8: reproducible segfault in bn_sub_words () at bn_asm.sparc.v8plus.S

[Peter Palfrader]

Package: libssl0.9.8
Version: 0.9.8a-4
Severity: normal

Hey,

on sparc64 the following snippet causes a segfault after a few
iterations (usually around i = 30).


#include <stdio.h>
#include <stdlib.h>
#include <openssl/dh.h>
#include <openssl/bn.h>

int main(int v, char **c)
{
  int which=-1, i;
  DH *dh;
  if (v!=2 || ((which=atoi(c[1]))<1 || which > 2)) {
    printf("I want 1 or 2 as an argument.\n");
    return 1;
  }
  printf("Running test %d\n",which);
  for (i=0; i < 10000; ++i) {
    if (!(i%1000)) { printf("...%d\n",i); }
    dh = DH_new();
    dh->g = BN_new();
    dh->p = get_rfc2409_prime_1024(NULL);
    BN_set_word(dh->g, 2);
    if (which == 2) {
      dh->length = 320;
    }
    DH_generate_key(dh);

    DH_free(dh);
  }
  return 0;
}

| maurice:/tmp/nick# ./a.out 1
| Running test 1
| ...0
| Segmentation fault
| maurice:/tmp/nick# ./a.out 2
| Running test 2
| ...0
| Segmentation fault

(gdb) bt
#0  bn_sub_words () at bn_asm.sparc.v8plus.S:555
#1  0x70228c20 in bn_sub_part_words (r=0x22b18, a=0x22930, b=0x22950, cl=7, dl=1) at bn_mul.c:87
#2  0x7022981c in bn_mul_recursive (r=0x22c80, a=0x22828, b=0x22930, n2=16, dna=0, dnb=-1, t=0x22af8) at bn_mul.c:437
#3  0x7022964c in bn_mul_recursive (r=0x22c00, a=0x227e8, b=0x228f0, n2=32, dna=0, dnb=-1, t=0x229f8) at bn_mul.c:499
#4  0x7022a50c in BN_mul (r=0x21074, a=0x21038, b=0x2104c, ctx=0x20f50) at bn_mul.c:1037
#5  0x70230a30 in BN_mod_mul_montgomery (r=0x21038, a=0x21038, b=0x2104c, mont=0x20d38, ctx=0x20f50) at bn_mont.c:89
#6  0x70226cf8 in BN_mod_exp_mont_consttime (rr=0x20f38, a=Variable "a" is not available.) at bn_exp.c:705
#7  0x7024a4c4 in generate_key (dh=0x21270) at dh_key.c:158
#8  0x7024a28c in DH_generate_key (dh=Variable "dh" is not available.) at dh_key.c:76
#9  0x000108f4 in main (v=2, c=0xeffff974) at x.c:24


When rebuilding openssl with the no-asm switch all is fine.

-- 
Peter