[Pkg-openssl-devel] Bug#670581: Bug#670581: openssl: ntpd segfaults with error 4 in libcrypto.so.0.9.8 on Debian squeeze

[Andris Kalnozols]

On 4/28/2012 1:57 AM, Kurt Roeckx wrote:
> On Fri, Apr 27, 2012 at 05:42:07PM -0700, Andris Kalnozols wrote:
>> On 4/27/2012 4:11 PM, Kurt Roeckx wrote:
>>> header and hdlen from frame 1 would be nice.
>>>
>>> Could you also install the libssl0.9.8-dbg package?  You'll get
>>> debug symbols then.
>>>
>>
>> I'm not a real pro with the debugger.  After installing the `dbg'
>> package, is
>> this all that's needed?:
>
> gdb should look load the debug symbols from
> /usr/lib/debug/usr/lib/libcrypto.so.0.9.8 automaticly,
> there is no need to add the symbol files in gdb.
>
>> Modifying the Makefile to link ntpd using libcrypto in
>> /usr/lib/debug/usr/lib
>
> There is also no need for that, and it won't work, because it
> only contains the debug symbols.
>
> The library in /usr/lib/ tells that it's debug symbols have been
> removed and placed in the file in /usr/lib/debug/usr/lib/, and gdb
> should automaticly pick this up.

Thanks, Kurt.

Before I got your reply, I went to "Plan B" and did the following:

   * apt-get source openssl
   * changed the optimization from -O3 to -O0 in the Configure script
     for the "debian-amd64" section
   * make -f debian/rules
   * relinked the ntpd program with the libcrypto.static library

I also added the following code:

--- ntp_crypto.c.orig   2009-12-08 23:36:35.000000000 -0800
+++ ntp_crypto.c        2012-04-29 15:16:50.181208921 -0700
@@ -230,6 +230,38 @@
                 break;
         }
         EVP_DigestInit(&ctx, EVP_get_digestbynid(crypto_nid));
+       /*
+        * Begin debugging output
+        */
+       switch(AF(srcadr)) {
+       char src_buf[INET6_ADDRSTRLEN],
+            dst_buf[INET6_ADDRSTRLEN];
+
+       case AF_INET:
+               inet_ntop(AF_INET, &header[0], src_buf, INET6_ADDRSTRLEN);
+               inet_ntop(AF_INET, &header[1], dst_buf, INET6_ADDRSTRLEN);
+               msyslog(LOG_INFO, "session_key() IPv4 data: header[0]=%u, "
+                                 "src_addr=%s, header[1]=%u, dst_addr=%s",
+                                 header[0], src_buf, header[1], dst_buf);
+               break;
+       case AF_INET6:
+               inet_ntop(AF_INET6, &header[0], src_buf, INET6_ADDRSTRLEN);
+               inet_ntop(AF_INET6, &header[4], dst_buf, INET6_ADDRSTRLEN);
+               msyslog(LOG_INFO, "session_key() IPv6 data: 
header[0-3]=%u, "
+                                 "%u, %u, %u, src_addr=%s, 
header[4-7]=%u, "
+                                 "%u, %u, %u, dst_addr=%s", header[0],
+                                 header[1], header[2], header[3], src_buf,
+                                 header[4], header[5], header[6], 
header[7],
+                                 dst_buf);
+               break;
+       }
+       msyslog(LOG_INFO, "remaining session_key() data [host byte order]: "
+                         "hdlen=%u, keyno=%u, private=%u, crypto_nid=%u, "
+                         "ctx.flags=%lu",
+                         hdlen, keyno, private, crypto_nid, ctx.flags);
+       /*
+        * End debugging output
+        */
         EVP_DigestUpdate(&ctx, (u_char *)header, hdlen);
         EVP_DigestFinal(&ctx, dgst, &len);
         memcpy(&keyid, dgst, 4);

This logged the following output:

29 Apr 16:06:29 ntpd[23221]: session_key() IPv4 data: 
header[0]=3096497361, src_addr=209.204.144.184,
    header[1]=84048844, dst_addr=204.123.2.5
29 Apr 16:06:29 ntpd[23221]: remaining session_key() data [host byte 
order]: hdlen=16, keyno=0,
    private=3604904099, crypto_nid=4, ctx.flags=0

A full backtrace of the first two frames after the segfault shows:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff7fe9700 (LWP 23221)]
0x0000000000497c02 in EVP_DigestUpdate (ctx=0x7fffffffda20,
     data=0x7fffffffd9b0, count=16) at digest.c:325
325             return ctx->digest->update(ctx,data,count);
(gdb) backtrace full
#0  0x0000000000497c02 in EVP_DigestUpdate (ctx=0x7fffffffda20,
     data=0x7fffffffd9b0, count=16) at digest.c:325
No locals.
#1  0x000000000041d23f in session_key (srcadr=0x8110f0, dstadr=0x8171b0,
     keyno=0, private=3604904099, lifetime=0) at ntp_crypto.c:265
         ctx = {digest = 0x0, engine = 0x0, flags = 0, md_data = 0x0}
         dgst = "%\000\000\000\000\000\000\000\035JB\000\000\000\000
                 \000\004\000\000\000\377\177\000\000P\266\377\367
                 \377\177\000\000\000\003G\323\000\000\000\000<\002A
                 \000\000\000\000\000P\332\377\377\377\177\000\000
                 \000\000\000\000\000\000\000"
         keyid = 32767
         header = {3096497361, 84048844, 0, 2742607574, 4294960736,
                   32767, 0, 0, 0, 0}
         hdlen = 16
         len = 32767

The other notable thing is that it's always the same source IP address
[209.204.144.184] which appears before the server aborts.  A packet
capture of traffic from/to this IP address is at:

   ftp://ftp.hpl.hp.com/outgoing/andris/openssl/ntpd.pcap

This client IP address did not abort the server right away.  Five NTP
queries were answered.  The sixth query is what caused this abort.

Regards
Andris