[Pkg-nagios-devel] Bug#849417: Bug#849417: nagios-nrpe-server: segfault during SSL negotiation with older NRPE 2.15 plugin

Sun Jan 1 03:07:24 UTC 2017

Sebastiaan Couwenberg <sebastic at xs4all.nl> writes:

> The debug symbols are already available, no need to a rebuild. Just
> install the nagios-nrpe-server-dbgsym package.
[...]

Thanks, that system is new to me!

>>> Due to the signal handler in NRPE you won't easily get a backtrace since
>>> SIGSEGV is caught too and NRPE just continues instead of terminating. If
>>> you can get a backtrace (with debug symbols installed) that would be
>>> helpful.

It didn't really give me too much trouble.  I think gdb replaces all the
signal handlers anyhow.

To recap my current behavior, in case things maybe changed subtly here,
here's the logging I get in daemon.log with ssl_debug set to 0x0f:

Dec 31 21:37:22 salsa nrpe[24931]: Allowing connections from: 127.0.0.1,192.168.1.5
Dec 31 21:37:27 salsa nrpe[24935]: Connection from 192.168.1.5 port 42463
Dec 31 21:37:27 salsa nrpe[24935]: Host address is in allowed_hosts
Dec 31 21:37:27 salsa nrpe[24935]: Error: Could not complete SSL handshake with 192.168.1.5: 1
Dec 31 21:37:27 salsa nrpe[24935]: Connection from 192.168.1.5 closed.

Whereas if I set it to 0xff:
Dec 31 21:36:23 salsa nrpe[24897]: Allowing connections from: 127.0.0.1,192.168.1.5
Dec 31 21:36:30 salsa nrpe[24899]: Connection from 192.168.1.5 port 41951
Dec 31 21:36:30 salsa nrpe[24899]: Host address is in allowed_hosts

and then in kernl.log:
Dec 31 21:36:30 salsa kernel: [632644.965865] nrpe[24899]: segfault at
b0935335 ip 00007f3fafd3d496 sp 00007ffee43c9dc8 error 4 in
libc-2.24.so[7f3fafcbd000+195000]

Here's my gdb session and the best backtrace I was able to get out:

# gdb /usr/sbin/nrpe 24967
(gdb) set follow-fork-mode child
(gdb) c
Continuing.
[New process 25047]
[New process 25048]

Thread 3.1 "nrpe" received signal SIGSEGV, Segmentation fault.
[Switching to process 25048]
strlen () at ../sysdeps/x86_64/strlen.S:106
106	../sysdeps/x86_64/strlen.S: No such file or directory.
#0  strlen () at ../sysdeps/x86_64/strlen.S:106
#1  0x00007fc8e3c34da3 in _IO_vfprintf_internal (s=s at entry=0x561cf790d280, format=<optimized out>, format at entry=0x561cf6be9eb8 "Error: Could not complete SSL handshake with %s: %s", 
    ap=0x7fff6996e188) at vfprintf.c:1637
#2  0x00007fc8e3ce2f66 in ___vfprintf_chk (fp=fp at entry=0x561cf790d280, flag=flag at entry=1, format=format at entry=0x561cf6be9eb8 "Error: Could not complete SSL handshake with %s: %s", 
    ap=ap at entry=0x7fff6996e188) at vfprintf_chk.c:33
#3  0x00007fc8e3ccfad8 in __GI___vsyslog_chk (pri=<optimized out>, flag=1, fmt=0x561cf6be9eb8 "Error: Could not complete SSL handshake with %s: %s", ap=ap at entry=0x7fff6996e188)
    at ../misc/syslog.c:222
#4  0x00007fc8e3ccffd2 in __syslog_chk (pri=<optimized out>, flag=<optimized out>, fmt=<optimized out>) at ../misc/syslog.c:129
#5  0x0000561cf6be51ba in syslog (__fmt=0x561cf6be9eb8 "Error: Could not complete SSL handshake with %s: %s", __pri=3) at /usr/include/x86_64-linux-gnu/bits/syslog.h:31
#6  handle_conn_ssl (sock=<optimized out>, ssl_ptr=0x561cf78f7b70) at ./nrpe.c:1753
#7  0x0000561cf6be6a53 in handle_connection (sock=6) at ./nrpe.c:1491
#8  0x0000561cf6be7085 in wait_for_connections () at ./nrpe.c:1198
#9  0x0000561cf6be71c3 in run_src () at ./nrpe.c:506
#10 0x0000561cf6be288c in main (argc=<optimized out>, argv=<optimized out>) at ./nrpe.c:198

(gdb) frame 6
#6  handle_conn_ssl (sock=<optimized out>, ssl_ptr=0x561cf78f7b70) at ./nrpe.c:1753
1753	./nrpe.c: No such file or directory.
nerrs = 0
c = <optimized out>
buffer = "\000\000\000\000\000\000\000\000\324\006\000\000\000\000\000\000\250\310\311\344\310\177\000\000\220\375\276\343\310\177\000\000\070п\343\310\177\000\000SI\250\344\310\177\000\000\324\006\000\000\000\000\000\000\070п\343\310\177\000\000\250\310\311\344\310\177\000\000\070\343\226i\377\177\000\000\064\343\226i\377\177\000\000\313B\250\344\310\177\000\000\020\265\370\343\310\177\000\000(\252\370\343\310\177\000\000\070\343\226i\377\177\000\000\066\025\025e\000\000\000\000TT\224\001\000\000\000\000\070п\343\310\177\000\000\020\344\226i\377\177\000\000\220\375\276\343\310\177\000\000\064\343\226i\377\177\000\000\000\344\226i\377\177\000\000PF\306\344\310\177\000\000\b", '\000' <repeats 47 times>...
ssl = 0x561cf78f7b70
peer = <optimized out>
rc = <optimized out>
x = <optimized out>

Let me know if you're still stumped.   I think my next step would be to
have to try to hack sources and come up with a diff which fixes matters.

Also, I'm clearly missing some debug symbols, covering
.../sysdeps/x86_64/strlen.S, but not sure what package I need to install
to cover that.

-- 
...Adam Di Carlo...<adam at onshored.com>.......<URL:http://www.onshored.com/>