[Pkg-clamav-devel] Bug#749027: Bug#749027: The ClamAV daemon stops working.

[Jim Barber]

Hi Andreas.

On 29/05/2014 5:10 PM, Andreas Cadhalpun wrote:
> On 28.05.2014 02:55, Jim Barber wrote:
>> On 2014-05-27 18:39, Andreas Cadhalpun wrote:
>>> I think the problem here is that there is a second 'Restarting
>>> on-access scan' message only 4 seconds after another one and without
>>> the usual 'ERROR: ScanOnAccess: fanotify_init [...]' between them.
>>>
>>> This looks as if the reload is started again, while the previous
>>> reload is not yet finished. Probably this should not happen, as it
>>> seems to cause a deadlock.
>>
>> It does appear that way.
>> Higher up in the log there are also instances where two restarts happened
>> within seconds of each other, but I guess were far enough apart to
>> avoid the race/deadlock situation that the daemon seems to be encountering.
>
> Yes, it's strange. I tried to reproduce this, but failed. What I did:
>   * Set ScanOnAccess to true.
>   * Set SelfCheck to 10 seconds.
>   * Stop the freshclam daemon: sudo service clamav-freshclam stop
>   * Delete /var/lib/clamav/bytecode.cvd.
>   * Run 'freshclam --daemon-notify=/etc/clamav/clamd.conf'.
>
> Repeating the last two steps, I saw three different patterns in the log.
>   * Only one reload, triggered by freshclam.
>   * Two reloads, but far enough apart, so that the 'ERROR:' message
>     appears between them.
>   * Two reloads, but without the 'ERROR:' message between them.
>
> Since it didn't hang in any of my tries, there must either be a really precise timing to cause the hang and I was lucky not to have
> it or there is something else needed.

Since I enabled 'Debug true' in clamd.conf I haven't had a crash.
Doing that must have modified the timing just enough to avoid the
race/deadlock situation by the looks of things.
It's been quite a number of days since I did this change now...

>> I have upgraded to 0.98.4-rc1 again, but at this stage I've left the
>> freshclam notications on to see if we can find the cause of the hangs.
>
> Thanks for your help in finding the cause.
> Since the logs are inconclusive, it would really help if you could get us a backtrace, when clamd is hanging.
> For this install clamav-dbg and gdb.
> When it hangs again, please determine the PIDs of all clamd threads, e.g. with:
> ps -eL | grep clamd
> The second column contains this number.
> For every thread of clamd, run (replacing <PID> with the actual number):
> sudo gdb /usr/sbin/clamd <PID>
> And then in gdb:
> bt full
> The output of this should help to figure out, where exactly clamd is hanging.

Okay. When I get a chance I'll install the -dbg packages and I'll set
'Debug false' again since it seems to be preventing the hang, and I'll see
if it hangs with the dbg packages in place and get the gdb info.

If the -dbg packages also prevent the hang because of different timings then
I guess we're out of luck to find the real cause of the hangs...

Regards,

Jim.