Bug#1053861: systemd-journal-remote: systemd-joural-uploading killed by watchdog every 3 minutes

[Tom Cameron]

Package: systemd-journal-remote
Version: 254.5-1
Severity: normal

Dear Maintainer,

After upgrading a test host from stable to testing, I noticed systemd-journal-upload
stopped sending logs to my logging host running Debian stable. The test
host notes that the service is killed for watchdog failure, and when the
service restarts the receiver logs a message about handshake message out
of context.

I have attempted to remove the upload state file, in the event there was
an issue with the last log data uploaded. This did not change the
status.


Source Host:
systemd 254 (254.5-1)

Log entries:
Oct 09 23:40:51 host-04 systemd[1]: Started systemd-journal-upload.service - Journal Remote Upload Service.
Oct 09 23:43:51 host-04 systemd[1]: systemd-journal-upload.service: Watchdog timeout (limit 3min)!
Oct 09 23:43:51 host-04 systemd[1]: systemd-journal-upload.service: Killing process 4709 (systemd-journal) with signal SIGABRT.
Oct 09 23:43:51 host-04 systemd[1]: systemd-journal-upload.service: Main process exited, code=killed, status=6/ABRT
Oct 09 23:43:51 host-04 systemd[1]: systemd-journal-upload.service: Failed with result 'watchdog'.


Destination Host:
systemd 252 (252.17-1~deb12u1)

Log entries:
Oct 09 19:43:51 rhodes systemd-journal-remote[6125]: microhttpd: Error: received handshake message out of context.


I can confirm that no log lines appear from this host on the target
host, while log lines from other Debian stable hosts do appear. The
journal files for this particular host don't seem have any updates since
around the time the upgrade from stable to testing completed.

I have checked the libmicrohttpd bug tracker to see if anything recent
seemed to matchi this behavior in the event that the hosts aren't
handshaking due to incompatible crypto settings, but nothing stands out
to me.

I have also looked at the systemd issue tracker on github, and while
there has historically be several issues with journal-upload, none of
them seem to have been reported recently.



-- System Information:
Debian Release: trixie/sid
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: arm64 (aarch64)

Kernel: Linux 6.5.0-1-arm64 (SMP w/4 CPU threads)
Kernel taint flags: TAINT_CRAP
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages systemd-journal-remote depends on:
ii  libc6              2.37-12
ii  libcurl4           8.3.0-2
ii  libmicrohttpd12    0.9.77-3
ii  libsystemd-shared  254.5-1
ii  systemd            254.5-1

systemd-journal-remote recommends no packages.

systemd-journal-remote suggests no packages.

-- no debconf information