[Pkg-matrix-maintainers] Bug#927057: 1Gb of logs is too much

[Willem Mulder]

On Wed, 24 Apr 2019 01:03:35 +0000 Linda Lapinlampi <linda at lindalap.fi>
wrote:
> I also agree the logs should be compressed on daily rotation, but it
> remains unclear to me how one would change this in Synapse without big
> hacky behaviors. Preferably I'd use logrotate(8) if at all possible.
>
> This might be helpful:
> https://docs.python.org/3/library/logging.handlers.html
>
> So maybe change logging.handlers.RotatingFileHandler in debian/log.yaml
> to WatchFileHandler (use with logrotate(8)) if I've understood
> correctly. TimedRotatingFileHandler is an alternative to let Python
manage log
> rotation by itself, but no logs will be compressed then.

On my personal setup, I have the following changes to log.yaml:

diff --git a/log.yaml.dpkg-dist b/log.yaml
index caa3fac..187bc7c 100644
--- a/log.yaml.dpkg-dist
+++ b/log.yaml
@@ -14,12 +14,12 @@ filters:
 
 handlers:
   file:
-    class: logging.handlers.RotatingFileHandler
+    class: logging.handlers.WatchedFileHandler
     formatter: precise
     filename: /var/log/matrix-synapse/homeserver.log
-    maxBytes: 104857600
-    backupCount: 10
     filters: [context]
+    level: DEBUG
+    encoding: utf8
   console:
     class: logging.StreamHandler
     formatter: precise
@@ -31,12 +31,18 @@ handlers:
     SYSLOG_IDENTIFIER: synapse
 
 loggers:
+    twisted:
+        level: WARN
+
     synapse:
-        level: INFO
+        level: WARN
 
     synapse.storage.SQL:
         level: INFO
 
+    synapse.http.matrixfederationclient:
+        level: ERROR
+
 root:
     level: INFO
     handlers: [file, journal]

And the following in /etc/logrotate.d/matrix-synapse:

/var/log/matrix-synapse/homeserver.log {
    rotate 7
    daily
    missingok
    notifempty
    delaycompress
    compress
}

This gives the following result in /var/log/matrix-synapse:

total 12M
-rw-r--r-- 1 matrix-synapse nogroup 2.0M Apr 24 16:14 homeserver.log
-rw-r--r-- 1 matrix-synapse nogroup 8.4M Apr 24 06:24 homeserver.log.1
-rw-r--r-- 1 matrix-synapse nogroup 295K Apr 23 06:24 homeserver.log.2.gz
-rw-r--r-- 1 matrix-synapse nogroup 253K Apr 22 06:25 homeserver.log.3.gz
-rw-r--r-- 1 matrix-synapse nogroup 258K Apr 21 06:25 homeserver.log.4.gz
-rw-r--r-- 1 matrix-synapse nogroup 277K Apr 20 06:25 homeserver.log.5.gz
-rw-r--r-- 1 matrix-synapse nogroup 292K Apr 19 06:24 homeserver.log.6.gz
-rw-r--r-- 1 matrix-synapse nogroup 311K Apr 18 06:24 homeserver.log.7.gz

In my experience, this leaves logging useful enough to debug problems,
while drastically cutting in used disk space.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/pkg-matrix-maintainers/attachments/20190424/18ea9093/attachment.sig>