[Pkg-matrix-maintainers] Bug#1026013: matrix-synapse: exceptions in federation
Russell Coker
russell at coker.com.au
Tue Dec 13 08:35:12 GMT 2022
Package: matrix-synapse
Version: 1.72.0-1
Severity: normal
I have 2 accounts on different Matrix-synapse instances on different VMs on the
same hardware. For a while they have been working well. Now I can't talk from
an account one one of them to an account on another. It happened after (but
not immediately after) upgrading to 1.72. Here's the relevant part of
homeserver.log:
2022-12-13 19:23:03,379 - synapse.http.server - 124 - ERROR - GET-55- Failed handle request via 'SyncRestServlet': <SynapseRequest at 0x7f11e06783a0 method='GET' uri='/_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s2749_28677_2_1756_2106_5_308_49_0' clientproto='HTTP/1.1' site='8448'>
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/synapse/storage/database.py", line 710, in new_transaction
r = func(cursor, *args, **kwargs)
File "/usr/lib/python3/dist-packages/synapse/storage/databases/main/push_rule.py", line 198, in have_push_rules_changed_txn
txn.execute(sql, (user_id, last_id))
File "/usr/lib/python3/dist-packages/synapse/storage/database.py", line 388, in execute
self._do_execute(self.txn.execute, sql, *args)
File "/usr/lib/python3/dist-packages/synapse/storage/database.py", line 436, in _do_execute
return func(sql, *args, **kwargs)
psycopg2.OperationalError: SSL SYSCALL error: EOF detected
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/synapse/http/server.py", line 307, in _async_render_wrapper
callback_return = await self._async_render(request)
File "/usr/lib/python3/dist-packages/synapse/http/server.py", line 513, in _async_render
callback_return = await raw_callback_return
File "/usr/lib/python3/dist-packages/synapse/rest/client/sync.py", line 190, in on_GET
sync_result = await self.sync_handler.wait_for_sync_for_user(
File "/usr/lib/python3/dist-packages/synapse/handlers/sync.py", line 301, in wait_for_sync_for_user
res = await self.response_cache.wrap(
File "/usr/lib/python3/dist-packages/synapse/util/caches/response_cache.py", line 265, in wrap
return await make_deferred_yieldable(result)
File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "/usr/lib/python3/dist-packages/synapse/util/caches/response_cache.py", line 246, in cb
return await callback(*args, **kwargs)
File "/usr/lib/python3/dist-packages/synapse/handlers/sync.py", line 369, in _wait_for_sync_for_user
result = await self.notifier.wait_for_events(
File "/usr/lib/python3/dist-packages/synapse/notifier.py", line 562, in wait_for_events
result = await callback(prev_token, current_token)
File "/usr/lib/python3/dist-packages/synapse/handlers/sync.py", line 367, in current_sync_callback
return await self.current_sync_for_user(sync_config, since_token)
File "/usr/lib/python3/dist-packages/synapse/handlers/sync.py", line 410, in current_sync_for_user
sync_result = await self.generate_sync_result(
File "/usr/lib/python3/dist-packages/synapse/handlers/sync.py", line 1393, in generate_sync_result
account_data_by_room = await self._generate_sync_entry_for_account_data(
File "/usr/lib/python3/dist-packages/synapse/handlers/sync.py", line 1658, in _generate_sync_entry_for_account_data
push_rules_changed = await self.store.have_push_rules_changed_for_user(
File "/usr/lib/python3/dist-packages/synapse/storage/databases/main/push_rule.py", line 202, in have_push_rules_changed_for_user
return await self.db_pool.runInteraction(
File "/usr/lib/python3/dist-packages/synapse/storage/database.py", line 881, in runInteraction
return await delay_cancellation(_runInteraction())
File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "/usr/lib/python3/dist-packages/synapse/storage/database.py", line 848, in _runInteraction
result = await self.runWithConnection(
File "/usr/lib/python3/dist-packages/synapse/storage/database.py", line 976, in runWithConnection
return await make_deferred_yieldable(
File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
File "/usr/lib/python3/dist-packages/twisted/enterprise/adbapi.py", line 306, in _runWithConnection
compat.reraise(excValue, excTraceback)
File "/usr/lib/python3/dist-packages/twisted/python/compat.py", line 464, in reraise
raise exception.with_traceback(traceback)
File "/usr/lib/python3/dist-packages/twisted/enterprise/adbapi.py", line 297, in _runWithConnection
result = func(conn, *args, **kw)
File "/usr/lib/python3/dist-packages/synapse/storage/database.py", line 969, in inner_func
return func(db_conn, *args, **kwargs)
File "/usr/lib/python3/dist-packages/synapse/storage/database.py", line 728, in new_transaction
conn.rollback()
File "/usr/lib/python3/dist-packages/synapse/storage/database.py", line 197, in rollback
self.conn.rollback()
File "/usr/lib/python3/dist-packages/twisted/enterprise/adbapi.py", line 70, in rollback
raise ConnectionLost()
twisted.enterprise.adbapi.ConnectionLost
-- System Information:
Debian Release: 11.5
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 5.10.0-19-amd64 (SMP w/2 CPU threads)
Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: SELinux: enabled - Mode: Enforcing - Policy name: default
Versions of packages matrix-synapse depends on:
ii adduser 3.118
ii debconf [debconf-2.0] 1.5.77
ii init-system-helpers 1.60
ii libc6 2.36-5
ii libgcc-s1 10.2.1-6
ii libjs-jquery 3.5.1+dfsg+~3.5.5-7
ii libpython3-stdlib 3.9.2-3
ii lsb-base 11.1.0
ii python3 3.9.2-3
ii python3-attr 20.3.0-1
ii python3-bcrypt 3.2.0-1~bpo11+1
ii python3-bleach 3.2.1-2.1
ii python3-canonicaljson 1.6.2-1
ii python3-cryptography 3.3.2-1
ii python3-distutils 3.9.2-1
ii python3-frozendict 1.2-3~bpo11+1
ii python3-ijson 3.1.4-1
ii python3-importlib-metadata 1.6.0-2
ii python3-jinja2 3.0.3-1~bpo11+1
ii python3-jsonschema 3.2.0-3
ii python3-lxml 4.6.3+dfsg-0.1+deb11u1
ii python3-matrix-common 1.3.0-2~bpo11+3
ii python3-msgpack 1.0.0-6+b1
ii python3-netaddr 0.7.19-5
ii python3-openssl 20.0.1-1
ii python3-packaging 20.9-2
ii python3-phonenumbers 8.12.1-1
ii python3-pil 8.1.2+dfsg-0.3+deb11u1
ii python3-prometheus-client 0.9.0-1
ii python3-psycopg2 2.8.6-2
ii python3-pyasn1 0.4.8-1
ii python3-pyasn1-modules 0.2.1-1
ii python3-pydantic 1.10.2-1
ii python3-pymacaroons 0.13.0-4
ii python3-service-identity 18.1.0-6
ii python3-signedjson 1.1.1-2
ii python3-sortedcontainers 2.1.0-2
ii python3-systemd 234-3+b4
ii python3-treq 18.6.0-0.2
ii python3-twisted 20.3.0-7+deb11u1
ii python3-typing-extensions 4.3.0-2
ii python3-unpaddedbase64 2.1.0-2
ii python3-yaml 5.3.1-5
Versions of packages matrix-synapse recommends:
pn matrix-synapse-ldap3 <none>
pn python3-pympler <none>
Versions of packages matrix-synapse suggests:
pn python3-authlib <none>
ii python3-jwt 1.7.1-2
-- Configuration Files:
/etc/matrix-synapse/homeserver.yaml changed [not included]
-- debconf information:
* matrix-synapse/server-name: coker.com.au
* matrix-synapse/report-stats: true
More information about the Pkg-matrix-maintainers
mailing list