[Python-modules-team] Bug#960499: python3-psycopg2: fails with "psycopg2.OperationalError: insufficient data in "T" message"

[Karsten Hilbert]

On Wed, May 13, 2020 at 09:13:42AM -0400, Scott Kitterman wrote:

> > Any ideas where else to look ?
>
> It would surprise me if this turned out to be relevant, but since it's
> probably easy to check, I'll toss it out there:
>
> In psycopg2 2.8 the function where the traceback is triggered has been modified
> to use OrderedDict.
>
> "/usr/lib/python3/dist-packages/psycopg2/extras.py", line 142, in
> execute return super(DictCursor, self).execute(query, vars)
>
> It looks like the commit where that change was made it pretty self contained
> [1].  You might try putting the 2.7 version of DictCursor back (reverting that
> commit) and see if the problem persists.
>
> It's kind of a shot in the dark, but one has to start somewhere.

It turned out to be an unholy interaction between threads
inadvertently sharing a connection. Using a connection per
thread solves the issue.

However, the strange parts:

1) The very same code (sharing the connection between
   threads) _does_ work with psycopg2 2.7.

2) psycopg2 states that sharing connections among threads is
   considered to be safe (threadsafety=2). To note, the threads
   do _not_ share cursors.

At any rate, as far as GNUmed is concerned, the issue has
been fixed or, per(!)spectively, worked around so the bug can
be closed.

Maybe one cannot hold open cursors concurrently on the a
transaction shared by threads.

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B