[Git][qa/jenkins.debian.net][master] reproducible debian: sqla, wrap all queries into transactions

[Mattia Rizzolo (@mattia)]

Mattia Rizzolo pushed to branch master at Debian QA / jenkins.debian.net


Commits:
ef1b588c by Mattia Rizzolo at 2025-08-11T17:42:52+02:00
reproducible debian: sqla, wrap all queries into transactions

this saves me the effort of figuring out how 2.0 triggers

    sqlalchemy.exc.InvalidRequestError: This connection has already initialized a SQLAlchemy Transaction() object via begin() or autobegin; can't call begin() here unless rollback() or commit() is called first.

in scheduler.

While doing that, I can also drop some manual begin/commit that were
being done in the scheduler.

Signed-off-by: Mattia Rizzolo <mattia at debian.org>

- - - - -


3 changed files:

- bin/rblib/__init__.py
- bin/reproducible_json.py
- bin/reproducible_scheduler.py


Changes:

=====================================
bin/rblib/__init__.py
=====================================
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright © 2015-2018 Mattia Rizzolo <mattia at debian.org>
+# Copyright © 2015-2025 Mattia Rizzolo <mattia at debian.org>
 # Copyright © 2015-2017 Holger Levsen <holger at layer-acht.org>
 # Licensed under GPL-2
 
@@ -44,9 +44,11 @@ def query_db(query, *args, **kwargs):
         # be lazy (and probably a bit unsafe) and make them into ClauseElement
         query = text(query)
     try:
-        result = conn_db.execute(query, *args, **kwargs)
+        with conn_db.beging() as transaction:
+            result = conn_db.execute(query, *args, **kwargs)
+            transaction.commit()
     except OperationalError as ex:
-        print_critical_message('Error executing this query:\n' + query)
+        print_critical_message(f'Error «{ex}» executing this query:\n{query}')
         raise
 
     if result.returns_rows:
@@ -68,11 +70,11 @@ def get_trailing_bug_icon(bug, bugs, package=None):
             if bug in bugs[package].keys():
                 html += '<span class="'
                 if bugs[package][bug]['done']:
-                    html += 'bug-done" title="#' + str(bug) + ', done">#'
+                    html += f'bug-done" title="#{bug}, done">#'
                 elif bugs[package][bug]['pending']:
-                    html += 'bug-pending" title="#' + str(bug) + ', pending">P'
+                    html += f'bug-pending" title="#{bug}, pending">P'
                 elif bugs[package][bug]['patch']:
-                    html += 'bug-patch" title="#' + str(bug) + ', with patch">+'
+                    html += f'bug-patch" title="#{bug}, with patch">+'
                 else:
                     html += 'bug">'
                 html += '</span>'


=====================================
bin/reproducible_json.py
=====================================
@@ -1,8 +1,8 @@
 #!/usr/bin/python3
 # -*- coding: utf-8 -*-
 #
-# Copyright © 2015-2018 Mattia Rizzolo <mattia at mapreri.org>
-# Copyright © 2015-2025 Holger Levsen <holger at layer-acht.org>
+# Copyright © 2015-2025 Mattia Rizzolo <mattia at mapreri.org>
+# Copyright © 2015-2023 Holger Levsen <holger at layer-acht.org>
 # Based on reproducible_json.sh
 # © 2014 Holger Levsen <holger at layer-acht.org>
 # Licensed under GPL-2


=====================================
bin/reproducible_scheduler.py
=====================================
@@ -1,7 +1,7 @@
 #!/usr/bin/python3
 # -*- coding: utf-8 -*-
 #
-# Copyright © 2015-2023 Mattia Rizzolo <mattia at mapreri.org>
+# Copyright © 2015-2025 Mattia Rizzolo <mattia at mapreri.org>
 # Copyright © 2014-2025 Holger Levsen <holger at layer-acht.org>
 #
 # Licensed under GPL-2
@@ -319,19 +319,15 @@ def update_sources_db(suite, arch, sources):
     log.info('Pushing ' + str(len(updated_pkgs)) +
              ' updated packages to the database...')
     if updated_pkgs:
-        transaction = conn_db.begin()
         update_query = sources_table.update().\
             where(sources_table.c.id == sql.bindparam('update_id'))
-        conn_db.execute(update_query, updated_pkgs)
-        transaction.commit()
+        query_db(update_query, updated_pkgs)
 
     # new packages
     if pkgs_to_add:
         log.info('Now inserting %i new sources in the database: %s',
                  len(pkgs_to_add), pkgs_to_add)
-        transaction = conn_db.begin()
-        conn_db.execute(sources_table.insert(), pkgs_to_add)
-        transaction.commit()
+        query_db(sources_table.insert(), pkgs_to_add)
 
     # RM'ed packages
     cur_pkgs_name = [x[0] for x in cur_pkgs]
@@ -426,7 +422,7 @@ def schedule_packages(packages):
     ]
     log.debug('IDs about to be scheduled: %s', packages.keys())
     if pkgs:
-        conn_db.execute(db_table('schedule').insert(), pkgs)
+        query_dbe(db_table('schedule').insert(), pkgs)
 
 
 def add_up_numbers(packages, arch):



View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/ef1b588cd76c42c34f85376c2336a51ae80fe443

-- 
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/ef1b588cd76c42c34f85376c2336a51ae80fe443
You're receiving this email because of your account on salsa.debian.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20250811/b3daa0cc/attachment-0001.htm>