[Git][qa/jenkins.debian.net][master] 2 commits: reproducble opensuse: sqla 2.0: fix delete() call

[Mattia Rizzolo (@mattia)]

Mattia Rizzolo pushed to branch master at Debian QA / jenkins.debian.net


Commits:
456cd734 by Mattia Rizzolo at 2025-08-11T17:57:47+02:00
reproducble opensuse: sqla 2.0: fix delete() call

Signed-off-by: Mattia Rizzolo <mattia at debian.org>

- - - - -
bc5a1deb by Mattia Rizzolo at 2025-08-11T18:03:09+02:00
reproducible opensuse: move everything into a single transaction

just try to have more consistent data if possible

Also, there seemed to be some confusion between query_db() and
conn_db.execute(), being mixed up where they shouldn't.

Signed-off-by: Mattia Rizzolo <mattia at debian.org>

- - - - -


1 changed file:

- bin/reproducible_opensuse_import_json.py


Changes:

=====================================
bin/reproducible_opensuse_import_json.py
=====================================
@@ -1,7 +1,7 @@
 #!/usr/bin/python3
 # -*- coding: utf-8 -*-
 #
-# Copyright © 2018 Mattia Rizzolo <mattia at mapreri.org>
+# Copyright © 2018-2025 Mattia Rizzolo <mattia at mapreri.org>
 #
 # Licensed under GPL-2
 #
@@ -50,60 +50,58 @@ for pkg in ostatus:
     pkgs_b[(pkg['package'], pkg['version'])] = data
 
 
-# just insert everything for now
-
-log.info('Dropping old data…')
-transaction = conn_db.begin()
-d = results.delete(results.c.package_id.in_(
-    select(sources.c.id).select_from(sources).where(sources.c.distribution == distro_id)
-))
-query_db(d)
-d = sources.delete(sources.c.distribution == distro_id)
-query_db(d)
-transaction.commit()
-
-
-log.info('Injecting new source packages…')
-transaction = conn_db.begin()
-conn_db.execute(sources.insert(), pkgs)
-transaction.commit()
-
-
-log.info('Injecting build results…')
-cur_pkgs = select(
-    sources.c.id, sources.c.name, sources.c.version,
-     sources.c.suite, sources.c.architecture
-).select_from(
-    sources.join(distributions)
-).where(
-    and_(
-        distributions.c.name == 'opensuse',
-        sources.c.suite == bindparam('suite'),
-        sources.c.architecture == bindparam('arch')
+# just insert everything for now, in a single transaction
+
+with conn_db.begin() as transaction:
+    log.info('Dropping old data…')
+    d = results.delete().where(results.c.package_id.in_(
+        select(sources.c.id).select_from(sources).where(
+            sources.c.distribution == distro_id
+        )
+    ))
+    conn_db.execute(d)
+    d = sources.delete().where(sources.c.distribution == distro_id)
+    conn_db.execute(d)
+
+    log.info('Injecting new source packages…')
+    conn_db.execute(sources.insert(), pkgs)
+
+    log.info('Injecting build results…')
+    cur_pkgs = select(
+        sources.c.id, sources.c.name, sources.c.version,
+        sources.c.suite, sources.c.architecture
+    ).select_from(
+        sources.join(distributions)
+    ).where(
+        and_(
+            distributions.c.name == 'opensuse',
+            sources.c.suite == bindparam('suite'),
+            sources.c.architecture == bindparam('arch')
+        )
+    )
+    cur_pkgs = conn_db.execute(
+        cur_pkgs.params({'suite': 'factory', 'arch': 'x86_64'})
     )
-)
-cur_pkgs = query_db(cur_pkgs.params({'suite': 'factory', 'arch': 'x86_64'}))
 
-builds = []
-for pkg in cur_pkgs:
-    # (id, name, version, suite, architecture)
-    data = pkgs_b[(pkg[1], pkg[2])]
-    if data['status'] == 'nobinaries':
-        continue
-    pkg_status = data['status'].\
-        replace('unreproducible', 'FTBR').\
-        replace('notforus', 'NFU').\
-        replace('waitdep', 'depwait')
-    p = {
-        'package_id': pkg[0],
-        'version': pkg[2],
-        'status': pkg_status,
-        'build_date': data['build_date'],
-        'build_duration': data['build_duration'],
-        'job': 'external',
-    }
-    builds.append(p)
-if builds:
-    transaction = conn_db.begin()
-    conn_db.execute(results.insert(), builds)
-transaction.commit()
+    builds = []
+    for pkg in cur_pkgs:
+        # (id, name, version, suite, architecture)
+        data = pkgs_b[(pkg[1], pkg[2])]
+        if data['status'] == 'nobinaries':
+            continue
+        pkg_status = data['status'].\
+            replace('unreproducible', 'FTBR').\
+            replace('notforus', 'NFU').\
+            replace('waitdep', 'depwait')
+        p = {
+            'package_id': pkg[0],
+            'version': pkg[2],
+            'status': pkg_status,
+            'build_date': data['build_date'],
+            'build_duration': data['build_duration'],
+            'job': 'external',
+        }
+        builds.append(p)
+    if builds:
+        conn_db.execute(results.insert(), builds)
+    transaction.commit()



View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/compare/cc77f894b8ea15eef365f30d94e08bab8b0d771b...bc5a1deb4b6d6ddacd008e3c87d7ebcccc671e73

-- 
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/compare/cc77f894b8ea15eef365f30d94e08bab8b0d771b...bc5a1deb4b6d6ddacd008e3c87d7ebcccc671e73
You're receiving this email because of your account on salsa.debian.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20250811/0bc5acc5/attachment-0001.htm>