[Piuparts-devel] Bug#996771: piuparts: fails to parse docker container ID if pulling
Philipp Hahn
hahn at univention.de
Mon Oct 18 13:29:40 BST 2021
Package: piuparts
Version: 1.1.4
Severity: normal
Dear Maintainer,
`piuparts --docker-image $img` fails if the image is not yet available
and needs to be pulled from a repote docker image repository first. In
that case the output of `docker run` contains progress information in
addition to the container ID.
The 1. attached patch fixed this by using `--cidfile`, which forces
`docker run` to write the generated container ID into that named file,
from where it is read afterwards.
The 2. patch simplifies getting the merged directory path from `docker
inspect`.
The 3. patch is optional and helps with debugging by always using log
level `DUMP` for the given log file and using `--log-level` only for
logging to STDOUT.
<https://docs.docker.com/engine/reference/run/#pid-equivalent>
-- System Information:
Debian Release: 11.1
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 5.10.0-9-amd64 (SMP w/4 CPU threads)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), LANGUAGE=de:en_US
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages piuparts depends on:
ii debootstrap 1.0.123
pn debsums <none>
ii libjs-sphinxdoc 3.4.3-2
ii lsb-release 11.1.0
ii lsof 4.93.2+dfsg-1.1
ii mount 2.36.1-8
pn piuparts-common <none>
ii python3 3.9.2-3
ii python3-debian 0.1.39
Versions of packages piuparts recommends:
pn adequate <none>
Versions of packages piuparts suggests:
ii docker.io 20.10.5+dfsg1-1+b5
pn schroot <none>
-------------- next part --------------
>From 74fc04769f062cc9818512d5e33755a8c42e2616 Mon Sep 17 00:00:00 2001
Message-Id: <74fc04769f062cc9818512d5e33755a8c42e2616.1634559960.git.hahn at univention.de>
From: Philipp Hahn <hahn at univention.de>
Date: Mon, 18 Oct 2021 12:59:50 +0200
Subject: [PATCH 1/3] fix[docker] getting Container ID
in case the image does not exists locally docker will pull it from any
remote registry, but will include the progress information in the
output:
> Unable to find image 'docker-registry.XXX/YYY:ZZZ' locally
> ZZZ: Pulling from YYY
...
> Digest: sha256:4b9bfe7b0a6c970e3613c04f267ba6319cfceb8cc120b0435d9ee7b8037a1f06
> Status: Downloaded newer image for docker-registry.XXX/YYY:ZZZ
> 8be38c89d12937b98c8be5ab7466dd45b0e4a306862f282b58077ac7193251eb
The old code expected the output to be the container ID, which fails in that case.
---
piuparts.py | 15 ++++++++++-----
1 file changed, 10 insertions(+), 5 deletions(-)
diff --git a/piuparts.py b/piuparts.py
index f05db2ff..be99abf7 100644
--- a/piuparts.py
+++ b/piuparts.py
@@ -922,11 +922,16 @@ class Chroot:
def setup_from_docker(self, docker_image):
self.check_if_docker_storage_driver_is_supported()
- ret_code, output = run(['docker', 'run', '-d', '-it', docker_image, 'bash'])
- if ret_code != 0:
- logging.error("Couldn't start the container from '%s'" % docker_image)
- panic()
- self.docker_container = output.strip()
+ with tempfile.TemporaryDirectory() as tmpdir:
+ tmp = os.path.join(tmpdir, "cid")
+ ret_code, output = run(['docker', 'run', '-d', '-it', '--cidfile', tmp, docker_image, 'bash'])
+ if ret_code != 0:
+ logging.error("Couldn't start the container from '%s'" % docker_image)
+ panic()
+
+ with open(tmp, "r") as fd:
+ self.docker_container = fd.read().strip()
+
ret_code, output = run(['docker', 'inspect', self.docker_container])
container_data = json.loads(output)[0]
self.name = container_data['GraphDriver']['Data']['MergedDir']
--
2.30.2
-------------- next part --------------
>From 672e844798017ac35a9ac62a226f0cd8bfd932fa Mon Sep 17 00:00:00 2001
Message-Id: <672e844798017ac35a9ac62a226f0cd8bfd932fa.1634559960.git.hahn at univention.de>
In-Reply-To: <74fc04769f062cc9818512d5e33755a8c42e2616.1634559960.git.hahn at univention.de>
References: <74fc04769f062cc9818512d5e33755a8c42e2616.1634559960.git.hahn at univention.de>
From: Philipp Hahn <hahn at univention.de>
Date: Mon, 18 Oct 2021 14:18:25 +0200
Subject: [PATCH 2/3] refactor[docker] Let docker inspect return path
Directly use golang templating to only get the MergedDir path instead of
using json in Python.
Add path to deebug output.
Let debug output do the variable substitution.
---
piuparts.py | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/piuparts.py b/piuparts.py
index be99abf7..e9c9934d 100644
--- a/piuparts.py
+++ b/piuparts.py
@@ -46,7 +46,6 @@ import os
import tarfile
import stat
import re
-import json
import pickle
import subprocess
import traceback
@@ -932,10 +931,9 @@ class Chroot:
with open(tmp, "r") as fd:
self.docker_container = fd.read().strip()
- ret_code, output = run(['docker', 'inspect', self.docker_container])
- container_data = json.loads(output)[0]
- self.name = container_data['GraphDriver']['Data']['MergedDir']
- logging.info("New container created '%s'" % self.docker_container)
+ ret_code, output = run(['docker', 'inspect', '-f', '{{ .GraphDriver.Data.MergedDir }}', self.docker_container])
+ self.name = output.strip()
+ logging.info("New container created %r at %r", self.docker_container, self.name)
def setup_from_lvm(self, lvm_volume):
"""Create a chroot by creating an LVM snapshot."""
--
2.30.2
-------------- next part --------------
>From e4f94200cad5e01c68edee648e57b58b5ee3008b Mon Sep 17 00:00:00 2001
Message-Id: <e4f94200cad5e01c68edee648e57b58b5ee3008b.1634559960.git.hahn at univention.de>
In-Reply-To: <74fc04769f062cc9818512d5e33755a8c42e2616.1634559960.git.hahn at univention.de>
References: <74fc04769f062cc9818512d5e33755a8c42e2616.1634559960.git.hahn at univention.de>
From: Philipp Hahn <hahn at univention.de>
Date: Mon, 18 Oct 2021 13:04:13 +0200
Subject: [PATCH 3/3] feat[logging] Always DUMP logging to file
if things fails it is very useful for the logfile contain all debug
information.
---
piuparts.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/piuparts.py b/piuparts.py
index e9c9934d..b1db0492 100644
--- a/piuparts.py
+++ b/piuparts.py
@@ -466,11 +466,12 @@ def setup_logging(log_level, log_file_name):
logging.addLevelName(DUMP, "DUMP")
logger = logging.getLogger()
- logger.setLevel(log_level)
+ logger.setLevel(DUMP)
formatter = TimeOffsetFormatter("%(asctime)s %(levelname)s: %(message)s")
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(formatter)
+ handler.setLevel(log_level)
logger.addHandler(handler)
HANDLERS.append(handler)
--
2.30.2
More information about the Piuparts-devel
mailing list