[Qa-jenkins-scm] [Git][qa/jenkins.debian.net][master] 2 commits: reproducible Archlinux: use unregister_build() were appropriate
Holger Levsen
gitlab at salsa.debian.org
Thu Sep 27 01:31:54 BST 2018
Holger Levsen pushed to branch master at Debian QA / jenkins.debian.net
Commits:
c138b325 by Holger Levsen at 2018-09-27T00:09:30Z
reproducible Archlinux: use unregister_build() were appropriate
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
9f4a0ec0 by Holger Levsen at 2018-09-27T00:31:40Z
reproducible Archlinux/Debian/rpms: refactoring
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
5 changed files:
- TODO
- bin/reproducible_build.sh
- bin/reproducible_build_archlinux_pkg.sh
- bin/reproducible_build_rpm.sh
- bin/reproducible_common.sh
Changes:
=====================================
TODO
=====================================
@@ -318,35 +318,32 @@ See link:https://jenkins.debian.net/userContent/about.html["about jenkins.debian
** check for archlinux schroot sessions which should not be there and delete them. complain if that fails.
* use db - see https://tests.reproducible-builds.org/reproducibledb.html
-** extend scheduler.sh:
+** scheduler.sh:
*** comparing versions is probably needed: if its not empty it must be higher (due to repo constraints), but it can be even higher than in the repo, because we build trunk
*** dont delete packages which are building?
-*** check/make sure that packages which are newer in trunk than repo are only scheduled once
-** make _archlinux_build.sh
-*** refactor and reuse unregister_build from _build.sh
-*** we share /var/log/jenkins/reproducible-race-conditions.log with debian, is that ok? (else fix maintenance job and handle_race_condition())
-*** make build respect pacman exit code, see FIXME in _html_.sh
-** html
+*** check/make sure that packages which are newer in trunk than repo are only scheduled once (they are not scheduled when trunk is updated at all... we need to use asp to find that out)
+** html:
*** leave all pkg.html files, delete them (much) later, then check all pkg.* files are gone
*** recreate them newly as index.html
-**** create new job to recreate all pkg pages
-** create json
-** debian uses a table removed_packages, should this be used here too?
-** debian uses a table manual_scheduler to limit the amount of packages someone can schedule per day...
-** should breakages job look for archlinux breakages?
+*** create new job to recreate all pkg pages
+*** create json
+** misc:
+*** we share /var/log/jenkins/reproducible-race-conditions.log and reproducible-remote-error.log with debian, is that ok? (else fix maintenance job and handle_race_condition() and handle_remote_error())
+*** make build.sh respect pacman exit code, see FIXME in _html_.sh
+*** debian uses a table removed_packages, should this be used here too?
+*** debian uses a table manual_scheduler to limit the amount of packages someone can schedule per day...
+*** should breakages job look for archlinux breakages?
+** using notes.git could be next
* rebuilding against the archlinux repos
-** only needs to be done once
+** only needs to be done once using https://github.com/Foxboron/devtools-repro
** to not (pacman know about trunk and thus) build trunk, we need to configure asp.
-** wel'll keep building against repo+trunk as we do now (so that archlinux can also benefit from the QA effects)
+** we'll keep building against repo+trunk as we do now (so that archlinux can also benefit from the QA effects)
* fix build.sh:
** build2.log doesnt get deleted if build1 fails
** -> rename build2.log to $version_build2.log (dont include package name...)
-* test against real archlinux pkgs:
-** https://github.com/Foxboron/devtools-repro
-
* things to be done before enabling more builders:
** build in /srv/workspace instead of /tmp (once this has been done reduce /tmp size back to 15G)
=====================================
bin/reproducible_build.sh
=====================================
@@ -317,15 +317,6 @@ handle_env_changes() {
exit 0
}
-handle_remote_error() {
- unregister_build
- MESSAGE="${BUILD_URL}console.log got remote error $1"
- echo "$(date -u ) - $MESSAGE" | tee -a /var/log/jenkins/reproducible-remote-error.log
- echo "Sleeping 5m before aborting the job."
- sleep 5m
- exit 0
-}
-
handle_enospace() {
unregister_build
MESSAGE="${BUILD_URL}console.log hit diskspace issues with $SRCPACKAGE on $SUITE/$ARCH on $1, sleeping 60m."
@@ -744,6 +735,7 @@ remote_build() {
rsync -e "ssh -o 'BatchMode = yes' -p $PORT" -r $NODE:$TMPDIR/b$BUILDNR $TMPDIR/
local RSYNC_RESULT=$?
if [ $RSYNC_RESULT -ne 0 ] ; then
+ unregister_build
handle_remote_error "when rsyncing remote build #$BUILDNR results from $NODE"
fi
fi
@@ -767,6 +759,7 @@ remote_build() {
0) # build succcessfully completed
;;
*)
+ unregister_build
handle_remote_error "with exit code $RESULT from $NODE for build #$BUILDNR for ${SRCPACKAGE} on ${SUITE}/${ARCH}"
;;
esac
=====================================
bin/reproducible_build_archlinux_pkg.sh
=====================================
@@ -30,16 +30,6 @@ cleanup_all() {
fi
}
-handle_remote_error() {
- MESSAGE="${BUILD_URL}console got remote error $1"
- echo "$(date -u ) - $MESSAGE" | tee -a /var/log/jenkins/reproducible-remote-error.log
- echo "Sleeping 5m before aborting the job."
- sleep 5m
- cleanup_all
- exec /srv/jenkins/bin/abort.sh
- exit 0
-}
-
update_pkg_in_db() {
local ARCHLINUX_PKG_PATH=$ARCHBASE/$REPOSITORY/$SRCPACKAGE
cd "$ARCHLINUX_PKG_PATH"
@@ -456,23 +446,22 @@ remote_build() {
SLEEPTIME=$(echo "$BUILDNR*$BUILDNR*5"|bc)
echo "$(date -u) - $NODE seems to be down, sleeping ${SLEEPTIME}min before aborting this job."
sleep ${SLEEPTIME}m
- cleanup_all
- exec /srv/jenkins/bin/abort.sh
+ unregister_build
+ exit 0
fi
ssh -o "Batchmode = yes" -p $PORT $FQDN /srv/jenkins/bin/reproducible_build_archlinux_pkg.sh $BUILDNR $REPOSITORY ${SRCPACKAGE} ${TMPDIR} ${SOURCE_DATE_EPOCH}
RESULT=$?
if [ $RESULT -ne 0 ] ; then
ssh -o "Batchmode = yes" -p $PORT $FQDN "rm -r $TMPDIR" || true
if [ $RESULT -eq 23 ] ; then
- echo "$(date -u) - remote job could not end schroot session properly and sent error 23 so we could abort silently."
- cleanup_all
- exec /srv/jenkins/bin/abort.sh
+ unregister_build
+ handle_remote_error "job on $NODE could not end schroot session properly and sent error 23 so we could abort silently."
elif [ $RESULT -eq 42 ] ; then
- echo "$($date -u) - sigh, failure after not being able to verify pgp signatures. work to debug why ahead."
- cleanup_all
- exec /srv/jenkins/bin/abort.sh
+ unregister_build
+ handle_remote_error "job on $NODE after not being able to verify pgp signatures. work to debug why ahead..."
else
- echo "Warning: remote build failed with exit code $RESULT from $NODE for build #$BUILDNR for ${SRCPACKAGE} from $REPOSITORY."
+ unregister_build
+ handle_remote_error "Warning: remote build failed with exit code $RESULT from $NODE for build #$BUILDNR for ${SRCPACKAGE} from $REPOSITORY."
fi
fi
rsync -e "ssh -o 'Batchmode = yes' -p $PORT" -r $FQDN:$TMPDIR/b$BUILDNR $TMPDIR/
@@ -483,8 +472,8 @@ remote_build() {
rsync -e "ssh -o 'Batchmode = yes' -p $PORT" -r $FQDN:$TMPDIR/b$BUILDNR $TMPDIR/
RESULT=$?
if [ $RESULT -ne 0 ] ; then
- #handle_remote_error "when rsyncing remote build #$BUILDNR results from $NODE"
- echo "Warning: error rsyncing remote build #$BUILDNR results from $NODE."
+ unregister_build
+ handle_remote_error "when rsyncing remote build #$BUILDNR results from $NODE"
fi
fi
ls -lR $TMPDIR
=====================================
bin/reproducible_build_rpm.sh
=====================================
@@ -26,15 +26,6 @@ cleanup_all() {
rm -f $DUMMY > /dev/null || true
}
-handle_remote_error() {
- MESSAGE="${BUILD_URL}console got remote error $1"
- echo "$(date -u ) - $MESSAGE" | tee -a /var/log/jenkins/reproducible-remote-error.log
- echo "Sleeping 5m before aborting the job."
- sleep 5m
- exec /srv/jenkins/bin/abort.sh
- exit 0
-}
-
update_mock() {
echo "$(date -u ) - checking whether to update mock and yum for $RELEASE ($ARCH) on $HOSTNAME."
local STAMP="${RPM_STAMPS}-$RELEASE-$ARCH"
=====================================
bin/reproducible_common.sh
=====================================
@@ -709,6 +709,14 @@ unregister_build() {
NOTIFY=""
}
+handle_remote_error() {
+ MESSAGE="${BUILD_URL}console.log got remote error $1"
+ echo "$(date -u ) - $MESSAGE" | tee -a /var/log/jenkins/reproducible-remote-error.log
+ echo "Sleeping 5m before aborting the job."
+ sleep 5m
+ exit 0
+}
+
#
# create the png (and query the db to populate a csv file...)
#
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/compare/72a6c362fce4229e9e3a53709255583d5526d4b8...9f4a0ec08600a0c273baa88ca768ab78514444ab
--
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/compare/72a6c362fce4229e9e3a53709255583d5526d4b8...9f4a0ec08600a0c273baa88ca768ab78514444ab
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20180927/c422f417/attachment-0001.html>
More information about the Qa-jenkins-scm
mailing list