[Debian-salsa-ci] Optimization of SalsaCI with sbuild [Debugging]

[Enoch Kaxada]

In line with the email thread and email title, The purpose of this email is
to start a collaboration on how best we can optimize Salsa CI by migrating
from using *dpkg-buildpackage* to *sbuild* for building packages. The
reason is to reduce the build time it may take for testing packages. Other
reasons are referenced in this MR
<https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404>'s
description.

So far there has been a lot of progress as reflected in the diffs
<https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404/diffs>.
However. One major block that we are having is integrating sbuild with
ccache. The pipeline generally has 3 stages;

   1. Images - This creates reusable docker containers using these
   Dockerfiles
   <https://salsa.debian.org/kaxada/pipeline/-/tree/optimization-with-sbuild/images/dockerfiles>
that
   are stored in GitLab to be reused anytime a job is called. these images
   need to be built and availed so that the build time is reduced.
   2. Build - This fetches the necessary upstream resources for a
   particular package and uses builds the package depending on the *host,
   architecture and build environment variables*. Currently, this process
   is using dpkg-buildpackage and this is the section we want to integrate
   with sbuild
   3. Clean - This cleans up the used images for further use.

We have been thinking of integrating ccache with sbuild but we are having
issues with which stage we can implement that so that it runs successfully.
I created a long comment that tries to explain some of my analysis of the
whole pipeline
<https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404#note_393173>
and why it would be failing. please use it as a reference. The latest build
error that I'd also need some enlightenment on is on this line.
<https://salsa.debian.org/kaxada/pipeline/-/jobs/4117029#L283>

I would have been verbose in my here but I have tried to provide the
necessary links and a basic explanation of what the scenario is.

I hope to collect perspectives and also answer some questions where I can.

Regards,
Enock


On Fri, Apr 7, 2023 at 6:46 AM Johannes Schauer Marin Rodrigues <
josch at debian.org> wrote:

> Hi Enoch,
>
> Quoting Enoch Kaxada (2023-04-07 01:32:24)
> > I am writing this email to open up a collaboration so that we can work
> > together.
> >
> > We are working on integrating sbuild into salsa CI to reduce the build
> time
> > mostly. Here is the MR
> > <https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404>.
> > There are several threads of discussions under the MR to see that we set
> up
> > sbuild in the best way possible but we think we need extra eyes.
> >
> > Our main challenge is using ccache together with sbuild and the latest
> > implementation is on this HEAD
> > <
> https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404/diffs
> >
> > .
> >
> > We are looking so solid explanations as to why the pipelines errors are
> the
> > way they are.
>
> I got recently diagnosed with multiple sclerosis and am still unable to
> feel my
> feet and fingers. The latter makes using my computer very hard and
> frustrating.
> Please be more specific what problem you are having with sbuild so that I
> can
> minimize my time in front of the keyboard.
>
> Also note, that while I probably know subild best, I have no idea about
> ccache
> integration with it. So I might not be of much help.
>
> Thanks!
>
> cheers, josch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/debian-salsa-ci/attachments/20230407/4aba744e/attachment.htm>