[Debian-salsa-ci] Question about artifacts and ccache

[Andrea Pappacoda]

Hi Otto thanks for your reply!

On Wed Sep 18, 2024 at 2:33 AM CEST, Otto Kekäläinen wrote:
> The artifacts are basically the build results: source code, binary 
> packages and build logs.
> See example 
> https://salsa.debian.org/mariadb-team/mariadb-10.5/-/jobs/6295411/artifacts/browse/debian/output/
>
> The artifacts will be used by whatever later job defines it 'needs:' 
> the build and the artifacts.
>
> I don't see any problems with this as only jobs that need them depend 
> on them, but I would gladly review an improvement suggestion.

As I said, this is mainly a matter of clearly specifying the inputs and 
outputs of each step. Zipping up everything in the current working 
directory as an artifact isn't exactly clean in my opinion (assuming 
that's what's happening)


> The ccache speeds up C/C++ heavy projects remarkably (like 10x), we 
> should definitely keep it.

I'm aware of what ccache does (I'm mainly a C/C++ developer/maintainer), 
but could you please point me at actual cases where it gets used in the 
Salsa pipeline? In my head, I see containers as ephemeral; where does 
ccache's cache get cached?

> Yes, the images the pipeline uses are already updated once every 24h 
> so having 'apt update' run on them all them is probably unnecessary.  
> It also makes the point of container versions moot, as running any old 
> version of the container will always self-update anyway. For example 
> if there is a regression, and I want to test running the pipeline with 
> an old image from say 1 week ago to see if the regression existed in 
> an old version, I cannot, because even the old image would just update 
> itself and the regression (if introduced by a new package version) 
> would be visible in all runs.

Completely on point.

> The background to all your questions I think is the current slight 
> mess we have in the code base. Salsa CI has been organically expanding 
> for many years now, and it is hard for new contributors to reason 
> about what everything does.

Yeah, I've observed this :)

> What could help here if you would team up with Ahmed to work on 
> https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/528.  
> Splitting out the shell scripts from the yaml files into separate 
> scripts that can - if needed - even be run locally to reproduce 
> individual steps/jobs. Such a refactoring would make the pipeline 
> orchestration code in the yaml files smaller and easier to understand, 
> and the separate scripts would be easier to test for input/output in 
> various situations.

I thought I'd finish my refactor in a short amount of time, but it might 
make sense to clean things up before making improvements. I'll check out 
Ahmed's branch locally and work a bit on it.

Bye :)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 256 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/debian-salsa-ci/attachments/20240918/16bbdfe1/attachment.sig>