[Debian-salsa-ci] Question about artifacts and ccache

[Andrea Pappacoda]

Hi all!

I've been working on what I've described in 
<https://salsa.debian.org/salsa-ci-team/pipeline/-/issues/296#note_519376>, 
i.e., changing the pipeline to use a dsc instead of an extracted 
package.

This of course requires touching pretty much all of the pipeline files, 
and over time I've accumulated some questions which I'd like to ask to 
you, since you obviously know more about the pipeline than I do.

1. How are artifacts handled? From what I've gathered looking at the 
   code, it seems that everything contained in the "$WORKING_DIR" 
   directory gets zipped by Salsa as an artifact, which is then 
   available for all the other jobs. Is this right?

   If it is, I find this a bit suboptimal. The artifacts represent the 
   inputs and outputs of the various jobs, and if you think about jobs 
   as functions, you should be very careful about what you take as input 
   and what you return as output.

   I'd propose instead to split the artifacts directory and working 
   directory into two well separated variables, so that stuff gets 
   only included in artifacts purposely via explicit file moves/copies. 
   This would also allow reducing the risk of reaching the artifacts 
   size limit.

2. How's ccache setup? Looking at the code, I really cannot wrap my head 
   around it. ccache files are first deleted, then created, then ccache 
   is installed, then some other files are deleted, then ccache is 
   setup... It's a bit of a mess. I don't know much about ccache (I get 
   the concept, but never actually used it), but it definitely looks 
   hacky to me. Also, is it working? Does it save significant time? 
   Otherwise, I'd drop it in favour of simplicity. There are other parts 
   of the pipeline which can be sped up (like I recently did in 
   <https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/537>).

Another thing I find a bit suboptimal about the current pipeline setup 
is that builders repeatedly call apt update, apt upgrade, and apt 
install. I've observed that most of these calls are simply redundant, 
while others might be done in advance when building the images, so that 
they get ran only once.

Please let me know! I have a lot of messy ideas, and I really need some 
guidance :)

Bye!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 256 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/debian-salsa-ci/attachments/20240918/75930891/attachment.sig>