[Pkg-raspi-maintainers] Proposal: pythonize recipe generation

[Cyril Brulebois]

Hi,

It seems we keep stumbling upon the same kind of issues over and over
again: we have 8 similar-yet-subtly-different blocks with lots of
sed/grep/etc. calls to generate raspi_${family}_${release}.yaml from a
raspi_master.yaml (using per-release temporary representation which
means a second pass), and we keep running into side effects from the
various piping, need for escaping, etc.

Upon the past few days, it occurred to me we could just declare a few
variables based on appropriate conditions on Pi family and Debian
release, and use a non-shell-based approach to string replacement.

I've implemented a little proof of concept in Python, which I've pushed
to the pythonize branch:
  https://salsa.debian.org/raspi-team/image-specs/-/tree/pythonize

Important commits are:
  https://salsa.debian.org/raspi-team/image-specs/-/commit/4483f4cdfbb411618340f643b2123b1ede8c12ca
  https://salsa.debian.org/raspi-team/image-specs/-/commit/0f8884c39cb83cf0e59957491ff5308cd0403a8c
  https://salsa.debian.org/raspi-team/image-specs/-/commit/252a6948489e8c15ef1cd9e714dc98dd427b16dc

The first one is the first draft of that recipe generation script,
that's getting slightly adjusted (alongside raspi_master.yaml) in the
next few commits. This was verified by comparing all (final) generated
YAML files (e.g. raspi_${family}_${release).yaml) to those generated
from the master branch, hence the long-ish commit message that
discusses the few changes.

The second one is the “target standardization” at the Makefile level:
the shell piping goes away, and we use two nested loops to call one
single tool that's going to do the work for us in a standardized
fashion. The third one makes it obvious we have two nested loops by
splitting them across several lines.


Questions, comments, and suggestions are welcome. As mentioned on IRC,
I think this should fix our immediate and recurring issues regarding
random breakages in this or that recipe because of a side effect of a
fixup for another family/release combination, even if it doesn't help us
start supporting more boards, see scalable branch by Andres:
  https://salsa.debian.org/dilinger/image-specs/-/commits/scalable/

As as side note, I implemented this in Python since it seems to be
quite ubiquitous and hopefully not too contentious, but (obviously)
I'm no Python superstar, so feel free to improve it! :)


Cheers,
-- 
Cyril Brulebois -- Debian Consultant @ DEBAMAX -- https://debamax.com/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-raspi-maintainers/attachments/20210828/cc7a5500/attachment.sig>