[Pkg-xen-devel] Bug#452721: irt: irt: Bug#452721 notes from explorations

[Elliott Mitchell]

Synthesizing things since I hadn't been copied on previous message...

On Mon Jul 31 18:10:34 BST 2023, zithro wrote:
> 
> On 31 Jul 2023 03:39, Elliott Mitchell wrote:
> 
> > Presently I hope to convince the Xen core to allow full Python in domain
> > configuration files, but no news on that front so far.  This would mean
> > /etc/default/xendomains would need to change to match Python syntax.
> 
> There was an answer today on xen-devel: the ability to use scripts in
> domU cfg files has been explicitely removed for various reasons.
> This does not prevent you from "source"-ing teh cfg files in your
> script(s) if they are proper Python syntax. Or you could simply
> parse/regex the values you want.

Though the reasons given seem orthogonal to my thinking.  I'm thinking
use libpython as the parser since that allows dictionaries and guarantees
the syntax remains a subset of Python.  Whereas the responses read like
they think I'm asking for full Python scripts as domain configurations
(which is a very large superset of what I'm proposing).

> And as Marek suggested in his answer, you can also put any arbitrary
> settings in the comments.

I had already thought of that as it is a common strategy for such things.
This though has substantial limitations and since Python has all the
capabilities needed, strategies based on Python seem very attractive.

I was thinking Perl for a bit, but Python provides a simple strategy for
extracting required information out of configurations.  Crucially the
UUID which lets you match running domains to their configuration.


> Although ...
> 
> > My thinking for adding to domain configuration files would be something
> > along these lines:
> >
> > init = {
> >       'tool': 'xendomains-ng',
> >       'version': 0,
> >       'order': 9,
> >       'startwait': 60,
> >       'stopaction': 'save',
> > }
> 
> The problem with adding this to a domU config file is that it could
> cause problems for (live) migrations. The start/stop order is "per
> dom0", and may be different on another one.
> Imagine two dom0s, one storing the domain files "locally", while the
> other uses NFS. Only in the second case the domU should wait for the NFS
> server/domain to be available.
> 
> To me, the start/stop logic should be in a dom0 config file.

I'm not understanding the situation you're thinking of.

The closest I can come is you're thinking of a situation which would be
handled by having host defaults, but also overrides in domain.cfg files.
Generic VMs would act according to the host settings, only domains which
had overridden values would act differently.

You could have a network of VM hosts where normal hosts specify 'migrate'
in /etc/default/xendomains.  Then you have the magic host which specifies
'save' or 'shutdown'.  You would also specify something other than
'migrate' for domains handling services local to a particular host.

> > 'startwait' would tell the script to wait that long before starting
> > subsquent domains.
> 
> A time-based wait may be useful for when everything goes well, but what
> about when there are problems ?
> If you want to be sure a domain is up (ie. ready to serve), you would
> need to peek at the related "service".
> For example, to be sure a DNS domU is up, you would have to try a DNS
> request, as a ping or "xl list" would not be enough.
> Also, domains in xen/auto are started with a mix of serialization AND
> parallelization, as "xl create" returns once the domain has started (ie.
> in the Xen point of view, not the user's).

Indeed.  I'm well aware what I'm suggesting has major limitations.  I'm
proposing what I consider feasible given available time.  What you're
suggesting could be a feature for v2, which might be written based on
what I manage.


> > 'stopaction' would allow different actions if the machine was to stop.
> > The 3 options which come to mind are 'stop' (shutdown), 'save' (save to
> > specified storage location), and 'migrate'.
> 
> Then, each time you do NOT want to follow the usual action, you'd have
> to edit -each- domU cfg file ?

Usually if you didn't want to follow the usual action, you would invoke
`xl` manually.

What has come to mind though is perhaps the action should be uploaded to
the xenstore.  Then when an unusual action was desired, the xenstore
information could be changed and the action would follow the domain.
This though seems a feature for a future version.


> > If full Python doesn't become available, this might take the format:
> > init = 'tool=xendomains-ng,version=0,order=9,startwait=60,stopaction=save'
> > Not needing to parse the string though does make one's life simpler.
> 
> Well, it makes -your- life easier, not the maintainers' one ;)

Given what the parser looks like, it shouldn't take much to make things
easier.  I suspect getting rid of the Bison/Flex parser and using
libpython will be easier.  Though the maintainers may disagree.

Dunno until things reach full implementation.  What I've sent so far is
merely to identify the border between the lower-level parser and
upper-level stuff.

I suspect for a while it might be possible to switch between the two.
Until it turned out one or the other had very little uptake.

> > I'm basically certain writing a new xendomains script in Python is the
> > way to go.  Now to get an answer as to whether full Python in domain
> > configuration files could be reenabled.
> 
> I'm not sure a Python script would solve anything, as (ba)sh variables
> are imported from other files.
> (see for example
> https://salsa.debian.org/xen-team/debian-xen/-/blob/master/tools/hotplug/Linux/xendomains.in)

I'm well aware of this.  I would turn those into Python-fragments (sort
of how domain.cfg files conform to Python-syntax).  This would need some
shell scripting during installation to upgrade old files, but this seems
feasible.

Roughly:
xen-utils-common.postinst:
if grep -q -eXENDOMAINS /etc/default/xendomains; then
	. /etc/default/xendomains
	export XENDOMAINS_AUTO XENDOMAINS_MIGRATE XENDOMAINS_RESTORE \
		XENDOMAINS_SAVE XENDOMAINS_STOP_MAXWAIT
	/var/lib/dpkg/info/xen-utils-common.newdomainswriter.py
fi

Where `xen-utils-common.newdomainswriter.py` is used to ensure quoting
matches Python's requirements.  There would also be a need to ask
permission to upgrade the configuration file to something which older
tools won't accept.  Worst case a replacement filename could be used.

> Everything considered, I'm not sure why Xen should provide such
> functionnality.
> I think custom scripts can handle all the various use cases, don't you
> think ?
> PS: as mentionned by diederik, the "dependency" logic is already handled
> by Qubes since years, and it never made it to Xen (I don't know the
> reasons though).

Indeed.  This is why my only desire for the Xen project is to allow full
Python syntax, by using libpython instead of the Bison/Flex parser.
Having generic scripts in the Xen Project might be nice though...

> But I agree the shutdown sequence could be adapted to :
> 1. first shutdown the domains NOT in xen/auto
> 2. then shutdown the domains in xen/auto, in reverse order

I was thinking of slightly different logic.  Domains without a specified
order are order 0. Domains with higher order start first and are shutdown
last.  This has two implications:

If a domain doesn't have a configuration file in /etc/xen/auto then it
has order 0 and will be stopped at the same time as other order 0
domains.

There is no requirement for the order to be positive.  I'm not sure what
sorts of things would start after defaults, but I'm sure someone will
come up with a use case.

I wouldn't bother explicitly making unlisted domains stop earlier.
Simply stop them with the rest of the commoners.  Anything which needs to
stop after unlisted domains should have a positive order.

> For fine grained start/stop order, maybe having a dom0 config file
> handling this could be added, like:
> 
>      # START/STOP ORDER
>      # domains not in these lists will be started after and stopped
>      # before the ones here
>      start-order=(list of domU names)
>      stop-order=(list of domU names)

That should be feasible.  Are you volunteering to write such a thing?

Several issues though:

Can this express domains which could nominally be started in parallel?
(domains in my proposal can have the same order)

How would this express domains which should have different shutdown
actions?

Then of course your last question equally applies to your proposal:

> But then again, this only ensures "domains" start order, not "services
> availability" in said domains.

Indeed.  I'm suggesting something which seems feasible, not something
which is ideal.  My hope is having "startwait" to specify a delay would
be Good Enough(tm) for a first version.

How would your proposal handle this?


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445