[PKG-Openstack-devel] debian openstack and puppet

Tue Jun 24 08:59:22 UTC 2014

On 06/24/2014 03:46 PM, Gaudenz Steinlin wrote:
>>> I'm installing an openstack cluster, that is spanned over several nodes.
>>> All config files (and also package installation) is handled with puppet.
>>> The Debian packages do quite a lot of configuration work using
>>> preseeding etc. But actually these configuration do not work together
>>> well with the puppet manifests.
>>
>> This is not at all the experience that reported the
>> company that sponsors my packaging work.
> 
> Just out of curiosity, do they use puppet or another configuration
> management system and if they use puppet do they use the official
> openstack modules or something home grown?

They use puppet, yes. The "official" openstack modules are known to be
buggy and not feature complete. I wouldn't recommend them. I've just
asked where to download what we use.

>> and select the non-interactive mode, so that debconf
>> is disabled forever (note: this can also be
>> preseeded or edited somewhere in /etc) or use
>> DEBIAN_FRONTEND=non-interactive each time you do the
>> upgrades.
> 
> Reconfiguring debconf is not really an appropriate solution if you only
> want to "opt out" of the debconf machinery for the openstack packages. It
> sets the priority to low for ALL packages.
> 
> Using a lower priority does not opt out of debconf. It just disables the
> prompting and uses the default values and does not mark the questions as
> "seen" (thus the prompting on interactive upgrades).

Did I talk about the priority? No... I talked about the
debconf/frontend, which you shall set to "Noninteractive".

> Using the noninteractive frontend in an interactive session has many
> disadvantages like the fact that you don't see any notes and errors or
> that you don't get prompted for all the usefull questions like the one
> if like to restart daemons on certain library upgrades (where the answer
> might change from case to case depending on the circumstances (severity
> of the bug fixed, deamons affected, ...).

Well, either accept to have prompts, or not. You can't have both prompts
and no prompts, it's like that, unfortunately...

>> I very much doubt that. All config scripts are
>> reading the configuration files and do the db_set
>> calls accordingly, just like the Debian policy
>> dictates. If you find such behavior in one of the
>> packages, make sure you can reproduce it, and then
>> please report it as a bug!
>>
> 
> This sound like the correct approach.
> 
> Are you also respecting user modifications for debconf questions that
> are not marked as "seen" but the corresponding value in the
> configuration file has been changed by the user? Or is the user just
> presented with the default value if he first sees the question and his
> modifications are overwriten when he just accepts this value (knowing
> that he want's to manage the configuration with his config management
> system anyway)?

The packages are respecting the Debian policy. Meaning that it goes like
this:
- Read the configuration file (if it exists, otherwise, use the default)
- Do a db_set with the value set in the file
- Prompt the user (the value currently set in the configuration file
will appear, unless the prompt is marked as see)
- Set the value which was (or was not) modified in the debconf prompt

> I don't want to imply this is a bug. But as probably all
> the machinery to read value from configuration files is already in
> place, it would be nice to substitute the actual value into the debconf
> prompt if it was changed before the question is first asked.

Which is exactly what the Debian policy tells about, and what the
packages are doing.

>>> Even worse db synchronisation (something made by the postinst script) is
>>> not executed correctly, since the database connection string is at its
>>> default value at the time of execution....
>>
>> Same here. Directives are read from the config
>> files. So this cannot happen. Well, unless you are
>> producing obsolete directives which have moved to a
>> new section / have been renamed in Icehouse, which
>> is possible, but then packages can't be blamed:
>> update your puppet scripts...
> 
> I very much doubt that you can just blame the puppet scripts if the
> directives still work. IMO you should at least allow for a transition
> period during which old deprecated but still working directives are
> taken into account by package scripts. The Debian policy is quite clear
> about the fact that user configuration has to be preserved in any
> circumstances even if they use deprecated configuration syntax.

In theory, I agree. In practice, well... I welcome you to submit such a
complicated patch. Being alone doing the work do not permit me to
support deprecated configuration directives and automate the transition
of them.

> This section contains several errors. The default priority in Debian is
> not critical. It's the same you use during the installation with
> debian-installer and the default in the installer is "high".

Woops, indeed. I'll correct this. Not sure how it went in this way.

> This
> section also contains the same recomendation to reconfigure the debconf
> package to select non-interactive mode without telling the user that
> this is in no way related to the openstack packages but for the whole
> system.

I'm not sure how to phrase it then. Do you think you could produce a
patch for the doc? FYI, the git URL is at:
git://github.com/openstack/openstack-manual.git

> It also states that this opts out of using debconf. You can't
> opt out of that, you can only suppress the prompts with the
> non-interactive interface.

It certainly would need some improvements with things described in a
better way. I'd welcome patches, and will try to produce one myself if I
can find the time to do so.

> IMO the better way to do this is to use the DEBIAN_FRONTEND
> environment variable (at least as a recommendation in an OpenStack
> specific manual). Another possibility is to set
> DEBCONF_NONINTERACTIVE_SEEN while installing packages with a
> configuration management. This marks questions answered with the non
> interactive frontend as seen and avoids this question on upgrades. Thus
> only new questions are asked on (interactive) upgrades.

Good to know. I didn't know about DEBCONF_NONINTERACTIVE_SEEN. Do you
just do something like this?

DEBCONF_NONINTERACTIVE_SEEN=yes DEBIAN_FRONTEND=non-interactive apt ...

> If you can point me to the source code repository of the manual I can
> try to write a patch for the documentation.

Wonderful, thanks! :)

See above for the Git URL. FYI, you'll need to use "git-review" and
gerrit, and sign the contributor agreement. See this:
https://wiki.openstack.org/wiki/How_To_Contribute

Once you have a patch in review, please add me as a reviewer to your
patch. I can also do the work of having it merged, as I'm more familiar
with the process, if you prefer.

Cheers,

Thomas Goirand (zigo)