[Openstack-devel] Rewriting the .ini parsing bit of openstack-pkg-tools

[Thomas Goirand]

On 03/19/2013 05:07 PM, James Page wrote:
> Hi Thomas
> 
> On 11/03/13 07:55, Thomas Goirand wrote:
>> When Roland was doing some packaging work, we had discussions on
>> how to do the parsing of .ini files inside Openstack. I first wrote
>> a very ugly shell script function, which was about 150 lines, which
>> is completely lame, for such a small task. Then he decided
>> (rightly) that it was crap, and tried rewriting it in python. The
>> current code in openstack-pkg-tools is from him, and is like this:
> [...]
> 
> I took a look through the use of debconf in glance just to get a feel
> of how it currently works for the openstack packages in Debian (for
> reference we don't use debconf at all in the Ubuntu packages).
> 
> I'm uncomfortable about how the maintainer scripts directly manipulate
> the configuration files installed by the packages, for example:
> 
>          sed -i "s,^flavor\s*=.*$,flavor = $FLAVOR,"
> /etc/glance/glance-registry.conf
>             FLAVOR=keystone
>             db_get glance/auth-url
>             sed -i "s,^auth_uri\s*=\s*.\+$,auth_uri = $RET,"
> /etc/glance/glance-registry-paste.ini
>             db_get glance/auth-token
>             sed -i "s/^admin_token\s*=\s*.\+$/admin_token = $RET/"
> /etc/glance/glance-registry-paste.ini

Hi James,

These are coming from Glance Essex in SID/Wheezy. These maintainer
scripts where not written by me but by Ghe Rivero and Julien Danjou (as
per git blame results).

What is currently in Folsom (in Debian experimental) is done a very
different way, and this time, in a policy compliant way.

> If the end user has elected to use debconf to configure packages, then
> this is OK; however debconf is just one way to manage configuration -
> puppet, chef, juju or whatever configuration management tool of choice
> might be in use will also manipulate these files.
> 
> With the direct manipulation that the package does, there is potential
> for the packaging to stamp all over local changes to configuration
> files; the debconf managed files should be installed and managed using
> ucf so user changes are always preserved (at least that is the way
> I've always used dbconfig-common and debconf).

Yes, I do agree with the above. What is in Wheezy is not Debian policy
compliant. But I have decided to *not* work on these, and go forward
working on Folsom and Grizzly. Please disregard the old implementation
that has multiple flaws:
- The config script do not read what is in the config file, and use
db_set accordingly before doing any db_input.
- The maintainer scripts are modifying "conffiles" files, which is
explicitly forbidden by the Debian policy.

The reason I haven't worked on these is simply because Wheezy is frozen,
and now isn't the time to change anything. I believe I repaired "a bit"
the logic for Keystone, but it was so painful with the release team,
that I gave up doing some more for the stable distro (since, like for
every human being, my days are only made of 24 hours).

> How about you take a completely different approach to this issue;
> rather than parsing/modifying the upstream provided configuration
> files, make templates as part of the packaging with suitable
> replacement tokens (easy to sed), populate using debconf and install
> using ucf.  The downside of this of course is that you have to manage
> your own versions of configuration files; but this might be a suitable
> compromise.
> 
> Anyway - that's probably how I would approach this issue.

Could you please have a look in the Alioth repository to see what I've
been working on since Wheezy is frozen? I think you will see huge
differences.

I do not use ucf, I simply store the default configuration file in:
/usr/share/<package-name>
then I just copy it (with BSD install) on /etc if it isn't present. It's
been on my todo list for many years to learn how to use ucf. This is
done in pkgos_write_new_conf() in openstack-pkg-tools, and would be easy
to replace everywhere (since every package use that function). Probably
using ucf would be better. Could you give some examples, or even better,
a patch for openstack-pkg-tools.git/pkgos_func, so that the
configuration files are maintained with ucf instead of BSD install? That
would be very nice of you.

By the way, please do have a look at openstack-pkg-tools in the
debian/grizzly branch, I am done with my clean-up (I've checked with my
preseeder script, and everything seems to install smoothly), and it is
now version 4 which is in use for all of Grizzly. Version 3 is for
Folsom, and will probably be gone "soon" (eg: after we all start using
Grizzly well after the release).

Cheers,

Thomas Goirand (zigo)