[Openstack-devel] Splitting the nova package (and possibly others)?

[Thomas Goirand]

On 11/20/2012 05:26 PM, Roland Mas wrote:
> Thomas Goirand, 2012-11-20 10:06:06 +0800 :
> 
>> Hi Roland,
>>
>> I believe the main issue we are having here is that pgsql doesn't
>> support remote configuration of users, and that dbconfig-common doesn't
>> take this into account. But that's it. Please read further.
> 
> No.  That was the previous main issue.  The main issue now is that you
> reverted something that was required and worked, without testing, based
> on flawed assumptions, and for the wrong reasons.
> 
>> On 11/20/2012 12:24 AM, Roland Mas wrote:
>>>   For the rest, I did some fact-checking, and it appears that:
>>> - dbconfig-common is indeed able to create remote databases; however,
>>>   that is hidden behind a low-priority Debconf question, so most users
>>>   won't even see that;
>>
>> Yeah! Which is why we explicitly wrote in all of our templates: "You can
>> change this setting later on by running "dpkg-reconfigure -plow
>> <pkg-name>".
> 
> Wrong.  The low-priority Debconf question is in dbconfig-common, so
> unless you dpkg-reconfigure -plow *dbconfig-common*, you'll never see
> the possibility to use remote databases.

Quite not. If I do dpkg-reconfigure -plow dbconfig-common, I have the
following questions:

Keep "administrative" database passwords? no
Will this server be used to access remote databases? no

Note that if I answer no or yes to this last question "access remote
database" debconf screen, it doesn't change at all what's below, so I
don't really get why it's there. Anyway, our users wont ever need to do
that, since dbconfig-common asks for it anyway.

After, with debconf set with priority=high (so, not even medium or low),
I can do the following on my server:

dpkg-reconfigure glance-common (note that I don't even add -plow)

The here's the set of questions, and things I answer:

Pipeline flavor: keystone
Auth server hostname: 127.0.0.1
Auth server tenant name: admin
Auth server username: admin
Auth server password: XXXX
Set up a database for glance? yes

Then automatically, I get the dbconfig-common questions as follow:

Reinstall database for glance-common? yes
Database type to be used by glance-common: mysql
Connection method for MySQL database of glance-common: tcp/ip
*Host name of the  database server for glance-common*: 127.0.0.1
Port number for the  service: <I-leave-it-empty>
Name of the database's administrative user: root
Password of the database's administrative user: XXXX
username for glance-common: glance-common
database name for glance-common: glancedb

Then I have the following output on my terminal:

dbconfig-common: writing config to
/etc/dbconfig-common/glance-common.conf


granting access to database glancedb for
glance-common at dom0.node4407.gplhost.com: already exists.
creating database glancedb: already exists.
dbconfig-common: flushing administrative password
Now doing glance-manage db_sync: this may take a while...

Now, please tell me, what's the problem??? What is wrong, according to
you, and deserves a change of behavior? Also, how come you are saying
above something that doesn't match what I'm seeing on my screen?

Now, after this, I hope you wont call me a liar again, and say that I'm
writing cracks...

>> The important bit is that it wouldn't *fail* in the case of MySQL.
>> Attempting to create a db which already exists isn't a problem, but it
>> doesn't even do that. dbconfig-common sees the db already exists and
>> doesn't creates it.
> 
>   Yes, and in the case of PostgreSQL, it requires the admin credentials,
> to check that, so it's broken.

After a phone conversation with our friends at eNovance, I have the
confirmation that we wont really care about PostgreSQL. What we (and
here, I believe it should be you) should do is just send a bug report
against dbconfig-common to report the remote pgsql problem. Please do
file such bug in the Debian BTS, and move on...

Also, do file a bug against dbconfig-common with the wishlist item that
you would like to configure dbs without giving root access. It's not a
problem for me, but I can understand that it might be one for some
bigger organization, as you said.

>> In this case, feel free to contribute patches against dbconfig-common
>> itself. But IMO, this is not a sufficient reason to reimplement it in
>> a different way, with new debconf templates asking the same thing
>> dbconfig-common does.
> 
> Which is why I agree the patch may be improved.  However, just
> reverting it is *wrong*.

Just saying "reverting it is *wrong*" doesn't make a valid technical
point at all.

>> I never tried PostgreSQL. Almost everyone uses MySQL with Openstack,
>> anyway, so don't bother with PostgreSQL too much.
> 
>   You don't bother testing with two computers, you don't bother testing
> anything else beyond MySQL, and yet you revert others' work.  Please
> stop doing that.

Well, I would *like* to use more than one server, but I don't have it
available. If you would like to donate one to me, be my guess. In the
mean while, I'm working this way.

If I reverted, that was for the reasons I have already explained, and on
which I haven't seen much counter arguments.

Please stop such sentences like "you don't bother testing", and instead,
reply to my arguments, which I'll have to state once more:

- why do you insist in reimplementing things in our packages rather than
fixing dbconfig-common if you feel it's broken?

I really *needs* an answer from you. If you don't reply to that one,
then I'm sorry, I'll have to end replying on this topic, because it's
going nowhere.

>>>   Not really.  What exists in dbconfig-common is UI to create a
>>> remote database, not a UI to use an existing remote one.  It'll
>>> always try to create it with the database admin account, which means
>>> it needs the admin credentials.
>>
>> But that's not a problem for us (apart from the fact that you need the
>> admin credentials). Attempting to create the db, even if it exists
>> already, will not destroy it.
> 
>   It is a problem for us, because you need the admin credentials.
> Please *read*.

I did read. Also, once more, you are here complaining about a problem in
dbconfig-common. Please explain to me why you wouldn't patch it, if that
is where the problem is.

>> Well, your script is just calling db_input without even checking what's
>> stored in the configuration file. If there was something there, with the
>> info about how to connect to a remote PostgreSQL db, it would fail to do
>> what the user expect.
> 
>   Then just tell me that, and I'll fix it.  But don't revert.

Don't re-revert what I have reverted. Instead, answer the above question
about fixing dbconfig-common *FIRST*.

>>> Except we now also have the ability to use a remote database, either
>>> with interactive prompts or with preseeding, which we didn't have
>>> before.
>>
>> I don't agree. We did have such option. 
> 
>   Until you provide working Debconf and dbconfig-common preseed files,
> this is not true.

I must have dreamed when I was reading my screen then...

>> Your code:
>> - needs new debconf templates which are otherwise also available in
>> dbconfig-common, and for which the l12n team will complain very loudly
>> when they will translate (believe me, they will! if you don't trust this
>> fact, ask them now, before attempting any reimplementation...).
>> - removes the ability to be able to just reads everything from the
>> config file (your code doesn't read it, but of course, this could be
>> implemented...).
> 
>   So it could be improved.  Fair enough.

You could fix point 2 above, but not point 1. And by fixing point 2, you
are reimplementing everything twice, duplicating strings and adding more
strings to translate for the Debian l12n team, when all you have to do
is fixing the problem in dbconfig-common.

>> - is attempting to re-invent something we have already.
> 
>   No, it implements something that was not previously available.

But in the wrong package, and twice... One implementation for
dbconfig-common (in the config and poistinst script) and once for your
own debconf thingy (config and postinst script as well).

>> Now and again, if you think the current dbconfig-common is a problem
>> (which it is, apparently, with PGSQL), then fix it in the
>> dbconfig-common package.
> 
>   I agree this would be the best place for it to be fixed.  In the
> meantime, a workaround is required.

*NO*

We do not need duplication of this functionality.

>>>   I'm not saying the changeset is perfect, and maybe it can be made
>>> better.  But the current code is actually working and actually useful,
>>> so I don't understand your objections to it.
>>
>> My objection is that it did work before. 
> 
>   For you.  On your one-node, MySQL, setup.

Again, eNovance agree with me, problems with PGSQL in dbconfig-common
shouldn't be our focus (for the moment at least), and they agree that
almost everyone that uses Openstack is using MySQL. There's indeed quite
a consensus around this, even though it shouldn't really mater. That's
by the way one of the design choice you can see in the howto on our
wiki, so we aren't going away from our initial focus.

>   The first thing about that would be to actually make sure of your
> claims.  You say that there's UI in dbconfig-common to use a remote
> database (which is wrong)

You write "wrong, wrong, wrong".
You tell that I didn't check, whatever, whatever...

Yet, I saw these screens.

Also, here:

http://anonscm.debian.org/gitweb/?p=dbconfig-common/dbconfig-common.git;a=blob;f=debian/dbconfig-common.templates;h=29e9744d015021c6c88ef3a2d703e6ca7d3fd65b;hb=81fc50a096e8098e231130c7f34b43c1cb524998

search for the text: "Template: dbconfig-common/remote/host"

Do you need more proofs that it is there?

>> Going back to this dbconfig-common, we discussed these changes, I told
>> you I strongly did not agree with them, and even strongly opposed to it,
>> and you still went into that direction, broke 5 packages doing so (see
>> just above), then went away for 4 days, saying that you had something
>> else to deal with (which I respect, but you shouldn't break things and
>> run...). In this kind of situation, you can expect that I would revert a
>> patch (at least temporarily).
>>
>> Don't take me wrongly, I do believe there's a problem, and that it shall
>> be fixed. But not this way.
> 
>   Okay.  Fix it your way then.  But fix it.

I'm not using PGSQL, and I don't intend to. I have better things to do
than fixing dbconfig-common for this special case which isn't our focus.
For me, dbconfig-common is working perfect, and as expected. So I wont
fix anything.

If I was you, I would also be careful on what you choose to implement,
because we have no time to spare, and, again, PGSQL isn't our focus. But
if you want to fix dbconfig-common in your spare time, that'd be great,
a lot of DD and users will be happy about it. If eNovance agrees that
you work on it, that would be even nicer, IMO.

>> Yup. As written in our templates. For example (from
>> debian/nova-common.templates):
>>
>>  You can change this setting later on by running "dpkg-reconfigure
>>  -plow nova-common".
> 
>   It doesn't allow using a remote db without the database admin's
> credentials, as explained before.

But that's a problem in *dbconfig-common*, not in our packages. We
decided to use dbconfig-common (and it's not even me who did that
original choice), so we do use it. Now, if you think dbconfig-common is
flawed (which I don't), feel free to fix it.

>>> Until you do that, those of us who actually try to run the
>>> packages on several hosts are stuck.
>>
>> I don't agree with this statement. A dpkg-reconfigure -plow does work, I
>> did use it before, it does ask for the host, and configures the db
>> correctly. The problem we are having is that it doesn't work with
>> pgsql,
> 
> So it does not work. Stop claiming it does.

Right! Call me a liar...
Don't you think you are going a little bit too far here?

> If you do, back it with
> facts, and provide a set of preseeds that I can use in my
> non-interactive scripts.

I haven't talked about preseeding at all, did I?
It might well be possible that preseeding is broken in dbconfig-common.
But, if preseeding is broken, that's another problem entirely, we should
discuss is *separately*, and I would like to know about it.

But I assure you, *please believe me*, doing dpkg-reconfigure -plow
glance-common does work, and it does ask for the hostname (see what I
wrote above, doing cut/past of absolutely all debconf questions that I
saw on my screen).

>> because pgsql doesn't have the feature to be administered remotely. If
>> you wish to explicitly write a warning about this in our debconf
>> templates, and/or patch dbconfig-common so that it will not do what's
>> needed in this case, I support this idea!
> 
>   Honestly, I'd rather provide a working solution than documenting
> failures.

Sure! Fix dbconfig-common then. But *please* do not attempt to
reimplement it *AND* use it on the same package.

Now, I'm telling you this: this is the last time that I am telling you
it did work for me with MySQL, and that I saw these debconf screens.
Don't call me a liar once more, because it is really starting to add-up
(and everyone has limits...).

Cheers,

Thomas