[PKG-Openstack-devel] openstack debian image improvements (was: Re: Bug#784334: Bug#784334: Bug#784334: Inaccessible emergency shell on serial console)

[Gaudenz Steinlin]

Thomas Goirand <zigo at debian.org> writes:

> On 05/19/2015 08:49 AM, Gaudenz Steinlin wrote:
>>
>> Hi Thomas
>>
>> Taking this off the bug report as it's no longer relevant there.
>>
>> Thomas Goirand <zigo at debian.org> writes:
>>
>>> Gaudenz,
>>>
>>> first thanks a lot. This is very constructive, and I appreciate a lot
>>> this kind of contribution, which I'm sure, everyone will enjoy a lot.
>>>
>>> About the quiet mode of the kernel, I'm really not sure. I've been
>>> thinking about it for a long time, and I still can't make up my mind
>>> about if we need the non-quiet output or not. Let me share what I have
>>> in mind.
>>>
>>> All that kernel log is a lot of text wont be very useful. There's no
>>> advantage, in the cloud use case to see it, as we don't expect any
>>> kernel failures. More likely, if there's a problem, it will be because
>>> of something else (cloud-init, networking, etc.).
>>>
>>> Why do you think kernel logs is useful?
>>
>> I don't care that much about the kernel logs, but quiet suppreses much
>> more than just the kernel logs. It also makes the systemd output much
>> more terse. I think it's usefull to see which services were started in
>> which order.
>
> If we want all systemd output, then we can set this:
>
> systemd.show_status=true

That's another option, but it's very specific, see below for my
arguments.

>
>> And I'm not very convinced that we are really not interested in the
>> kernel logs either. If things go wrong, there may very well be a hint in
>> there.
>
> Hints of what kind? What kind of failures are you expecting to see?

It's the nature of future failures that we don't know what they are and
how we would debug them. So IMO it's best to have as much debug output
as possible available at a place that's not obstrusive. I see the serial
console on a openstack instance more like a logfile than like a console.
If there is too much output you can easily grep your "nova console-log"
output for things that are specific to your actual case.

Another point why I would like to see logging turned on by default is
that it's quite hard (if not impossible) to turn it one once your
instance does not boot anymore.

>
>> I don't think they hurt in any way, so I would enable them.
>
> I do. The logs become too big, and you see irrelevant things, on which 
> you will waste your time searching for what you are looking for (ie: a 
> daemon which didn't start or something like this).
>
>>> On 05/18/2015 04:39 PM, Gaudenz Steinlin wrote:
>> One other advantage of grub is that it's much better integrated into the
>> Debian system. update-grub is automatically called and makes sure that
>> it always boots the latest kernel with all security fixes. It also
>> automatically adds a rescue option.
>
> In OpenStack, there's an option to add things on the kernel command 
> line, so the grub thing for it is irrelevant.

I would be very suprised if this worked with disk based images like the
Debian OpenStack image. How would the parameters passed from nova
through the boot loader to the kernel? I'm quite sure this only works
with separate kernel and initrd base images like AMI. So there is no way
to adjust the boot parameters once your instance does not boot anymore.

> Upgrading to the latest 
> kernel image will also work with syslinux, AFAIK. (If not, let me know, 
> and then it must be easy to fix.)

It works if you rely on the /vmlinuz and /initrd.img symlink pointing to
the right place. This is one part of my patch. But then still there will
be no menu entry for the previous image and for rescue boot and so on.

> Also, I do not think it's the cloud 
> workflow to upgrade a kernel on a running VM, then reboot it. IMO, you 
> should just update your image and boot that instead. If you're not doing 
> that, then you are missing the point of using cloud computing.

I don't think the purpose of this image should be defined so narrowly.
While this is the standard use case for "cloud computing", OpenStack is
in no way limited to that. Notably in private cloud settings it's also
used for long running instances. And there is also the use case where
you use the image to create a long lived volume from it.

>
>>> Sure. But this means renaming the output file after the facts, which we
>>> may not want to do. Best would be to find a way to know what the is the
>>> current point release number out of a wget / curl command (or something
>>> like this...). I'm opened to any suggestion for this.
>>
>> I don't see why creating the image to a temporary file that get's
>> renamed at the end is problematic. I'd prefer that to the hardcoded
>> version number.
>
> Because I'd like to implement a check to see if the file is present on 
> the HDD or not before starting the build. Something like this:
>
> if [ -e ${AMI_NAME} ] ; then
>      echo -n "There's already a file named ${AMI_FILE}: overwrite it?"
>      read overwrite_ami_file
>      if [ "${overwrite_ami_file}" = "n" ] ; then
>          exit 1
>      fi
> fi
>
> If we don't know the file name before starting, then it will do all the 
> process of debootstrap before knowing that probably it should stop. 
> That's really not user friendly.

I don't think there is much need for such a check, most Unix tools
overwrite files without prompting. IMO this is to be expected. Or you
could just reverse the logic so that the user has the option to choose a
different filename in that case or to rename the already existing file
or whatever you like.

>
>> But if you find a way to get the version number before starting the
>> build that's fine too.
>
> Would you know how I may get the content of /etc/debian_version from 
> base-files in Jessie with couples of wget / curl calls? That may be one 
> of the ways. Maybe there's some more easy way to go. Adding Steve 
> McIntyre as Cc: as he may have the answer.

I don't know anything other than downloading the base-files deb,
unpacking it and looking at the file. If you want to implement that,
that's fine too. I don't really care. I just think hardcoding the version
number does not scale at all, is prone to errors and makes building
images for different distributions impossible.

Gaudenz