[Openstack-devel] New version of cloud-init & OpenStack cloud image

[Thomas Goirand]

On 05/31/2013 01:38 AM, Jeff Licquia wrote:
> On 05/29/2013 09:45 AM, Thomas Goirand wrote:
>> I'm not sure how it compares to other scripts for building a cloud
>> image. I didn't have a look into them, as my goal was to make an image
>> specifically for OpenStack.
> 
> I had started on an OpenStack profile for build-debian-cloud, but ran
> into roadblocks getting the bootloader set up, and then got distracted.
>  Would it make sense, do you think, to integrate some of the features of
> this script into build-debian-cloud?

I just had a quite long (mail) discussion with someone who replied to my
ITP for the openstack-debian-image package. I think it would make sense,
but not if build-debian-cloud stays the way it is right now. Let me
explain in more details.

There are few things that bother me a bit. The most important one being:

boto_ec2_dir='/usr/share/pyshared/boto/ec2'
patch_url='https://bugzilla.redhat.com/attachment.cgi?id=455857'
[...]
wget -qO - $patch_url | patch -sfr - -d $boto_ec2_dir

This kind of wget from a random URL from the redhat bug tracker,
patching an existing python module without even doing a diversion, is a
show stopper. This would be strongly rejected by anyone being serious
about integrating the OpenStack cloud image into the Debian CD build
process as well.

Also, things in the init.d folder (eg: ec2-get-credentials
ec2-run-user-data expand-volume generate-ssh-hostkeys) are all
implemented in cloud-init (which by the way does other things). We
already invested some times for the cloud-init Debian package, and I
think it is a much nicer approach. I believe also that cloud-init is a
kind of standard which OpenStack will follow, so it is a good thing to
use it for this reason as well.

Also, the image build script is currently less than 200 lines of shell
script in a single file, while build-debian-cloud contains 147 files.
While I understand the need of having modular design, it might be easier
to maintain and gather contribution as a single smaller script. If it
isn't broken down into so many smaller files, it is harder to follow the
logic. In other words: I prefer keep things simple than modular.

Also, build-debian-cloud is using S3 volumes to do its work, and is
building an AMI partition format, while I'm working with a disk image
(using parted, kpartx and so on). These are different needs.

Don't take me wrong. I'm sure that build-debian-cloud has some value,
and that it's great for EC2 and GCE, directly building from an instance
there. Though for an image for OpenStack, built out of only Debian
packages, with in mind the goal to have it released together with the
rest of the Debian CDs, I'm really not sure that is the way to go.

So yes, I would be pleased if we had a single package to handle all
cases. Though *in its current form*, it's going to be a lot of work to
integrate into the build-debian-cloud, work which I have no time for.
Also, rather than doing this integration, I would much prefer
integrating my work into a larger project like Debian live (eg:
live-build), which is more or less what Ubuntu has done (they use a
patched version of live-build 3.x).

> I'm using OpenStack from the grizzly repositories on wheezy, and gave
> the image a spin.  One minor nit: the script doesn't write to the
> console log (console=ttyS0?).

Yeah, there is an issue with that. When I do, on the kernel command
line, something like: "console=tty console=ttyS0,115200", then only the
last one is taken into account. So, either "nova console-log" works, or
my spice console in Horizon. So I decided to ignore this problem for the
moment (as I lack time to investigate). If you have a solution, I'd be
very happy to use your fix.

> Also, not sure if the networking
> parameters were meant to be set up via cloud-init; networking worked out
> of the box, but via DHCP.

Right, the image is setup to use DHCP indeed (for up to 3 network
cards). Cloud-init starts just right after the image gets an IP from
quantum-dhcp-agent.

> Thanks for providing this!  I'm migrating off a hand-hacked virtual
> system with libvirt, and getting a good working Debian cloud image going
> was a roadblock.

FYI, I have uploaded this as a package, which is currently waiting in
the FTP master NEW queue. The package is also stored in the Alioth Git,
at the below URL:
Vcs-Git: git://anonscm.debian.org/openstack/openstack-debian-images.git

It's nice to have it as a package, so that it can pull dependencies.

Cheers,

Thomas Goirand (zigo)