[Soc-coordination] Compute Clusters Integration for Debian Development and Building - Report 4

[Rudy Godoy]

Hello, this is the fourth report for the project. First, I'd like to
apologize for being late, I took a short trip over the weekend and
missed the deadline. This time I've good news for my project. As
expected I've made the necesary changes to have Eucalyptus support ARM
images. The approach was extending and exposing an 'arch' field that
will be used by libvirt's XML domain file. I'm working currently on
having the involved functions working with the new extension and I
expect to have this done over the middle of the next week. 

What was done so far
--------------------
- Extended the ncInstance_t struct on util/data.h adding the archId
  parameter. This struct keeps the information regarding a node (nc),
  say RAM, Kernel, etc. Currently it has no way to set an arch in order
  to be used later.

typdef struct ncInstance_t{
...
    char reservationId[CHAR_BUFFER_SIZE];
    char userId[CHAR_BUFFER_SIZE];
    char archId[CHAR_BUFFER_SIZE]; // arch field added
    int retries;
...
}
  The Eucalyptus team was OK with this approach that I've proposed few
  weeks ago. For all my changes the approach is maintain compatibility
  with existing features. 
  We still have to define if we are setting a default value, say amd64,
  or keep it unset until the user sets one. For now we will be supporting
  amd64 and arm as valid fields.

- Modified the allocate_intance() function to support the archId
  field. This involved adding a new parameter and detecting whether its 
  present and setting de value accordingly. This function is later used
  by the doRunInstance() function in node/handlers_kvm.c

- Modified the involved functions that are called by the KVM handlers
  and result in passing parameters to the XML file generator for being
  used by libvirt's and KVM and have the image running under
  Eucalyptus. The new parameters and field are not required and only
  being used if they are set with a value. The archId parameter has been
  added as the last one, so existing function calls can work. This could
  be rearranged later, but will involve a more extensive revision with
  Eucalyptus developers.

- I've set a git local repo in order to keep track of the changes to
  Eucalyptus. Since this part is a sort of a Eucalyptus branch I took
  this approach. Steffen and I are discussing whether to release this as
  a pkg-eucalyptus branch (as dpatch, using existing packaging). Either
  way I'm putting the diff files form my git repo over the wiki and will
  put them on my site too. I plan to use git-svn in order to send them
  later to the pkg-eucalyptus repo once we are settled with this.

- Updated the project's wiki[1] page with information regarding all that's
  involved on my work, so it can be 're-played' or resumed later. 

1- http://wiki.debian.org/SummerOfCode2011/BuildWithEucalyptus/ProjectLog

What I plan to to over the next weeks
-------------------------------------
- Finish the modifications to the relevant functions. Expected dealine:
  next week.
- Do integration tests:
  - Write a test program, there's an example under node/ that could be
    helpful.
  - Generate the XML domain definition from Eucalyptus functions. The
  full cycle: handler_kvm -> allocate_instance -> get_instance_xml ->
  have the XML file properly generated -> qemu-kvm runs the instance
  using the proper emulator (arm in our case).
- Test runnning the image I've worked on the project's begining.
- Fixing bits and cooperate with the pkg-eucalyptus team on the
  packaging bits.
- Talk to Eucalyptus so they adopt our patches.
- Rock and roll :)


What would be nice to have after the project ends
-------------------------------------------------
- The Debian packaging *needs* work. I've already discussed this on the
  pkg-eucalyptus mailing list. I've also committed myself to help on
  that side even after the SoC ends. 
- Talk with upstream and Ubuntu to coordinate the packaging efforts and
  modifications to the software.
- Get adopters to use our work :)


By now, few weeks about to finish the SoC, I can say that it's been a
good experience and as I stated I'm planning to keep maintaing/working
on the software after that. I've already joined the pkg-eucalyptus team
and plan to contribute actively. I'm sad I didn't managed to get Debconf
this year, time was a constraint again, hopefully we can meet in 2012! I
guess that all by now!

Best regards,
Rudy