Extending buildserver.net (debian/ ubuntu buildd environment)

[Kilian Krause]

Mark,

Am Montag, den 08.05.2006, 20:05 +0100 schrieb Mark Purcell:
> Kilian,
> 
> The other team I do a fair bit of packaging with is the kde-extras team.
> 
> Like pkg-voip we are jointly maintaining a number of packages for both Debian 
> and Ubuntu, and something like the http://buildserver.net/ setup which 
> automatically builds packages for etch, sarge, sid, breezy and dapper from 
> the same set of sources in svn.debian.org would be excellent and save an 
> awful lot of cross distribution/ release porting time and effort.

Hehe, sure would. ;) The limitation I'll be facing first is running out
of CPU time and surely disk/traffic at some point. Yet compilation power
for 5*arches*packs is quite some demanding lot already. This does not
yet include that we would love to do some BSD ports, win32 and rpm packs
through the same infrastructure too. ;)
For this, I guess I do need to check whether pkg-kde-extras is small
enough to include it into the current infrastructure. 

The other overhead I'm currently also facing is the buildd-setup
limitations with the current scripts. Bastian Blank and I are already
preparing some new-style buildd code so that efforts can finally go into
packaging rather than the buildd. The new code should fiddle correct
versions of build-depends etc. itself and basically "just work". ;)

Right now, clearly every distinct new "source" to the buildserver.net
means yet another set of chroots (6 arches with one "arch" being "all"
right now). We will extend this to 6+1 arches with the new sparc buildd
ReallySoonNow and I hope that we can get first tests with the new buildd
code in the near future, yet Bastian is pretty busy with d-i work, so
for now we have to live with what's there.

> Have you considered extending buildserver.net, perhaps for another 
> svn.debian.org project such as kde-extras, but in the fullness of time across 
> all of svn.debian.org?  Perhaps as a debian.org resource?

For *ALL* of svn.d.o would for sure exhaust all free cpu power there
currently is in the buildserver.net network. Moreover I'd rather offer
people to commit a "ready-for-verfication"-version when they feel like
rather than just snapshotting at random. Yet, to secure the integrity,
I'd prefer to make this with the new code which will jail builds
correctly into LVM and Vserver isolation. 

If you have any decent proposal where to gather more free and
sufficiently large resources of cpu time, then I'm all ears. Probably
SPI could help if we drop them a note. You know by chance, who's in
charge there for investments like this? Or is it rather
leader at debian.org whom we should approach? It'll be for sure not a
problem of scalability with the setup itself. We have always had
scalability in mind and I'm entirely positive, that the backend will be
able to cope with quite some more buildds. The sore spot as far as we
know yet is wanna-build which will deadlock itself if you tease it too
hard with buildd requests. That's another part of the reason why new
buildd code is needed.

> I presume you have just taken the buildd architecture and extended it for the 
> various distributions/ releases in Debian & Ubuntu.

Yes, I have not taken any of the Debian buildds, but it's the code and
setup idea of the original sbuild/buildd/dak which was enhanced to fit
what we needed. All of which is available in the
http://archive.buildserver.net/build for those who would like to check
the diff. I'll put up the modified britney code as soon as the build
distribution will also exist in dak for each dist (right now there's
only one "build" to server all clients) which obviously can't be sid
+etch+sarge at the same time. ;)

Thus, bottom line: I'll keep your request in mind and add pkg-kde-extras
as soon as I have an idea how much impact this would be on the buildds.
Adding the script is a matter of minutes from what's already there. =)

-- 
Best regards,
 Kilian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://lists.alioth.debian.org/pipermail/pkg-voip-maintainers/attachments/20060508/72ea83f3/attachment.pgp