Fwd: Re: sound distribution practices

[Allison Randal]

(For background. I'll CC the Debian packaging lists on my reply.)

-------- Original Message --------
Subject: Re: sound distribution practices
Date: Sat, 07 Apr 2012 21:34:47 +0200
From: Jonathan Worthington <jonathan at jnthn.net>
To: Allison Randal <allison at parrot.org>
CC: parrot-dev at lists.parrot.org, perl6-compiler at perl.org

On 4/7/2012 19:11, Allison Randal wrote:
> Okay, instead of just complaining about this, I want to do something
> about it. The Rakudo packages are so fragile on Debian, that they need
> special constraints to make sure the Parrot packages are held back until
> the Rakudo packages are updated.
>
> So, why is Rakudo so dependent on one specific month release of Parrot?
> Parrot isn't moving so fast that the APIs are all changing every month.
> Is NQP or Rakudo not respecting API boundaries? Are there some libraries
> currently shipping in Parrot that should be in NQP, Rakudo, or broken
> out into their own distributions instead?
Easiest way to analyze it is just git log on
build/tools/PARROT_REVISION. Here I've done a very rough categorization
of the last 20 entries (which takes us back to October).

Some of them are to take advantage of Parrot feature additions:

    bump PARROT_REVISION to get separate unlink and rmdir primtives

    use parrot's new Hash.update method to speed up method cache generation

    bump parrot revision to one that has the is_inf_or_nan opcode

    use parrot op to switch to the profiling runcore

So not bumping would mean Rakudo forgoing making user-facing improvements.

Some are to get Parrot fixes:

    bump parrot revision to get MacOS build fixes

    Bump to a Parrot version with a fix for ByteBuffer segfaults, which
caused programs using Buf to sometimes segfault.

Again, there are Rakudo user benefits to following this.

Some are due to chasing API changes in Parrot:

    Update to new version of getprop op

    Bump parrot revision to kill_props_vtables marge commit.

The getprop op changed at a PIR level. No guts poking here, just a (for
the better) change. I should add that when I say "chasing" in reality
Parrot folks have supplied the patches here, and it's been managed smoothly.

One is due to getting a PAST improvement:

    bump parrot revision to get directaccess support, use directaccess
for most var lookups

We're currently in the process of redeveloping PAST so it's written in
NQP, can do native types better, is more memory efficient and so forth,
and that work will be in the NQP repository, which would have avoided a
bump of this nature - but it was just one bump out of 20.

Which leaves the biggest group, which are a combination of a policy of
targeting the latest Parrot in releases and wanting to have those using
Rakudo from git testing against latest Parrot after sizable changes, so
we can have faster feedback if there are problems, which would seem
desirable for both Rakudo and Parrot devs.

    bump parrot revision to 4.2.0 release

    bump parrot revision
    just to get some better testing of newer birds, no specific reason

    bump parrot revision to something after the cont_reuse merge

    bump parrot requirement to 4.1 release

    bump parrot revision to get some testing

    bump PARROT_REVISION to 3.11 release

    bump parrot revision

    Bump PARROT_REVISION

    Bump to Parrot 3.10 release.

    Bump to latest Parrot (not quite release yet) in order to get more
testing against it.

    bump PARROT_REVISION to after the green_threads merge to get some
more testing

    bump PARROT_REVISION

Granted some in this category aren't explicit about the reason why, so
there's some error in the analysis here.

> Rakudo should be able to run on any Parrot release within the past year
> (at least). The fact that it can't is a sign of poor software. We need
> to figure out what's broken and fix it. I'm happy to help. Highly
> motivated too, as it would remove a regular source of irritation.
There seems to be a few things going on here. One is that we don't
differentiate Parrot revision requirement updates that are "must have"
from those where we're bumping it for development reasons rather than
because it's a hard dependency. Another is that Rakudo's feature needs
often drive feature additions to Parrot and drive some bug fixes too; it
seems inevitable that Rakudo will want to quickly take advantage of them.

Generally, I think things are currently more -Odevelopment than
-Odistribution. I agree that makes things harder for those working on
distributions, and I'm interested to make things better - I want it to
be as easy for people to get their hands on recent Rakudo releases, and
having it packaged for distributions does help with that. On the other
hand, it's also an accurate reflection of the current state of both
projects: in need of development work, and optimizing for it.

Also just to check - are you packaging Rakudo compiler releases, or
Rakudo distribution releases (which are the ones we more intend for
packaging)? Mostly asking due to the timing of the email; we didn't do a
distribution release for March (so last interested one to package was in
Feb), and the next one is due this month.

Thanks,

Jonathan