sound distribution practices

[Patrick R. Michaud]

On Sat, Apr 07, 2012 at 03:24:37PM -0700, Allison Randal wrote:
> These are things that Rakudo shouldn't care about. A bug-fix or
> performance enhancement in Parrot is good, but doesn't actually affect
> Rakudo's ability to build or run.

Sometimes it does.  See the flurry of difficulties we had last April
and May when Rakudo testing discovered that Parrot performance had 
declined by >100% over the previous several months (and Parrot
developers were completely unaware of it).

> Did the actual APIs change?  Different arguments required, or
> substantially different behavior so that there was absolutely no way
> Rakudo could run on both the old and new versions of Parrot? 

I can think of several instances where the APIs changed, such that
Rakudo could no longer run on an older Parrot or would require
substantial detection of which version of Parrot is being used
and change code generation or behavior accordingly.

> If so, then Parrot could be doing a better job of supporting 
> language development here, by preserving old APIs.

> > One is due to getting a PAST improvement:
> [...]
> This is eager adoption of new features again. But, in this case I agree
> the fix is that PAST should move to NQP and out of the Parrot packages
> entirely. For one thing, having NQP depend on a library in Parrot that
> depends on NQP is nightmarish for packaging. 

FWIW, I don't think this circular dependency exists (Jonathan doesn't
think so either).  AFAIK, there's nothing in Parrot that has any sort 
of dependency on NQP.  (Note that there's a distinction between
"nqp" and "nqp-rx"; Parrot includes some nqp-rx stuff, but doesn't
make use of nqp.)

> Indeed, not enough detail there for much analysis. But, in general there
> is a difference between what you choose as your development process, and
> what you choose as your release process. It is very sensible to always
> test Rakudo against the latest versions of Parrot. But, that doesn't
> mean you can stop testing against older version of Parrot. The most
> important older versions to test against are the ones shipped in various
> distributions.
> [...]
> It would also be helpful if Rakudo and NQP adopted more strategies to
> keep the minimum required version as low as possible. Like "soft
> launching" new features so they're simply not available when they depend
> on a newer Parrot feature.

I will go back and test, but I'm willing to hazard a guess that
there are very few Rakudo releases that would be able to go very far back 
in Parrot's release history and still be functional, even with a sort of 
"soft launching" described here.  

For a recent and directly "on topic" example, take a look at 
https://github.com/perl6/nqp/commit/242f6e5997f6c454246fcf97b8a2a5d038d95931 ,
provided to NQP by a Parrot developer on March 5.  This commit means 
that subsequent versions of NQP and Rakudo are absolutely unable to
work with any version of Parrot prior to 4.2.0, because it's
an API change.  I also don't see any "soft launch" for this commit
that would make it easy for Rakudo/NQP to preserve access to
older versions of Parrot.

(Better would probably have been for Parrot to introduce a new opcode
with the correct order of arguments rather than changing the API of
an existing opcode, but that's not what happened here, and the API
change was not introduced at the request of Rakudo/NQP.  As far as
we can tell it came entirely from the Parrot development team.)

> I guess that raises the question for me of whether Rakudo is ready to be
> packaged. I don't have any trouble with Parrot packaging, but I do get a
> regular series of unusual requests from the Rakudo packagers. If the
> Rakudo project is totally focused on the use-case of downloading the
> source code or checking out from git and compiling from source, maybe
> it's just not ready for packaging.
> [...]
> I'm sure all the Rakudo packagers will agree with me that they'd really
> hate to remove it from Debian. But, if packaging isn't something Rakudo
> wants to support, then we shouldn't force the project into a different
> mindset. To give you an idea of what Debian packaging is looking for,
> take a skim through the upstream guide:

I think this line of thinking reaches the wrong conclusions.
What can be said with certainty is that for every Parrot release,
there is at least one corresponding Rakudo release that can make
use of that Parrot.  There's no reason to talk of "Rakudo isn't ready
for packaging", because for any release of Parrot there is at least
one contemporaneous release of Rakudo that can make use of it.

What I suspect may be happening is that the Rakudo packagers
are being a little over-eager about trying to provide the
later versions of Rakudo, when there's not a packaged version
of Parrot that can support them.  In this case, Rakudo's official
suggestion would be for Rakudo packagers to stick to the most recent
version of Rakudo that is compatible with whatever release of Parrot
is available in that environment.  Since Parrot has a much slower
release and packaging cycle, this does mean that Rakudo packaging
will be limited by progress in Parrot packaging, but the Rakudo
team is largely okay with that.  

I will do some work to find ways for Rakudo to maintain separate 
"desired" versus "minimum" versions of Parrot.  However, my
past experience tells me that this won't really resolve the
problem.  From Allison's original message:

> Parrot isn't moving so fast that the APIs are all changing every month.

Perhaps not every month, but our experience over the past year or so
is that they do change every couple of months, in ways that directly 
affect Rakudo/NQP development (as illustrated by the 242f6e59 commit
above).  Again, I'll do some investigation, but my suspicion is that
over the past year, Rakudo and NQP would have been unable to hold 
the same "minimum Parrot revision" for more than two consecutive
months.  Given the fact that both Parrot and Debian have "stable
release cycles" longer than that, it's not clear that a minimum
Parrot version would help much.

Pm