Permanent transition tracker for Perl6 ? (was: Re: Packaging of libraries with unstable ABI (D, Rust, Go, ...))

[Ian Jackson]

Dominique Dumont writes ("Re: Permanent transition tracker for Perl6 ? (was: Re: Packaging of libraries with unstable ABI (D, Rust, Go, ...))"):
> Sorry for the long delay

It can help to take a while to think about awkward problems...

> On Tuesday, 30 May 2017 17:37:44 CEST Ian Jackson wrote:
> > [someone:]
> > > is letting daemon write its own pre-compiled file a security risk ?
> > 
> > Possibly, but the cache area should be by uid not by USER.
> 
> uh, I fail to see the distinction... AFAIK, there's a 1-to-1
> relation between user and uid. What did I miss ?

Your assummption about uid-user mappings is not always true, I'm
afraid.  And the process can not reliably find its username.  The
username in $ENV{USER}, or implied by $ENV{HOME} is definitely
unsuitable.  But the euid is probably suitable, so one could just use
that (and do not try to map it to a name).

But actually I think the whole scheme should be done the way Python
does it.  If the compiled filename needs to be decorated with a raduko
version (or even a raduko hash) then that is probably fine, but it
should be in the same place as the source file (or at least, a
filename derived from the source filename so that the source filename
can be recovered from the compiledd filename).  This has worked fairly
well with Elisp and Python and we have fairly good strategies for
dealing with it.

> > If so you can do the compilation
> > opportunistically.  Python seems to take this approach.
> 
> You mean at run time when a python script is run ? 

Yes, indeed.

> If so, how can a script invoked by a user can write a .pyc file
> owned by root ?

It can't.  What happens instead is that there are hook scripts which
arrange for the .pyc files to be created during installation.  (In the
postinst or a trigger, I think.)  During installation there is a brief
time when there are no .pyc files or the .pyc files are out of date;
the Python interpreter notices this, uses the .py file instead, tries
to rewrite the .pyc, finds it can't, and carries on anyway; the
downside is a small performance hit during this period.  Most of the
time the .pyc is there and valid.

> > Can you patch Perl6 to put the precompiled files alongside the
> > original source files, the way Python does it ?
> 
> I don't think so. A pre-compiled file name is a hash sum made of the
> file content, the rakudo version and maybe some other data that I
> forgot. Relating a perl6 source to the precompiled file is not easy.

It seems like it should be *possible* to patch Perl6 to put the
compiled file alongside the non-compiled one, and to have it contain
the non-compiled filename as a prefix.

Hashing everything so as to make the precompiled file literally
impossible to relate to the original file seems rather perverse.
(But perhaps the original filename is still recorded inside the
precompiled file somehow, so that a cleanup script could still find
out what to do, just less efficiently.)

> I'm thinking about pre-compiling at installation time, keep track of the 
> written files and remove them at de-installation time. The trick is to take 
> care of upgrade, both module upgrade and rakudo upgrade...

Yes.  If Perl6 can function properly in the absence of the files,
without writing them anywhere, then I think you can do this in a
"lazy" way (eg due to triggers).

I suggest you look at dh_pysupport2.

Good luck!

Ian.