[pkg-php-pear] composer and debian

[Mathieu Parent]

2013/7/2 andrea rota <a at xelera.eu>:
> On Sun, Jun 30, 2013 at 07:07:47PM +0200, Mathieu Parent wrote:
> [...]
>> > 'php composer.phar install' also adds into the 'vendor' folder it
>> > creates some autoloader code, which basically makes available all the
>> > dependencies installed in a project's 'vendor' folder available to
>> > any code using them.
>>
>> Yes. I think this Autoloader part should not be integrated to the
>> Debian packages, as it is specific to each application. This is a
>> feature for developers but not for system administrators. Composer is
>> a "Dependency Manager" and not a package manager. It ease development
>> but has the same drawbacks of Ant for Java: Every lib is duplicated in
>> every application.
>
> yes, the vendor folder where Composer installs a project's dependencies
> is indeed for each project to manage.
>
> i think there is a benefit to having *some* libraries/binaries installed
> system-wide (like PEAR packages), e.g. Composer itself (with its main
> script installed as /usr/bin/composer), wp-cli [1] etc.

I totally agree.

> In these cases, the autoloader stub should be needed so that the
> system-wide binaries can use code in /usr/share/php (or similar location
> of Composer-based libraries should be installed somewhere else).
>
> In my current tinkering with a php-composer Debian package, i have tried
> running a 'composer install' to let Composer generate this stub for
> Composer itself, and packaged it (slightly modified to refer to
> libraries installed in /usr/share/php) as quilt patches on top of the
> debian-sid branch via gbp-pq.
>
> there are some interesting issues arising here - e.g. for Composer
> itself, three of its five dependencies (those from Symfony2) are (being)
> packaged for Debian as PEAR packages, using dh_phppear, whereas its
> other two dependencies are packaged via dh_phpcomposer (and luckily
> don't have other dependencies themselves, so at least to bootstrap the
> whole thing, the buck stops here); so it would be good to find a safe
> and reliable way to let Composer-based packages autoload code from PEAR
> packages.

Yes.

> the real "fun" starts with path names (see below)...
>
> [1] http://wp-cli.org/
>
>> > in case of system-wide installs (to start with Composer's own library
>> > code and that of its dependencies), this autoloader code needs to be
>> > generated in order to make the code installed under
>> > /usr/share/php/* available.
>>
>> The composer ".deb" doesn't have to be installed by composer. Why
>> don't you install it with dh_install?
>
> i'm not sure i follow you here, but yes, in my very tentative tinkering
> with these, i'm indeed using dh_install, although part of the code files
> installed via dh_install are pre-generated via Composer and manually
> added to the package source once (as quilt patches as mentioned above).
>
> There doesn't seem to be much going on in these autoloader stub files
> apart from mapping PHP namespaces to paths on the filesystem, so
> *perhaps* this autoloader stub code could be made a Debian package of
> its own, able to read an array of namespace->path mappings which should
> be installed by each Composer-based Debian package. this may be clean,
> and avoid messing around with patches to php-composer, but needs to be
> tested.
Agree.

> Probably .reg files from PEAR packages can be used as well as source
> data for the Composer autoloader, which should help interfacing
> Composer-based packages with PEAR packages.

Agree.

> I am not familiar yet with the details of .reg files in
> /usr/share/php/.registry/<channel>/ but the equivalent for
> Composer-based packages *should* be simpler (i guess .reg files
> do much more than providing information for an autoloader, but for
> Composer packages the small mapping needed could probably be generated
> directly from the source's composer.json file *and* from its Composer
> dependencies' composer.json - if so, pkg-php-tools' bijective mapping
> should provide all the information needed).

You don't need to read registry either. The mapping in PEAR is good:
Foo_Bar_Class => /usr/share/php/Foo/Bar/Class.php

>
>> > in a way, upstream addressing their issue #55
>> > (https://github.com/composer/composer/issues/55) should help here, but
>> > in the meanwhile i'm reviewing the workarounds mentioned in this thread.
>> > any further suggestions are welcome.
>> >
>> > if/when upstream supports that, Debian packages of composer modules
>> > should be able to dev-depend on php-composer and use it to install their
>> > library code in the appropriate locations, i guess through an updated
>> > dh_phpcomposer, which would be great - but it's early for this.
>>
>> Yes. But this requires that system-wide installation of Composer
>> package respect Debian policy. Particularly the section related to
>> configuration files.
>
> yes, and this is rather delicate as Composer doesn't seem to be
> enforcing much in terms of what goes where in (Composer) packages
> installed under a project's vendor folder. ideally this could benefit
> from some coordination with upstream (there seems to be interest, but not
> enough drive for use cases of 'global' installs as equivalent to npm -g
> for Node).

Yes

>> > another question i am wondering about is naming of packages: pkg-php-tools
>> > correctly uses the whole composer package name to infer a debian package
>> > name - e.g. seld/jsonlint translates to php-seld-jsonlint. this helps to
>> > differentiate upstram forks, but i'm wondering, in the longer term, how
>> > to deal with changes of name upstream (e.g. if the 'org' or 'user' part
>> > of a org/project or user/project composer package name is changed in
>> > packagist, or if the reference implementation becomes one from a
>> > different org); moreover, it's probably unusual to have user or org
>> > names in debian packages: i guess there isn't much that can be done
>> > about this, but wanted to highlight this side of the composer naming
>> > game.
>> >
>> > (so i should probably rename the two other composer dependency packages
>> > i'm working on to php-seld-jsonlint, from php-jsonlint, and
>> > php-justinrainbow-json-schema, from php-json-schema).
>>
>> There are several mappings:
>> - Composer name (project/lib) <-> Debian name (php-project-lib):
>> pkg-php-tools have a bijective algorithm for this one (minus case)
>> [note 1]
>> - Composer name <-> path: this is upstream choice [note 2]
>> - PEAR name (pear.project.org + Some_Name -> php-project-name):
>> pkg-php-tools algorithm is injective [note 3]
>> - PEAR name <-> path [note 4]
>>
>> Also, some packages have a PEAR name and a Composer name.
>>
>> I'm open to any change, but beware that changing PEAR naming scheme
>> will bring a transition (for example php5-* to php-* as PECL packages
>> are really PEAR packages and being PECL is an implementation detail).
>>
>> This easier for Composer as support for it is very new.
>>
>>
>> [1]: ... with th drawback you have mentioned, and the risk to have
>> several projects with the same shortname (using DNS name would have
>> been better here)
>> [2]: ... and sometimes it conflicts
>> [3]: Maybe we should move to something like Fedora
>> (https://admin.fedoraproject.org/pkgdb/acls/name/php-horde-Horde-Alarm)
>> or a bijective one: php-pear.horde.org-Horde_Alarm?
>> [4]: ...and sometimes it conflicts (but mostly when the PEAR name is identical)
>
> yup, this mapping part is where the "fun" starts, especially because a lot
> of Composer projects use github or similar, where forking is very cheap,
> compared to curated distribution packages, which can lead to some abuse
> of dependency hell where project A depends on library X and project B
> depends on fork X1 of library X, and so on, which crossed with the
> mappings possibilities you outline, can be quite an headache (although i
> imagine that very few Composer packages will end up being packaged for
> Debian or other distros, so the issue could be somewhat limited and
> controlled downstream).
>
> as a first attempt, i have used pkg-php-tools' answers for the Debian
> package names of the few Composer-only packages i am working on:
> * seld/jsonlint -> php-seld-jsonlint
> * justinrainbow/json-schema -> php-justinrainbow-json-schema
> * wp-cli/php-cli-tools -> php-wp-cli-php-cli-tools
> * mustache/mustache -> php-mustache-mustache
> * wp-cli/wp-cli -> php-wp-cli-wp-cli
>
> with various degrees of name awkwardness, despite being correct...
>
> (to add some fun, e.g. mustache/mustache is actually bobthecow/mustache
> on github.com...)
>
> <highly_speculative_mode>
> i'm not too bothered about what schema is suggested for the Debian
> package names, but clashes must be definitely avoided in filesystem
> paths, and this is not always happening upstream especially when forks
> are involved. my hunch is that the actual path can be made unique by
> generating an unique id/string (could be the debian package name which
> would be unique by definition, or something derived from packagists's
> project/lib, or even a SHA or similar), and as long as the Debian
> package registers the mapping path->namespace(s) somehow, perhaps just
> by saving somewhere under /usr/share/php the upstream composer.json, the
> PSR-0 autoloader assembled by Composer should just do the trick.
> </highly_speculative_mode>

Yes. But this will break packages not using autoload (but using
{include,require}{,_once}).

> i'll take a first stab at some code implementing this in a personal
> Debian package (e.g. php-composer-debian-autoloader) with a brief
> blueprint - it should be easier to see/test code rather than only
> following an email thread.

Yes. Coding is the way forward. We will upload this to unstable once
we think that the behavior is deterministic enough.

>
>> > and finally - pkg-php-tools should probably enforce dependency on
>> > php5-cli *only* (rather than php5 | php5-cli) via substvars when there
>> > is a 'bin' section in a project's composer.json (this is to avoid
>> > lintian error php-script-but-no-phpX-cli-dep). if i'm not missing
>> > something here, i can report a bug for this against pkg-php-tools.
>>
>> You can file a bug for this.
>
> ok, thanks, will do.
Seen. thanks

Additionnal note:

I think coordination with other distrib (and Composer and PEAR
maintainers) should be done. I'm short in time, so can someone provide
a summary of questions?


Cheers.

--
Mathieu Parent