<div dir="ltr">I'm trying to think through some various "obvious" alternatives to the current debcargo approach, and figured I'd dump them here in case it triggers useful discussion:<br><br>(I've used the term cargo "features" below. I realised after writing most of it that I probably should say "optional dependencies", since features include both optional dependencies and conditional compilation. I'm ignoring that last bit, because that doesn't affect packaging dependencies...)<div><br>*** Alternative: Ignore cargo features, install superset of build-deps<div><br></div><div>Build-dep on all possible source packages (superset of all cargo features). This was initially explored and rejected in favour of the current debcargo features approach - but maybe we can go back to this with enough work on the edge cases.<br><br>- Removes all the trivial binary "feature" packages (yay).<br>- Building a Rust binary package will download _lots_ of extra build-deps (eg: multiple TLS implementations), many of which will be ignored. Hopefully none conflict (pure-Rust shouldn't ?? but they might pull in non-Rust deps that do conflict)<br>- Circular Rust dependencies can't be built without bootstrap hacks because Debian tools don't deal with cycles in *build-dep* dependencies when building from scratch (boo).<br><br>Possibly(??) some extra automated tooling around that last point (bootstrapping circular build-deps) works out to be better for the project overall than the some extra automated tooling to help with the explosion of trivial "feature" binary packages. I think this is probably the most obvious point where the different points-of-view are arriving at different conclusions currently. </div><div><br></div><div>*** Alternative: Lift "features" into application build-deps, remove library deps<br><br>Rethinking everything from first principles:<br>- When building a Rust application, we "just" need to get the Rust source for the current package and all the library dependency packages onto disk somewhere, and then invoke cargo. Conceptually, it's not that hard.<br></div><div>- Installing other packages at build-time means build-deps, without a significant change to the way Debian works.</div><div>- That means Rust library source has to be in binary packages.</div><div>- The obvious way to do that is to have a 1:1 Rust library source->binary package for each upstream Rust crate.<br></div><div><br>(So far so good - now we get into the controversial parts.)</div><div><br>Split the Rust world into "library" (source bundle) and "application" (compiled binary[*]) Debian source packages.<br><br>[*] "Anything that actually gets compiled" also includes dylib plugins and other exotic things if we want to be strict. I'm using "applications" to include all these.<br><br>"Libraries" don't actually get compiled - they're just repackaged Rust source. So these packages don't _actually_ need any dependencies in the technical Debian sense. If we remove all the dpkg dependencies, then we have no problems with circular dependencies - they really are just source tarballs trivially repackaged into .debs.</div><div><br>"Application" packages now have to do all the hard work: They need to pull in all the source libraries they need (including transitive dependencies), _and resolving features_. Resolving features at the last compilation step is good, because it's actually how the Rust/cargo system works anyway. We need to write a debcargo-replacement that expands out the list of transitive dependencies _taking into account features_ into an explicit list of Debian "library" package build-deps.</div><div><br>- Removes all the trivial binary "feature" packages (yay. But see [+])<br>- No circular build-dep dependencies (yay).<br>- Applications will now have enormous build-dep lists (imagine mozilla). .. It's the same _actual_ build-dependencies they would have any other way, just flattened a bit. Is this a problem?<br>- Every change in a transitive dependency needs a change in the end-application build-deps .. but maybe this doesn't matter, since it needed to be rebuilt anyway (because static linking).<br>- Uploads to library packages are basically untested, since no compilation happens (that's the point). This is probably ok if tests are done some other way (autopkgtest?)<br>- [+] I think we need some new trivial _source_ packages for crates that include both "application" and "library". The "application" source package will depend on the library package (in the usual way described above) and invoke cargo as required to get the desired application binary. I don't know whether there will be more or fewer than the feature packages overall, but they should not churn NEW as much (since once they exist, they will tend to continue to exist, and there is probably at most one per upstream crate.)<br><br>*** Alternative: Improve Debian source tools<br><br>Some changes I haven't thought enough about. Maybe we make it so you can build-depend on *source* packages or something equally radical (introduce a new category of "archive" package?) - since source packages are "trivially" packaged in a way that allows tools to unpack a circular dependency without issues.<br><br>That seems hard, but maybe worth pursuing given this "new" rust/golang pattern of deferring just about all compilation until the "application" package. I think the right solution here would be bigger than Rust and outside the scope of this thread.</div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 19 Oct 2020 at 06:05, Joerg Jaspert <<a href="mailto:ftpmaster@debian.org">ftpmaster@debian.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 15925 March 1977, Geert Stappers wrote:<br>
<br>
> <a href="http://meetbot.debian.net/debian-rust/2020/debian-rust.2020-08-26-18.58.log.html#l-102" rel="noreferrer" target="_blank">http://meetbot.debian.net/debian-rust/2020/debian-rust.2020-08-26-18.58.log.html#l-102</a><br>
> is what I'm aware of.<br>
<br>
Thats more/better than just the video, thanks.<br>
<br>
Well, reading through that, let me comment on some points:<br>
<br>
- Yes, we *can* allow some packages to bypass bin-NEW. That list is <br>
naturally strictly limited.<br>
- No, thats currently not up for rust.<br>
- Sure, tech-ctte is a possible way to override us, but I hope we can <br>
get something sorted between us without that hammer.<br>
- Yes, empty packages that are nothing but a container for <br>
Depends/Provides lines are a problem, and yes, due to getting the <br>
indexes and database sizes needlessly up.<br>
- Insanely huge provides are also not good, but maybe better than the <br>
packages. Tools that break ought to be fixed.<br>
- Or maybe we can add something new, no idea. Lets see.<br>
<br>
>> and more important, did any action afterwards happen to come up with <br>
>> a nicer way?<br>
>> And then, which way would that way be?<br>
> Good question.<br>
> I do hope it is the impulse that this project, Rust in Debian, was <br>
> waiting for.<br>
<br>
> With the help of favorite $DEITY feel those who care about this <br>
> project<br>
> comfortable enough[1] to speak freely in the up coming discussion.<br>
<br>
Noone will be hit for suggesting something, so whoever has something to <br>
say, speak up. Or nothing will go forward, which isn't good.<br>
<br>
>> Or the other way round, for someone that has only a very basic <br>
>> understanding<br>
>> of rust - whats the goal here that should be supported? Maybe we can <br>
>> come up<br>
>> with a better way that doesn't let our tools or Package files explode<br>
>> needlessly?<br>
> In case it stays quiet in here:<br>
> Next scheduledd IRC-meeting in #debian-rust is<br>
> Wednesday October 28th 19:00 UTC<br>
> Inviting FTP-Masters for that meeting.<br>
<br>
I put it in my calendar, but can't make a hard promise. I try, though.<br>
Also, if we stay on the current point, I doubt it will gain anyone much. <br>
So hopefully more will happen by mail before then.<br>
<br>
-- <br>
bye, Joerg<br>
<br>
_______________________________________________<br>
Pkg-rust-maintainers mailing list<br>
<a href="mailto:Pkg-rust-maintainers@alioth-lists.debian.net" target="_blank">Pkg-rust-maintainers@alioth-lists.debian.net</a><br>
<a href="https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-rust-maintainers" rel="noreferrer" target="_blank">https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-rust-maintainers</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"> - Gus</div>