[Pkg-javascript-devel] Bug#995722: Not running tests because tests miss source code is not useful

[Thomas Goirand]

On 10/7/21 11:40 AM, Pirate Praveen wrote:
> 
> 
> On 7 October 2021 3:02:55 am IST, Thomas Goirand <zigo at debian.org> wrote:
>> On 10/6/21 6:53 PM, Pirate Praveen wrote:
>>> [adding -devel]
>>>
>>> On ബു, ഒക്ടോ 6 2021 at 12:16:07 വൈകു +0200 +0200, Jonas Smedegaard
>>> <jonas at jones.dk> wrote:
>>>> Quoting Yadd (2021-10-06 11:43:40)
>>>>>  On Lu, 04 oct 21, 16:40:48, Bastien Roucari�s wrote:
>>>>>  > Source: src:node-lodash
>>>>>  > Version: 4.17.21+dfsg+~cs8.31.173-1
>>>>>  > Severity: serious
>>>>>  > Justification: do not compile from source
>>>>>  >
>>>>>  > Dear Maintainer,
>>>>>  >
>>>>>  > The vendor directory should be emptied
>>>>>  >
>>>>>  > The debug version is compiled without source (lintian warn) and
>>>>> moreover the
>>>>>  > rest of file are already packaged
>>>>>  >
>>>>>  > grep -R vendor * gives only a few hit that could be cured by
>>>>> symlinking
>>>>>  >
>>>>>  > Bastien
>>>>>  Hi,
>>>>>
>>>>>  this files are used for test only, maybe severity could be decreased.
>>>>
>>>> I find the severity accurate: Relying on non-source code is a severe
>>>> violation of Debian Policy, not matter the purpose of relying on it.
>>>
>>> I think we should change the policy here. Running tests helps improve
>>> the quality of the software we ship. Many times the vendored code is
>>> used to ensure the code does not break in a specific situation. I don't
>>> think reducing test coverage in such situations is really helpful.
>>
>> Right, running tests helps improve the quality of software we ship.
>> Which is why you probably need to test using what's shipped in Debian
>> rather than using a vendored source-less code.
> 
> We are not shipping the source less code.

You are: Debian also ships source code.

> This is used only during tests. I don't think we are not gaining anything by removing tests here. Just making it harder for the package maintainer to run tests.

You would not gain anything by removing tests, but you would win by
making these tests completely free software.

>> If we rely on non-free code for tests, that's really bad too, and that
>> must be avoided just like we're avoiding source-less code everywhere
>> else in Debian. The policy shall not change, please.
>>
> 
> The code is not non-free here, just a specific version of a Free Software code built outside Debian.

We build from source...

> I think tools required for tests should be considered separately from tools required to compile. I think it should be treated similar to test data.

I don't agree.

> What you are proposing would require the package maintainer to adapt these tests to versions available (many times with different API versions) in Debian and the easier choice is disabling tests.

No. I believe it's ok to have an embedded version of the JS files in the
upstream code. This is a *very* different issue, please do not mix them.
What I don't like is using a minified version of the JS files. That's
*very* easy (hum... trivial?) to add a non-minified version in your
Debian folder, and use that for tests. You don't care if running the
tests is a little bit slower (because using a source-full version), do you?

However, there's this:

On 10/7/21 6:17 PM, Richard Laager wrote:
> Running tests against vendored dependencies one isn't going to use at
> run-time is of limited usefulness.

Best is, if you can, use the library packaged separately, in Debian,
both for tests, and runtime. This way, you do ensure that:
- patching Debian for security is still a thing
- the package can run with the Debian version of the lib

I think it's less grave than just saying "oh, we don't care about these
binary blobs, there's just for tests...". It's even worse, because by
using a different version for tests and runtime, you're faking tests...

If the lib are just use for tests and nothing else (ie: not for
runtime), then back to square one: it's ok to ship the non-minified
version in your debian folder, and use that for running tests. It's also
super easy and fast to implement.

> I think blindly applying a rule without thinking of any consequences is bad too.

I think blindly saying "oh, it's ok, it's only test things..." is a
*very* dangerous path that I would like Debian to avoid.

> Just because it is bad in one situation does not mean it will be bad in every situation. We should evaluate pros and cons of each situation before making a decision. Blind faith is more suitable for religions and not for a project like ours.

Sorry, but using free software from source is *NOT* opened for debate.
If you would like to do that, choose another distribution. We all
signed-up for it, when becoming DDs, this is the foundations of Debian.

> I think a nocheck build profile which excludes these files from build is sufficient to ensure we are not using these to create binary package.

What's the problem with using a non-minified version of the files? It's
not difficult, and it doesn't take too much of your packaging time.

> This way we guarantee only packages in main is used to generate the binary, but still allows to run tests optionally making it easy to find problems, especially during transitions. Currently when tests are missing transitions are harder because we can't find breakages easily since tests are disabled.

What you're proposing is making the life of anyone willing to modify the
software harder.

> The current policy is not making Debian better.

It is. It's ensuring we build from source, and we're able to modify
what's in Debian with reasonable conditions.

Cheers,

Thomas Goirand (zigo)