[Pkg-javascript-devel] Bug#995722: Not running tests because tests miss source code is not useful

[Pirate Praveen]

On 7 October 2021 3:02:55 am IST, Thomas Goirand <zigo at debian.org> wrote:
>On 10/6/21 6:53 PM, Pirate Praveen wrote:
>> [adding -devel]
>> 
>> On ബു, ഒക്ടോ 6 2021 at 12:16:07 വൈകു +0200 +0200, Jonas Smedegaard
>> <jonas at jones.dk> wrote:
>>> Quoting Yadd (2021-10-06 11:43:40)
>>>>  On Lu, 04 oct 21, 16:40:48, Bastien Roucari�s wrote:
>>>>  > Source: src:node-lodash
>>>>  > Version: 4.17.21+dfsg+~cs8.31.173-1
>>>>  > Severity: serious
>>>>  > Justification: do not compile from source
>>>>  >
>>>>  > Dear Maintainer,
>>>>  >
>>>>  > The vendor directory should be emptied
>>>>  >
>>>>  > The debug version is compiled without source (lintian warn) and
>>>> moreover the
>>>>  > rest of file are already packaged
>>>>  >
>>>>  > grep -R vendor * gives only a few hit that could be cured by
>>>> symlinking
>>>>  >
>>>>  > Bastien
>>>>  Hi,
>>>>
>>>>  this files are used for test only, maybe severity could be decreased.
>>>
>>> I find the severity accurate: Relying on non-source code is a severe
>>> violation of Debian Policy, not matter the purpose of relying on it.
>> 
>> I think we should change the policy here. Running tests helps improve
>> the quality of the software we ship. Many times the vendored code is
>> used to ensure the code does not break in a specific situation. I don't
>> think reducing test coverage in such situations is really helpful.
>
>Right, running tests helps improve the quality of software we ship.
>Which is why you probably need to test using what's shipped in Debian
>rather than using a vendored source-less code.

We are not shipping the source less code. This is used only during tests. I don't think we are not gaining anything by removing tests here. Just making it harder for the package maintainer to run tests.

>If we rely on non-free code for tests, that's really bad too, and that
>must be avoided just like we're avoiding source-less code everywhere
>else in Debian. The policy shall not change, please.
>

The code is not non-free here, just a specific version of a Free Software code built outside Debian.

I think tools required for tests should be considered separately from tools required to compile. I think it should be treated similar to test data.

What you are proposing would require the package maintainer to adapt these tests to versions available (many times with different API versions) in Debian and the easier choice is disabling tests.

I think blindly applying a rule without thinking of any consequences is bad too. Just because it is bad in one situation does not mean it will be bad in every situation. We should evaluate pros and cons of each situation before making a decision. Blind faith is more suitable for religions and not for a project like ours.

I think a nocheck build profile which excludes these files from build is sufficient to ensure we are not using these to create binary package. This way we guarantee only packages in main is used to generate the binary, but still allows to run tests optionally making it easy to find problems, especially during transitions. Currently when tests are missing transitions are harder because we can't find breakages easily since tests are disabled.

The current policy is not making Debian better.
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.