[From nobody Thu Jul 23 00:51:05 2026 Received: (at maintonly) by bugs.debian.org; 30 Sep 2020 21:03:45 +0000 X-Spam-Checker-Version: SpamAssassin 3.4.2-bugs.debian.org_2005_01_02 (2018-09-13) on buxtehude.debian.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=4.0 tests=BAYES_00,DIGITS_LETTERS, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, MD5_SHA1_SUM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.2-bugs.debian.org_2005_01_02 X-Spam-Bayes: score:0.0000 Tokens: new, 9; hammy, 150; neutral, 123; spammy, 0. spammytokens: hammytokens:0.000-+--autopkgtest, 0.000-+--H*UA:NeoMutt, 0.000-+--H*u:NeoMutt, 0.000-+--sk:README, 0.000-+--regressions Return-path: <sudipm.mukherjee@gmail.com> Received: from mail-wr1-x42a.google.com ([2a00:1450:4864:20::42a]:38868) by buxtehude.debian.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from <sudipm.mukherjee@gmail.com>) id 1kNjGL-0007cz-6q for maintonly@bugs.debian.org; Wed, 30 Sep 2020 21:03:45 +0000 Received: by mail-wr1-x42a.google.com with SMTP id g4so3309517wrs.5 for <maintonly@bugs.debian.org>; Wed, 30 Sep 2020 14:03:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:subject:message-id:mime-version:content-disposition :user-agent; bh=WeevOKhfEd/c0EaFW4Esg5RD2f94PLIPYWAKbvxoYEI=; b=A07qg7bVoaccG3bL6SFvzVsOAdcG8HAZSuQ4yF/7qbnLzCLb3hpTuP3ub4NB7h1Jfn c3JCRs8wcaTm5FX15jkhElZqAzQRsluJREONpzxg6YZ2c8OqJvYe/4GHQIZq6AAJca2N byhWb8LX7S2Cq0CaYU6v446g6Jg27joKLnC2z5kBegQQvwFHOqQLW1FPnh6/pSXqDVJL TudJPnCAHsjVmNtKiOtlFH/DtW7vLNJS25ZUAbXG+E2Cu+KEmEbDsR1Qa9nIkM529NPx x9Ai+z+F/XSUU2aHh4AqX0E6bPRwjGwcUQVcFwLquoynO8bbd8o2qlKj+NFMwix/WEim iNtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-disposition:user-agent; bh=WeevOKhfEd/c0EaFW4Esg5RD2f94PLIPYWAKbvxoYEI=; b=EpFTJ6MtRiVLpwYgr/oY44hm5ynvGVbH432iiMUGmVCOcLFYp6lsrZ2QBqOCMwkjPi zWO2jg1VpjLWd8TBHXPIrW3a2PjzXRuF7GgO+a58jJllvlh4rUrXc1vW5ZTreWwvxSQa TqBaLOjHOf4HoSt7xdl6LksWPo7TuUOdjqTAGLpMJoh8ysvQfNKC4LUWA5gieA/uaoMp 3J1+Q7SWA9mjSWEi+4n9YDNcETEK9UjIq7zeVVQriu9xODItVTjrcmd2ID36GtMKmE92 MLC74OkQbEFlQ7AaHDzQgV1+a12qxV/DZYQj4xvphzbmqXBRoLkZrNWdQsF5WY4tpF+H JGCg== X-Gm-Message-State: AOAM533lCMOsFEPiVNHUv+ZCkJQWqxf9lUGmGcIRxNa/8G+jisPw0iXH BM9++1tqMLmB+fONlGmUp+K3WuO9De0HmQ== X-Google-Smtp-Source: ABdhPJxAqJJREeuKbVoNpspZdh/0WgpNq4Hdl2RtIBh1G/Psxvs/JkHJs1BHMOgccB8oZieqvWk6YA== X-Received: by 2002:a5d:55c8:: with SMTP id i8mr5108208wrw.331.1601499822279; Wed, 30 Sep 2020 14:03:42 -0700 (PDT) Received: from debian (host-92-5-241-147.as43234.net. [92.5.241.147]) by smtp.gmail.com with ESMTPSA id s11sm4783957wrt.43.2020.09.30.14.03.41 for <maintonly@bugs.debian.org> (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 30 Sep 2020 14:03:41 -0700 (PDT) Date: Wed, 30 Sep 2020 22:03:40 +0100 From: Sudip Mukherjee <sudipm.mukherjee@gmail.com> To: maintonly@bugs.debian.org Subject: haskell-hopenpgp-tools: autopkgtest must be marked superficial Message-ID: <20200930210340.jvq7hv5ifoliye25@debian> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Generator: mass-bug from devscripts 2.19.5+deb10u1 User-Agent: NeoMutt/20170113 (1.7.2) Delivered-To: maintonly@bugs.debian.org Source: haskell-hopenpgp-tools Severity: important Usertags: superficialtest X-Debbugs-CC: elbrus@debian.org Dear Maintainer, It has been noticed that the autopkgtest in haskell-hopenpgp-tools is running a trivial command that does not provide significant test coverage: - testcommand: - hkt --version - hokey --version - hop --version - hot --version Executing that command is considered to be a trivial test, which does not provide significant coverage for a package as a whole. But these tests are a useful way to detect regressions in dependencies and prevent them from breaking your package. However, it is important that we are realistic about the level of test coverage provided by these commands: most regressions cannot be detected in this way. So it is not appropriate for packages with only superficial tests to have the reduced migration time to migrate from unstable to testing as that means less opportunity for testing by users compared to the package with no tests. To support this, the keyword "Restrictions: superficial" has been defined [1]. Packages where all tests are marked with this keyword are not considered for the reduced migration age from unstable to testing, and will not be allowed to migrate automatically in later stages of the freeze [2]. Its always better to have more extensive testing than having superficial testing, which again is better than having no test. Please consider i) Adding a non-trivial test, and/or ii) Mark the trivial test with "Restrictions: superficial", similar to [3] or [4]. The Release Team has listed this issue in the list of Release Critical Issues for bullseye [5] and has mentioned that the test must be marked superficial if it is not testing one of its own installed binary packages in some way. As a result, the severity of this bug report might be increased to serious in future. [1] https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst#defined-restrictions [2] https://release.debian.org/bullseye/freeze_policy.html [3] https://salsa.debian.org/utopia-team/dbus/-/commit/a80908df7d119b181eec5eb0542634a30c2ad468 [4] https://salsa.debian.org/apparmor-team/apparmor/-/commit/580667513a097088ebe579884b38ac8d8666d3b3 [5] https://release.debian.org/bullseye/rc_policy.txt -- Regards Sudip ]