[From nobody Sat Mar 21 22:51:06 2026 Received: (at maintonly) by bugs.debian.org; 11 Nov 2020 18:06:05 +0000 X-Spam-Checker-Version: SpamAssassin 3.4.2-bugs.debian.org_2005_01_02 (2018-09-13) on buxtehude.debian.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=4.0 tests=BAYES_00,DIGITS_LETTERS, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, MD5_SHA1_SUM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.2-bugs.debian.org_2005_01_02 X-Spam-Bayes: score:0.0000 Tokens: new, 8; hammy, 150; neutral, 123; spammy, 0. spammytokens: hammytokens:0.000-+--autopkgtest, 0.000-+--sk:README, 0.000-+--sk:readme., 0.000-+--sk:README., 0.000-+--apparmor Return-path: <sudipm.mukherjee@gmail.com> Received: from mail-wm1-x331.google.com ([2a00:1450:4864:20::331]:54271) by buxtehude.debian.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from <sudipm.mukherjee@gmail.com>) id 1kcuVR-0005pl-F8 for maintonly@bugs.debian.org; Wed, 11 Nov 2020 18:06:05 +0000 Received: by mail-wm1-x331.google.com with SMTP id p22so3082372wmg.3 for <maintonly@bugs.debian.org>; Wed, 11 Nov 2020 10:06:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:message-id:subject:to; bh=ptqDwQSUJ+Ad/y6OD594HMiTF+wCUtCRHaL/2aTLqqQ=; b=ShyDWvBCML+WCu/ZmiaOio9ehmS6s6TfFCHMADxABV91vqKkyyZMV10woeen7Jz+gE LZ5M4hDCLndhBnCO7YTRog+4/Lz0t9mVFoSDJwqXGDq9qVt+bW0AH03nwuj6LA2a2UVN 4DRXmq6EliAUSisamlC06uz9ztP3TdjN8ZpC2vo0/Yp7DvGZkU85wAGlrdVEYPxCF/HA 50ey2egbaRbzS6UZkk4oZeQZ3IOprzX2IBwvZRZGhsf6Hzw6P9h4DLg8Cjyrwd9qnjAU o/MnZyfZ30+QqjVfmv30H17/cbaYMdukMlxqlrouy6qfheSnCglPDGwSZAcWryFkDz+Z ubCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:message-id:subject:to; bh=ptqDwQSUJ+Ad/y6OD594HMiTF+wCUtCRHaL/2aTLqqQ=; b=oaeP/zoANjx//sKqqXA0FSHcGkhrcgxRYyLThI8oPhexD5OyY32xBgoEkyyAhpqgbP l3iSFXXivA0IY6U18ganTXPf2R0w4DISkrv4KaUPeYyA9T5aykSwTd0Mr49kDiWP9Rp5 zlZmSglWphNUNzI2JBBlFfSrnzM6C9nCY90ev6SPb3EV4n4A7xGx0jYq4wdQutSFDdpR kkShkNQMOOYTu7Om3kqRTP8LApE2ji9Ws1QvY7bju9l0HUIybhpjQ3UBGKZxIG9XU+3h AU41GceGbQTJ4WTzKryQBt3vH+Rygz7zWHElGy1u9iJzlaNqqGwQJTaMYTsSQA+HqETw FYHQ== X-Gm-Message-State: AOAM5334A6Ni7zMAGc+Au5X7fvkLxteQv7DmHA4b/MYgZkgk0Zka7PB0 DCvBLFPjaUIzqGekLya2T/eEwJO9h82wMA== X-Google-Smtp-Source: ABdhPJwIJ6681DR3sdmOjuWwza+sl+2nI+tx/JJwomKKJ/xWBDhXBxAWn9TqJPsfr92byOgGJPWewA== X-Received: by 2002:a1c:21c1:: with SMTP id h184mr5771921wmh.106.1605117963536; Wed, 11 Nov 2020 10:06:03 -0800 (PST) Received: from debian.vm (host-92-5-241-147.as43234.net. [92.5.241.147]) by smtp.gmail.com with ESMTPSA id 15sm3346583wmg.1.2020.11.11.10.06.02 for <maintonly@bugs.debian.org> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Nov 2020 10:06:02 -0800 (PST) Date: Wed, 11 Nov 2020 10:06:02 -0800 (PST) X-Google-Original-Date: Wed, 11 Nov 2020 18:02:16 GMT From: Sudip Mukherjee <sudipm.mukherjee@gmail.com> X-Google-Original-From: Sudip Mukherjee <sudip@debian.vm> Received: from debian.vm (localhost [127.0.0.1]) by debian.vm (8.15.2/8.15.2/Debian-20) with ESMTP id 0ABI2GYQ006697 for <maintonly@bugs.debian.org>; Wed, 11 Nov 2020 18:02:16 GMT Received: (from sudip@localhost) by debian.vm (8.15.2/8.15.2/Submit) id 0ABI2Gq9006696; Wed, 11 Nov 2020 18:02:16 GMT Message-Id: <202011111802.0ABI2Gq9006696@debian.vm> Subject: salt-pepper: autopkgtest must be marked superficial To: <maintonly@bugs.debian.org> X-Mailer: mail (GNU Mailutils 3.10) Delivered-To: maintonly@bugs.debian.org Source: salt-pepper X-Debbugs-CC: elbrus@debian.org Severity: important Usertags: superficialtest Dear Maintainer, It has been noticed that the autopkgtest in salt-pepper is running a trivial command that does not provide significant test coverage: - $py -c "import pepper; print(pepper)" and salt-pepper --help Executing that command is considered to be a trivial test, which does not provide significant coverage for a package as a whole. But these tests are a useful way to detect regressions in dependencies and prevent them from breaking your package. However, it is important that we are realistic about the level of test coverage provided by these commands: most regressions cannot be detected in this way. So it is not appropriate for packages with only superficial tests to have the reduced migration time to migrate from unstable to testing as that means less opportunity for testing by users compared to the package with no tests. To support this, the keyword "Restrictions: superficial" has been defined [1]. Packages where all tests are marked with this keyword are not considered for the reduced migration age from unstable to testing, and will not be allowed to migrate automatically in later stages of the freeze [2]. Its always better to have more extensive testing than having superficial testing, which again is better than having no test. Please consider i) Adding a non-trivial test, and/or ii) Mark the trivial test with "Restrictions: superficial", similar to [3] or [4]. The Release Team has listed this issue in the list of Release Critical Issues for bullseye [5] and has mentioned that the test must be marked superficial if it is not testing one of its own installed binary packages in some way. As a result, the severity of this bug report might be increased to serious in future. [1] https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst#defined-restrictions [2] https://release.debian.org/bullseye/freeze_policy.html [3] https://salsa.debian.org/utopia-team/dbus/-/commit/a80908df7d119b181eec5eb0542634a30c2ad468 [4] https://salsa.debian.org/apparmor-team/apparmor/-/commit/580667513a097088ebe579884b38ac8d8666d3b3 [5] https://release.debian.org/bullseye/rc_policy.txt -- Regards Sudip ]