[Piuparts-devel] Fwd: Re: RFC: feature/adequate-tags

[Andreas Beckmann]

On 2013-05-30 16:27, Holger Levsen wrote:
> Hi Andreas,
> 
> thanks for the review!
> 
> On Donnerstag, 30. Mai 2013, Andreas Beckmann wrote:
>> If I understand correctly, adequate produces "non-inheriting" errors (by
>> testing only the requested set of packages), so there is no need to fail
>> the test (and block other packages from being tested), given we properly
>> report these as errors - and your branch will generate nice individual
>> reports for pass/ logs.
> 
> Jup. That's why there is --fail-if-inadequate if you really want piuparts to 
> fail if "only" adequate fails. 

But we probably won't ever need --fail-if-inadequate (for master-slave support)
because there is nothing to gain by enabling this.

We have three possible outcomes of a piuparts test:
* passed
* failed, but dependencies can be tested
* failed, don't test dependencies

adequate targets the second group, and right now we report them as "pass"
and add multiple reports to list them and allow bug reporting

>> This does not work:
>>
>> -    ("inadequate_error.tpl", "due to logfile containing inadequate
>> errors"),
>> +    ("inadequate_exit_issue.tpl", "due to adequate exiting inadequately"),
>>
>> IIRC the _error.tpl vs. _issue.tpl suffix is currently a significant
>> property used in piuparts-report.
> 
> Yup, I've changed this to exit_issue.tpl. (Re-fetch.)
???

create_and_link_to_analysises has

            # sucessful logs only have issues and failed logs only have errors
            if (state == "failed-testing" and template[-9:] != "issue.tpl") \
                or (state == "successfully-tested" and template[-9:] == "issue.tpl"):

>> So maybe just restore inadequate_error.tpl for the time being to have a
>> separate combined report of all adequate output in fail/ and friends -
>> the tests din't fail because of adequate anyway so splitting them up for
>> fail/ is not really helpful.
> 
> I think the current/latest branch (fc4207e) is ok.

I don't think so. There won't be a report for fail/.


Andreas

PS: if adequate can be used for th emissing copyright check, we could consider
dropping it from scripst/