[Piuparts-devel] Bug#748796: Bug#748796: piuparts: should not use dist-upgrade to check for a single package upgrade

[Holger Levsen]

control: severity -1 normal
control: tags -1 + moreinfo

Hi Yann,

thanks for taking the time to file a bug against piuparts. Even though I 
basically dismiss it, I do appreciate the feedback. (And I have changed my 
mind in the past too :)

On Dienstag, 20. Mai 2014, Yann Dirson wrote:
> http://bugs.debian.org/726799 shows that, while using dist-upgrade may
> be useful and can reveal problems, it does not test *just* the package
> upgrade it claims to be testing (sounds obvious, but well... ;).

Yes, it's obvious and I'd also argue it's obviously the right thing to do. So 
in fact I'm also considering to just close your bug (as not a bug, but design) 
and/or make it wishlist and close it then.

The usual way to upgrade the system is precisely to use "apt-get upgrade" or 
"apt-get dist-upgrade" and _not_ to upgrade a single package with "apt-get 
install $package/$version" - that's not a common real world use case.

(Plus you seem to be having used aptitude in 726799, which IME not always 
produces the same results as apt-get.)
 
> In this case, a number of hours have been wasted hunting for a
> seemingly-unreproducible bug, that was in fact perfectly reproducible,
> but just wrongly characterized.

This is unfortunate but this happens. This would also happen sometimes if 
piuparts would test things differently.

> * we surely need a better test procedure for package ugrades
>   => what's wrong with just "apt-get install $PACKAGE/sid" ?

See above. Also, (I believe) you speak from the perspective of one bug in one 
package - I've watched ten thousands of logfiles so far. (The whole setup has 
roughly a quarter million current logfiles and for the vast majority the tests 
are fine.)
 
> * we do need a test procedure that would reproducibly find this kind
>   of bugs

we (=Debian, not just piuparts) surely can use more tests. Check the wishlist 
bugs against piuparts to learn about some more test cases (ie testing with 
/usr/local readonly) - and as said, those are just piuparts related ones.

>   What I'm thinking of is something like a tool that would start with
>   a large installation of packages from testing, and which would
>   test-upgrade each one of those packages separately.

Interesting idea. Though, IME nobody does this, I mean: no user in the real 
world upgrades her system like this, so whats the point?

OTOH I have implemented other upgrade tests, see 
https://jenkins.debian.net/view/chroot-installation

>   Similarly, triggers could be tested from a similar setup (both for
>   testing and sid).

As always: patches welcome. I think such tests would be more suited on 
jenkins.debian.net though.

>   (and probably a ton of other tests like that)

Yes, sure. https://jenkins.debian.net/userContent/todo.html is also quite 
long.


Hm, after writing this mail I indeed think about 
http://blog.liw.fi/posts/wishlist-bugs/ and closing this bug. Because, I 
cannot come up with a bug title which is sensible to me. Except maybe 
"piuparts should have an option to just upgrade a single package and not the 
system" but then I disagree thats a useful option/test... (though if someone 
sends a patch I'd probably take it. I don't disagree about the feature, even 
though I disagree with using it for piuparts.debian.org.)

Further feedback welcome.


cheers,
	Holger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.alioth.debian.org/pipermail/piuparts-devel/attachments/20140521/99d22ad7/attachment.sig>