[Piuparts-devel] LogDB refactoring

[Andreas Beckmann]

On 2013-05-25 00:46, Dave Steele wrote:
> 80e8414 packagesdb - Remove obsolete states from PackagesDB.

NACK, see this comment on top of _states:
    # FIXME: can we reorder this list or remove entries without breaking the counts.txt for the plot?

> 0a4c208 packagesdb - Remove get_time().

That is not unused:
./piupartslib/packagesdb.py:366:    def get_mtime(self):
./piuparts-master-backend.py:249:        if stamp_mtime < self._binary_db.get_mtime():


style: Let's try to follow PEP8 if possible, so please 
       *no* space in parenthesis: f( NONONO )  but  f(YESYESYES)


Currently creating a logfile takes a constant number of I/Os,
I'm not fond of making this a linear in n
a slave submits typically 50 logs and does *not* compute the states,
but switches to a different section and computes the state there
(state computation requires knowledge about all logfiles, therefore
bulk loading is advised)
And doing a full logdb initialization for a packagedb creation is
clearly a no-go: upon slave connection we are interested in two things:
age of stampfile and mtime of all subdirs - that is sufficient to decide in
constant I/Os whether a section is idle

Ideally there should only be one instance of a log in 
pass/fail/bugged/affected/untestable/reserved, maybe that can be enforced

But for recycle, there is usually a real log somewhere else.
recycle only contains a flag file, and that is useless if there is no real
log that can be recycled (outdated ones get cleaned up in reschedule_old_logs) 
using a hardlink of the existing log is cheap because it does not need an
extra inode (touching an------that empty file instead would need another inode)


For upgrade tests, the current solution of ${pkg}_${ver}.log is insufficient
to describe the full upgrade path: currently $ver is taken from the last
distribution (which is more frequently updated than older distros, but not the
only trigger). I would like to chnage this to have the explicit sequence
in the logfile name, although I'm not sure how to actually do this, e.g.
  ${pkg}_${ver1}_${ver2}_${ver3}.log
That should simplify rerunning the test whenever one of the versions changes.
And request an explicit upgrade path from the slave.
As a special value the string "None" is used to indicate that the package does
not exist in a distro.
Primary use case: A2bpo2B.


Andreas