[Piuparts-devel] the chdir story (was: Re: pull: five-holger)

[Andreas Beckmann]

On 2013-04-13 23:19, Dave Steele wrote:
> Dave Steele(1):
>      Remove need for chdir() before loading packagesdb.
> 
> Cleanup in prep for a reworked faster-rdep-calc/rdep-chain-len.
> 
> I've run the affected scripts, without failure.

Thanks.

But then again, what is the actual problem to be solved here?

The directory needs to be correct when the internal LogDB is being used.
Usually that is only while _compute_package_states() is run. That is
currently run implicitly upon the first access of some sort to the states.
But only the master really uses this "lazy computation" since it is
expensive and is actually only needed when reserve() gets called. The
master also accesses LogDB when it receives logs.
-report will always need the state computation (and does not need LogDB
beyond this)
dwke part 1 (generate .kpr) does not need a packagesdb at all.
dwke part 2 (generate .tpl) does need this in a similar way to -report.

What's a sane working directory for the different applications anyway?

* slave creates/uses stuff in the directory where it was started. This
is needed to allow multiple slaves and it is the task of the slave
starter to chdir properly.

* master does chdir(master-directory) at the very beginning

* report and dwke are a bit more complicated since they need multiple
"working" directories and are running over all sections, so they might
chdir per section. Furthermore there are several different directories
needed (master, output, known-problems) and for some reason it seems to
be allowed that these are to be specified *relative* to piupartsm's home
directory. Did someone ever try whether that is actually working?

If we could chdir to a sane base working directory in -report (and dwke)
at the very beginning - the master directory looks like the best
candidate - we could avoid both repeated chdiring to master-directory
and passing master-directory along. Of course that must not break the
output-directory, so it may need to be made absolute at the very
beginning ...

What's the memory footprint of a python dictionary with 50000 entries
and keys that are strings of on average 30 characters. How does this
change if the keys are enlarged to be 100 chars on average by prepending
a constant string of lenght 70?
At least I was a bit concerned that that could blow up memory usage
(especially with 20+ instead of 1 such dictinaries)

Andreas