Bug#642020: python-debian: ArFile (via DebFile) errors when given only a fileobj

[Simon Ruggier]

On Tue, 26 Aug 2014 16:43:43 -0700 John Wright <jsw at debian.org> wrote:
> Sorry for the very delayed response.  I'm trying to clean up our
> outstanding bug list...
>
> I'm not convinced that sharing the file is the correct thing to do - it
> introduces thread-unsafe-ness between DebFile/ArFile and its dependent
> objects that isn't obvious up front.
>
> In addition, the existing behavior or re-opening the file by name is bad
> for another reason: it leaks open files.
>
> I think a better solution is probably something like:
>   - If the input file is a "real" file, use mmap to get a view of the
>     object for members.
>   - Otherwise, we should probably iterate over the entire input object
>     to generate our own file-like view of it, which itself supports an
>     interface similar to mmap, and use that for member objects.

Hi, thanks for tending to the bug tracker.

I just want to raise this question: does it make sense for this API to be
thread-safe? Threads are essentially a performance optimization, and
creating a DebFile object and passing it to multiple threads (so that they
can read from it in parallel) is surely not the most performant way to
retrieve information from package binaries. So it has neither the
convenience of single-threading, nor good performance characteristics, and
it's therefore somewhat unreasonable for callers to expect that use case
would be supported.

On the other hand, it's very reasonable to expect an API to support file
objects, and right now, the lack of support for this has motivated two
separate people to fix the problem in forks. I think it would make sense to
incorporate changes from one of the downstream forks to fix this bug, and
not worry about thread safety at all. Any callers that need to access the
information from multiple threads can use a single thread to retrieve the
information, then pass it between threads internally without calling the
API in a concurrent way.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-python-debian-maint/attachments/20160620/bb0431d7/attachment.html>