[Pkg-haskell-maintainers] Bug#624389: System.Cmd.Utils.pipeBoth subject to pipeline stalls with ghc 7

[Joey Hess]

Package: ghc
Version: 7.0.3-1
Severity: normal

For quite a while I have been using missingh's pipeBoth with success; but as
soon as my program was rebuilt with ghc 7, it started stalling when large
quantities of data needed to be passed through the pipe.

Here is a simple test case. It needs to run in a git repository.

import System.Cmd.Utils

main = do
        as <- checkAttr "blah" $ map show [1..100000]
        sequence $ map (putStrLn . show) as

checkAttr attr files = do
        (_, s) <- pipeBoth "git" params $ unlines files
        return $ lines s
        where
                params = ["check-attr", attr, "--stdin"]

It queries git for attribute values for 100000 files. With ghc 6, it
should run to completion. With ghc 7, it stalls, deadlocked, after
a varying number of files, under 1000:

select(2, [], [1], NULL, {0, 0})        = 1 (out [1], left {0, 0})
write(1, "\"701: blah: unspecified\"\n", 25"701: blah: unspecified") = 25
select(2, [], [1], NULL, {0, 0})        = 1 (out [1], left {0, 0})
write(1, "\"702: blah: unspecified\"\n", 25"702: blah: unspecified") = 25
select(2, [], [1], NULL, {0, 0})        = 1 (out [1], left {0, 0})
write(1, "\"703: blah: unspecified\"\n", 25"703: blah: unspecified") = 25
select(2, [], [1], NULL, {0, 0})        = 1 (out [1], left {0, 0})
write(1, "\"704: blah: unspecified\"\n", 25"704: blah: unspecified") = 25
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
sigreturn()                             = ? (mask now [])
gettimeofday({1303958431, 174266}, NULL) = 0
select(7, [], [6], NULL, {0, 0})        = 1 (out [6], left {0, 0})
write(6, "345\n15346\n15347\n15348\n15349\n1535"..., 8096

The program is blocked trying to write to git-check-attr, and
git-check-attr is in turn blocked waiting for its output to be read.

I've skipping over missingh and filing this bug directly on ghc because
I think it's unlikely missingh is at fault. IIRC, pipeBoth works by
sparking off a helper thread, which is used to write input to a command.
Unless it made a bad assumption about that being a safe thing to do,
this must be a bug in GHC?

FWIW, I have worked around this in my code by forking a process, not a
thread, to do the writing. Which works fine, just a little more
heavyweight than needed. I'm concerned about all the other potential
callers of pipeBoth out there, however.

-- System Information:
Debian Release: wheezy/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: i386 (i686)

Kernel: Linux 2.6.32-5-686 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages ghc depends on:
ii  gcc                       4:4.5.2-5      The GNU C compiler
ii  libbsd-dev                0.2.0-1        utility functions from BSD systems
ii  libc6                     2.11.2-11      Embedded GNU C Library: Shared lib
ii  libc6-dev                 2.11.2-11      Embedded GNU C Library: Developmen
ii  libffi-dev                3.0.9-4        Foreign Function Interface library
ii  libffi5                   3.0.9-4        Foreign Function Interface library
ii  libgmp-dev                2:5.0.1+dfsg-7 Multiprecision arithmetic library 
ii  libgmp10                  2:5.0.1+dfsg-7 Multiprecision arithmetic library
ii  libncurses5               5.9-1          shared libraries for terminal hand

ghc recommends no packages.

Versions of packages ghc suggests:
ii  ghc-doc                       7.0.3-1    Documentation for the Glasgow Hask
ii  ghc-prof                      7.0.3-1    Profiling libraries for the Glasgo
pn  haskell-doc                   <none>     (no description available)
ii  perl                          5.10.1-20  Larry Wall's Practical Extraction 

-- no debconf information

-- 
see shy jo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 828 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/pkg-haskell-maintainers/attachments/20110427/143f4335/attachment.pgp>