[Pkg-haskell-maintainers] Bug#624389: Bug#624389: System.Cmd.Utils.pipeBoth subject to pipeline stalls with ghc 7

[John Goerzen]

Hi folks,

I haven't yet had the chance to look at the MissingH functions on 
Windows.  My general impression is that the POSIX forking/etc. functions 
are on their way out of working well in Haskell.  It is possible that 
some of these things out to be removed from MissingH as the built-in 
System.Process is now more powerful.

-- John

On 05/14/2011 09:42 AM, Giovanni Mascellani wrote:
> tag 624389 + confirmed
> thanks
>
> Hi.
>
> Cc:-ing, John Goerzen, the upstream maintainer, and full-quoting for his
> convenience. John, do you have anything to comment here?
>
> I can confirm this bug (on amd64), but I don't know how to work on it.
>
> Giovanni.
>
> On 28/04/2011 04:50, Joey Hess wrote:
>> Package: ghc
>> Version: 7.0.3-1
>> Severity: normal
>>
>> For quite a while I have been using missingh's pipeBoth with success; but as
>> soon as my program was rebuilt with ghc 7, it started stalling when large
>> quantities of data needed to be passed through the pipe.
>>
>> Here is a simple test case. It needs to run in a git repository.
>>
>> import System.Cmd.Utils
>>
>> main = do
>>          as<- checkAttr "blah" $ map show [1..100000]
>>          sequence $ map (putStrLn . show) as
>>
>> checkAttr attr files = do
>>          (_, s)<- pipeBoth "git" params $ unlines files
>>          return $ lines s
>>          where
>>                  params = ["check-attr", attr, "--stdin"]
>>
>> It queries git for attribute values for 100000 files. With ghc 6, it
>> should run to completion. With ghc 7, it stalls, deadlocked, after
>> a varying number of files, under 1000:
>>
>> select(2, [], [1], NULL, {0, 0})        = 1 (out [1], left {0, 0})
>> write(1, "\"701: blah: unspecified\"\n", 25"701: blah: unspecified") = 25
>> select(2, [], [1], NULL, {0, 0})        = 1 (out [1], left {0, 0})
>> write(1, "\"702: blah: unspecified\"\n", 25"702: blah: unspecified") = 25
>> select(2, [], [1], NULL, {0, 0})        = 1 (out [1], left {0, 0})
>> write(1, "\"703: blah: unspecified\"\n", 25"703: blah: unspecified") = 25
>> select(2, [], [1], NULL, {0, 0})        = 1 (out [1], left {0, 0})
>> write(1, "\"704: blah: unspecified\"\n", 25"704: blah: unspecified") = 25
>> --- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
>> sigreturn()                             = ? (mask now [])
>> gettimeofday({1303958431, 174266}, NULL) = 0
>> select(7, [], [6], NULL, {0, 0})        = 1 (out [6], left {0, 0})
>> write(6, "345\n15346\n15347\n15348\n15349\n1535"..., 8096
>>
>> The program is blocked trying to write to git-check-attr, and
>> git-check-attr is in turn blocked waiting for its output to be read.
>>
>> I've skipping over missingh and filing this bug directly on ghc because
>> I think it's unlikely missingh is at fault. IIRC, pipeBoth works by
>> sparking off a helper thread, which is used to write input to a command.
>> Unless it made a bad assumption about that being a safe thing to do,
>> this must be a bug in GHC?
>>
>> FWIW, I have worked around this in my code by forking a process, not a
>> thread, to do the writing. Which works fine, just a little more
>> heavyweight than needed. I'm concerned about all the other potential
>> callers of pipeBoth out there, however.
>>
>> -- System Information:
>> Debian Release: wheezy/sid
>>    APT prefers unstable
>>    APT policy: (500, 'unstable')
>> Architecture: i386 (i686)
>>
>> Kernel: Linux 2.6.32-5-686 (SMP w/2 CPU cores)
>> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
>> Shell: /bin/sh linked to /bin/dash
>>
>> Versions of packages ghc depends on:
>> ii  gcc                       4:4.5.2-5      The GNU C compiler
>> ii  libbsd-dev                0.2.0-1        utility functions from BSD systems
>> ii  libc6                     2.11.2-11      Embedded GNU C Library: Shared lib
>> ii  libc6-dev                 2.11.2-11      Embedded GNU C Library: Developmen
>> ii  libffi-dev                3.0.9-4        Foreign Function Interface library
>> ii  libffi5                   3.0.9-4        Foreign Function Interface library
>> ii  libgmp-dev                2:5.0.1+dfsg-7 Multiprecision arithmetic library
>> ii  libgmp10                  2:5.0.1+dfsg-7 Multiprecision arithmetic library
>> ii  libncurses5               5.9-1          shared libraries for terminal hand
>>
>> ghc recommends no packages.
>>
>> Versions of packages ghc suggests:
>> ii  ghc-doc                       7.0.3-1    Documentation for the Glasgow Hask
>> ii  ghc-prof                      7.0.3-1    Profiling libraries for the Glasgo
>> pn  haskell-doc<none>      (no description available)
>> ii  perl                          5.10.1-20  Larry Wall's Practical Extraction
>>
>> -- no debconf information
>
>