Bug#900173: git-annex: internal error: evacuate: strange closure type 4325404

Sun May 27 21:21:09 BST 2018

On 05/27/2018 01:20 PM, Joey Hess wrote:
> One person reported the same error message 7 years ago here:
> https://ghc.haskell.org/trac/ghc/ticket/5085
> They were using git-annex get, not the assistant when it crashed.
>
> They also were able to git bisect git-annex's code and found an utterly
> innocuous commit that triggered whatever the problem is.
> (commit 828a84ba3341d4b7a84292d8b9002a8095dd2382)
>
> It's probably a memory problem, or a ghc bug, or a bug in some library
> that is doing something memory related and messes up, such that ghc's
> garbage collector sees bad data.

First off, thank you for git-annex. It's really useful software. I can't 
imagine syncing all this data between several desktops, laptops, & a few 
tablets without it. And apologies in advance for this rambling message.

Just finished 2, almost 3 passes of memtest86+ with no errors. (Before 
starting using them machine, several years ago, it had over 24h of 
memtest). It's been stable, haven't seen any random 
crashes/corruption/etc. — so I doubt it's a hardware issue. It's also 
not overclocked or anything silly like that.

I'm not sure what bisecting this would entail; as far as I can tell... 
it's random.

(BTW: I use the CLI interface too, quite a lot, and have never seen a 
weird error from it. Only from the assistant).

Freshly after booting, I ran "git annex assistant"; it gave one of those 
errors in the log. So I ran "git annex assistant --debug"; on the 
console it gave:

[2018-05-27 15:56:53.403932801] read: uname ["-o"]
[2018-05-27 15:56:53.405388769] process done ExitSuccess
[2018-05-27 15:56:53.502601514] logging to .git/annex/daemon.log
[2018-05-27 15:56:53.503385967] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
[2018-05-27 15:56:53.505100304] process done ExitSuccess
[2018-05-27 15:56:53.505256854] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
[2018-05-27 15:56:53.50805146] process done ExitSuccess
[2018-05-27 15:56:53.517155935] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
[2018-05-27 15:56:53.517579777] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
[2018-05-27 15:56:53.522956836] logging to .git/annex/daemon.log

and in the log,

[2018-05-27 15:56:40.385313411] main: starting assistant version 6.20180509
[2018-05-27 15:56:40.39050583] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
[2018-05-27 15:56:40.393284694] process done ExitSuccess
[2018-05-27 15:56:40.393397832] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
[2018-05-27 15:56:40.395763112] process done ExitSuccess
[2018-05-27 15:56:40.396781131] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
git-annex: internal error: evacuate: strange closure type 4325407
     (GHC version 8.2.2 for x86_64_unknown_linux)
     Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

I ran the same command again; got the same (promising!). Then I added 
"--foreground" hoping to get some more debug outout... and isntead, it 
decided to work. The working log is (of course) longer, it looks like:

[2018-05-27 15:57:10.648904305] main: starting assistant version 6.20180509
[2018-05-27 15:57:10.652259294] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
[2018-05-27 15:57:10.712408937] process done ExitSuccess
[2018-05-27 15:57:10.712560137] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
[2018-05-27 15:57:10.715542944] process done ExitSuccess
[2018-05-27 15:57:10.716916651] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
[2018-05-27 15:57:10.717421932] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
[2018-05-27 15:57:10.843564188] Cronner: waiting Seconds {fromSeconds = 43369} for next scheduled fsck self 15m every day at 4 AM
[2018-05-27 15:57:10.958325684] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--stage","-z","--","."]
[2018-05-27 15:57:11.034783419] process done ExitSuccess
[2018-05-27 15:57:11.034973766] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--stage","-z","--","."]
[2018-05-27 15:57:11.162696] process done ExitSuccess
[2018-05-27 15:57:11.250761052] chat: nice ["ionice","-c3","nocache","/usr/bin/git-annex","remotedaemon","--foreground"]
[2018-05-27 15:57:11.251914205] TransferScanner: Syncing with zia, einstein
[2018-05-27 15:57:11.3242642] TransferWatcher: watching for transfers
[2018-05-27 15:57:11.324391068] MountWatcher: Using running DBUS service org.freedesktop.UDisks2 to monitor mount events.
[2018-05-27 15:57:11.412251164] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
[2018-05-27 15:57:11.413629769] process done ExitSuccess
[2018-05-27 15:57:11.413713615] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
[2018-05-27 15:57:11.415069811] process done ExitSuccess

... and continues for many more lines. Stopping it, "git annex 
assistant" and "git annex assistant --debug" both fail with the same 
error, but "git annex assistant --foreground" is OK.

So right now, it's just refusing the run in the background :-/

Willing to try any reasonable (and some not so reasonable) 
troubleshooting/debugging steps.

PS: There is another error I sometimes get from the assistant, typically 
after it's been running for a while: "fd:51: hPutStr: illegal operation 
(handle is closed)" (I've seen 38 as well). No idea if that's related or 
something else entirely. It shows up in the log, and sometimes is 
accompanied by files being left added but not committed.