Incoming Debian freeze

[Timo Paulssen]

Hi Dominique, and hello List,

Thanks for reaching out.

I've been poking at the freeze on amd64 occasionally over the last few 
weeks. Unfortunately, it apparently occurs only about once in every 
hundred runs or so. I caught it in the debugger but the state it's in 
when I hit the check doesn't explain how it could have reached the state 
in question. It remains mysterious and the way I usually deal with 
mysterious bugs like that is to run them under "rr record". 
Unfortunately, RR is currently x86_64 only. There is a way to run qemu 
(presumably qemu-system) under rr, but I haven't tried it yet. It might 
be quite slow, and there's no guarantee I will actually hit the error at 
all on an emulated system ...

Is it an option at all to skip the arches where we get freezes?

If I turn the freeze into a crash reliably (I can turn at least a 
fraction of the freezes into crashes, but I can't guarantee it's all) 
and we run the compilation steps of rakudo multiple times, would that be 
acceptable? Unfortunately, until I can verify the actual cause, it's 
probably also possible for these freezes (alternatively, crashes) to 
happen in regular user code.

It might be possible to react to the "it went wrong", correct the state 
we're in, and try again. My current theory is that we're not doing 
memory barriering properly for some inter-thread communication, so that 
one core reads bogus data for a little bit in a piece of code that 
essentially acts like a tiny interpreter. I have not yet investigated 
how exactly cache coherency and such differs from x86_64 and arm64 (and 
arm32, the other arch where build machines encounter freezes). Maybe 
that's the ticket.

I have also not yet had the time to check what exactly the current 
problem with FTBFS from the "outdated or wrong version of dependency" 
issue is.

My last status was that after the transition, everything should have 
been fixed.
Did we perhaps forget to kick off rebuild attempts?
Or did I need to mark all the relevant FTBFS bugs as "resolved"?

Maybe it would be as simple as that. It kind of feels like that last one 
is probably what's gone wrong.

Thanks again for holding my hand through all the processes.

Kind Regards,
   - Timo

On 1/11/25 12:15, Dominique Dumont wrote:
> Hello Timo
>
> Even if it's not yet announced, Debian should begin to freeze in a few weeks
> in preparation for next Debian release (usually in summer).
>
> Unfortunately, most rakudo packages were auto-removed from testing, which
> means they won't be shipped with Debian 13 unless they are fixed.
>
> As far as I understand, these packages are blocked for 2 problems:
> - rakudo build hangs in some arches (#1086555)
> - unpredictable pre-compilation results which leads to package conflicts
>
> I don't know what can be done for the former issue.
>
> For the latter, we could revert back to shipping only raku source in raku
> module (like perl5) and let raku do the pre-compilation at run time.
>
> The main drawback is that user will have to deal to raku pre-compilation cache
> even if user are not aware that such a cache exists.
>
> When pre-compilation issues are solved, we will be able to switch back to
> shipping pre-compiled files.
>
> Note that I do not wish to go back to the solution of doing pre-compilation at
> installation time. This is a waste of time since pre-compilation is not
> reproducible: the pre-compilation done at installation time has a good change
> of being ignored.
>
> Thoughts ?
>
> All the best
>
> Dominique
>
>