[pkg-cryptsetup-devel] Bug#1032734: OOM when unlocking encrypted root in initramfs

[Guilhem Moulin]

Control: tag -1 - moreinfo
Control: severity -1 important
Control: retitle -1 Argon2 memory cost is not future proof and might OOM on dist-upgrade on memory-constrained systems

On Sat, 11 Mar 2023 at 14:53:37 -0500, Jérôme Charaoui wrote:
>> Jérôme, what memory cost is the keyslot using?  (Paste the output of
>> `cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF:`.)
>
> ----- 8< -----
> # cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF:
>     PBKDF:      argon2i
>     Time cost:  4
>     Memory:     787907
>     Threads:    1
> ----- >8 -----
> […]
> So, I think what's happening is that the first VM may have been created with
> a different (larger) memory configuration, and was reduced at a later point
> in time. I don't have absolute certainty of this, but it would very well
> explain the discrepancy in memory cost.

I'm very sure it's the case: buster, bullseye, bookworm (and everything
in between) never set the memory cost to more than half the physical
memory.  So it's just not possible to end up with such a high memory
cost on a machine with only 1GiB RAM.  Memory-hard KDF parameters are
non portable and this is a feature not a bug :-)  Upon hardware change
one needs to run the benchmark again via cryptsetup-luksChangeKey(8) or
similar to tune the parameters to the new system; and downgrading
hardware needs to be done with care as folks who bootstrap images for
RPI-like boards are surely aware.

It's a coincidence that you triggered the OOM-killer only after the post
dist-upgrade reboot, you were already likely close to memory exhaustion
after reducing memory.

> I there any way we could make the cryptsetup-initramfs hook aware of this,
> and emit a warning if it finds that the encrypted root lacks a keyslot with
> appropriate (low-enough) memory cost?

I don't think the hook is the place for that: 1/ it might not have
access to the header where this information resides, 2/ it doesn't know
beforehand which keyslot will be used, and 3/ the issue is not specific
to cryptsetup-initramfs.  There is an upstream commit on the main branch
that adds a warning to libcryptsetup, might cherry-pick that to bookworm
instead.

Anyway, lowering this to sub-RC now that it's demystified.  In your case
the root issue (KDF parameter portability) is wontfix/notabug, but I'm
hijacking this to point out that KDF parameters are not future proof.
(This is what the forwarded upstream bug points to, and what I initially
thought you might be experiencing.)  Things work fine out of the box on
minimal systems (also with <1GiB RAM), but several releases down the
road we might ship a kernel or early boot daemons requiring a lot more
memory, and the KDF *will* exhaust memory at that point.  The upstream
fix in !490 (neither in bookworm nor sid yet) improves things a bit but
really is only buying time.  Milan suggested that systems with little
RAM are probably better off using a non-memory-hard KDF.  Perhaps
upstream could be convinced to have different defaults depending on the
amount of physical memory, if not then perhaps it could be done in
partman-crypto (I personally wouldn't feel comfortable carrying such a
patch in src:cryptsetup or have defaults that are unaligned with
upstream or other distros).  Won't help with existing keyslots though.

-- 
Guilhem.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-cryptsetup-devel/attachments/20230311/dcc4eca3/attachment.sig>