[pkg-cryptsetup-devel] Bug#1052290: cryptsetup-initramfs: askpass is not executed; cryptroot-unlock fails

[Tj]

On 20/09/2023 08:35, Guilhem Moulin wrote:
> Control: tag -1 moreinfo
> 
> On Tue, 19 Sep 2023 at 22:39:40 +0100, Tj wrote:
>> Error: Timeout reached while waiting for askpass.
>>
>> After using `break=mount` and investigating with `sh -x
>> /bin/cryptsetup-unlock` it seems it fails because it is not finding
>> `askpass` in the process list.
> 
> cryptsetup-unlock waits until the initramfs boot script is blocking on
> passphrase prompt.  This is only useful for injecting passphrases from
> outside the initramfs console (for instance from a remote shell).
> 

> When you set ‘break=mount’ our boot script isn't running in the
> background, so cryptsetup-unlock timeouts.  This is expected.  Why are
> you running cryptsetup-unlock in the first place instead of relying on
> the builtin initramfs logic?  Also, FWIW ‘break=mount’ breaks *after*
> unlocking whatever block devices need to be unlocked, so cryptsetup-initramfs
> has nothing more to do at this point.

Apologies for a mis-type there; I am using `break=premount`.

The install "appears" to fail to find and unlock both LUKS devices;
  it "hangs" (pauses) for a very long time with only kernel messages showing but
  no passphrase prompt is shown.

After debugging it turns out the kernel command-line option `debug` and initramfs-tools
  `init` script cause the passphrase prompts to be redirected to `initramfs.debug` when askpass
  is using "console" output.

It might be better if askpass writes the prompt directly to `/dev/console`. I've done a basic
  test and this does work when stdin/stderr are redirected. See end of this email.

Tapping `[Enter]` a few times will sometimes cause a few blank lines to
  scroll and seem to move things on since a few more kernel messages are
  generated (showing that more udev rules are being triggered).

Eventually, after more taps of `[Enter]` (or maybe due to a coincidental timeout)
  it does drop to shell. This is after something like 600 seconds (10 minutes).

Then I find:

REASON=ALERT! UUID=cb2a... does not exist. Dropping to shell!

`/proc/cmdline`: BOOT_IMAGE=/@/boot/vmlinuz... root=UUID=cb2a... ro rootflags=subvol=@ debug systemd.log_level=info

Looking at `/run/initramfs/initramfs.debug` today I see the reason  for it failing:
  when using `debug` the init script does:
```
   debug)
     debug=y
     quiet=n
     if [ -n "${netconsole}" ]; then
       log_output=/dev/kmsg
     else
       log_output=/run/initramfs/initramfs.debug
     fi
     set -x
     ;;
```
And looking at `/run/initramfs/initramfs.debug` it has 'captured' the passphrase prompts:
```
+ /scripts/local-top/cryptroot
Please unlock disk luks-88faf2e0-400b-4533-bdab-11ddb8d07fa7:
Nothing to read on input.
cryptsetup: ERROR: luks-88faf2e0-400b-4533-bdab-11ddb8d07fa7: cryptsetup
     failed, bad password or options?
Please unlock disk luks-88faf2e0-400b-4533-bdab-11ddb8d07fa7:
Nothing to read on input.
cryptsetup: ERROR: luks-88faf2e0-400b-4533-bdab-11ddb8d07fa7: cryptsetup
     failed, bad password or options?
Please unlock disk luks-88faf2e0-400b-4533-bdab-11ddb8d07fa7:
Nothing to read on input.
cryptsetup: ERROR: luks-88faf2e0-400b-4533    if (fwrite(prompt_ptr, line_len, 1, stderr) < 1 ||
         fwrite("\n", 1, 1, stderr) < 1) {
-bdab-11ddb8d07fa7: cryptsetup
     failed, bad password or options?
cryptsetup: ERROR: luks-88faf2e0-400b-4533-bdab-11ddb8d07fa7: maximum number of
     tries exceeded
```
Typing the passphrases blind when the kernel messages 'hang' with a decent pause between them
  to allow for the time it takes to 'unlock' allows one to continue starting successfully.

So the issue isn't askpass not running but the lack of a passphrase prompt when using
  `debug` on kernel command line AND askpass using its "console" method.
I set `debug` by default on all hosts but on most that use cryptsetup-initramfs the host
  is using key-files so never prompts for a passphrase and therefore I never hit this issue before.

Diagnosing why the prompt is 'lost' `run_keyscript()` does:
```
     elif [ "$keyscriptarg" = "none" ]; then
         # don't export the prompt message as CRYPTTAB_KEY
         keyscript="/lib/cryptsetup/askpass"
         keyscriptarg="Please unlock disk $CRYPTTAB_NAME: "
     fi

     exec "$keyscript" "$keyscriptarg"
```
and that is the source of the prompt (there are 2 places where an identical prompt
  is set/written).

I enabled DEBUG in `askpass` and copied it into the initrd to capture `/tmp/askpass.debug`
  and then allowed init to continue (`exit`).

I had to tap [Enter] a few times to get back to the shell, then looking at the log:
```
Enabling method systemd
Enabling method fifo
Enabling method plymouth
Enabling method console
method 1 has fd 4 and name fifo
method 3 has fd 0 and name console
Starting select with nfds 5
Writing 0 bytes to stdout
Disabling method ALL
```
This is repeated six times (presumably three tries each for the two LUKS devices).

So when using its "console" method stdout is redirected by initramfs-tools `init` when
  `debug` is set:
```
if [ -n "$log_output" ]; then
   exec >"$log_output" 2>&1
   unset log_output
fi
```
and as askpass.c::console_prepare() does:
```
     if (fwrite(prompt_ptr, line_len, 1, stderr) < 1 ||
         fwrite("\n", 1, 1, stderr) < 1) {
```
the output is swallowed.

I wrote a proof-of-concept to write to `/dev/console` to avoid the redirects and it
  appears to work successfully in the initialramfs:
```
(initramfs) /mnt/usb/test_console >/tmp/test.log 2>&1
Testing /dev/console
```
Code:
```
#include <stdio.h>
#include <unistd.h>

int main(int argc, char **argv, char **env)
{
   char *console = "/dev/console";
   if (access(console, W_OK) < 0) {
     perror("access('/dev/console') ");
     return -1;
   }

   FILE *out = fopen(console, "a");
   if (!out) {
     fprintf(stderr, "Failed to open %s\n", console);
     return -2;
   } else {
     fprintf(out, "\nTesting %s\n", console);
     fclose(out);
   }
   return 0;
}
```
So I wonder if this would be possible with askpass?