Bug#403075: [Pkg-cryptsetup-devel] Bug#403075: cryptsetup luksOpen
can kill unrelated processes (out of memory killer)
Rob Walker
rob at tenfoot.org.uk
Wed Dec 20 10:32:19 UTC 2006
> >>
> >> Are you able to reproduce the error with the version of cryptsetup that
> >> is currently in unstable?
> >
> >I can reproduce the bug using the cryptsetup from unstable. I've also
> > tried it on another machine with the same results.
>
> Ok, could you then provide me with the exact steps that you took to
> reproduce it cause so far I haven't managed to do so.
I think I've found the problem. In lib/utils.c, the sector_size function is
this:
static int sector_size(int fd)
{
int bsize;
ioctl(fd,BLKSSZGET, &bsize);
return bsize;
}
For a file, the ioctl will fail. Since bsize is not initialized, it's value
will be random (when I ran it under gdb, I got several million) and as this
value will later be used to allocate a buffer, this explains the runaway
memory allocation I saw, but you didn't.
The attached patch checks the return code from ioctl and returns -EINVAL if it
fails. The users of sector_size are also changed to abort if sector_size
returns a negative value.
With this patch applied, I get the appropriate error messages from cryptsetup
if I give it a file instead of a block device.
Regards
Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cryptsetup_sector_size_error.patch
Type: text/x-diff
Size: 983 bytes
Desc: not available
Url : http://lists.alioth.debian.org/pipermail/pkg-cryptsetup-devel/attachments/20061220/b507dbde/cryptsetup_sector_size_error.bin
More information about the pkg-cryptsetup-devel
mailing list