Bug#901888: libparams-validate-perl: Memory corruption and crashes in the jessie version

[Christoph Biedl]

Package: libparams-validate-perl
Version: 1.13-1+b1
Severity: important

Disclaimer: I'm aware this is a few days too late for regular jessie
updates, including the upcoming ultimate point release. But I consider
the issue serious enough to give it some publicity. About possible
resolutions, see below.

Dear maintainer,

when validating certain data structures, Param::Validate in jessie and
probably up to and including 1.22 corrupts memory, leading to crashes
like

| *** Error in `perl': free(): invalid next size (fast): 0x000000000237fc30 ***
| Aborted

See below for a reproducer. Run it a few times, crash timing and output
vary.

The data is a list of hashes, each hash should conform to a certain
specification, and Params::Validate is used to enforce that.

The interesting and somewhat confusing part: The data has to created
using JSON::XS - if the data is created using Perl statements,
everything works as expected. Still I'm confident JSON::XS is not to
blame here as I initially assumed: Using Perl storables instead leads
to these crashes as well.

The number of 122 records was found experimentally, actually, when
dealing with production data. Other numbers work as well if you
slightly change the content of the hash.

The crucial operation is the "regex" check - disabling it makes the
code pass.


How to resolve ...

Workarounds: Perhaps not too surprising, running the Dumper function on
the data before validation appearently sanitizes the internal
structures. Another workaround was to use the pure Perl implementation
of Params::Validate, probably at a significant performance cost.

Playing with several upstream versions since the one used for jessie
reveals this was not yet fixed in 1.16 but in 1.23. Checking the
upstream changelog suggests the latter is the first release that fixed
the issue.

About jessie, perhaps if the LTS team wishes to resolve that: The diff
on lib/Params/Validate/XS.xs between 1.13 and 1.23 is fairly huge, 671
lines. Perhaps somebody with a deeper understanding of XS might trim
this down to the essential changes but I reckon they will still be
somewhat big. So I'd rather recommend to go forward and use a jessie
backport of the stretch version (1.26-1). But that's not me to decide.

Regards,

    Christoph

=================================================================
#!/usr/bin/perl

use 5.010;
use strict;
use warnings;

use JSON::XS;
use Params::Validate qw<:all>;

my $spec = {
    'field1' => { 'type' => SCALAR|UNDEF, 'default' => undef },
    'field2' => { 'type' => SCALAR|UNDEF, 'default' => undef },
    'field3' => {
        'type' => SCALAR | UNDEF,
        'regex' => qr/./,
        'default' => 0,
    },
};

# create a buffer of JSON
my $buffer =
    "[\n"  . <<__EOS__ x 122 . "]\n";
    {
        "field1": "-",
        "field2": "content-122",
        "field3" : "0",
    },
__EOS__

# parse JSON into data
my $data = JSON::XS->new->utf8->relaxed (1)->decode ($buffer);

my @new_list =
    map {
        my %h =
            validate_with (
                'params' => $_,
                'spec' => $spec,
            );
        \%h;
    }
    @$data;

print "I: Here we go\n";
=================================================================


-- System Information:
Debian Release: 8.10
  APT prefers oldstable-updates
  APT policy: (500, 'oldstable-updates'), (500, 'oldstable-proposed-updates'), (500, 'oldstable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.14.48 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)

Versions of packages libparams-validate-perl depends on:
ii  libc6                          2.19-18+deb8u10
ii  libmodule-implementation-perl  0.09-1
ii  perl                           5.20.2-3+deb8u11
ii  perl-base [perlapi-5.20.0]     5.20.2-3+deb8u11

libparams-validate-perl recommends no packages.

libparams-validate-perl suggests no packages.

-- no debconf information

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-perl-maintainers/attachments/20180619/0413215a/attachment-0001.sig>