Bug#844233: python-passlib: FTBFS with probability of 0.22% due to non-deterministic testsuite

[Chris Lamb]

Source: python-passlib
Version: 1.6.5-4
Severity: serious
Tags: ftbfs
User: reproducible-builds at lists.alioth.debian.org
Usertags: randomess
X-Debbugs-Cc: reproducible-builds at lists.alioth.debian.org

Hi,

Whilst working on the Reproducible Builds effort [0], we noticed
that python-passlib's testsuite will non-determinstically FTBFS:

======================================================================
FAIL: test_getrandstr (passlib.tests.test_utils.MiscTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/python-passlib-1.6.5/2nd/passlib/tests/test_utils.py", line 129, in test_getrandstr
    self.assertEqual(sorted(set(x)), [u('a'),u('b'),u('c')])
AssertionError: Lists differ: [u'a', u'b'] != [u'a', u'b', u'c']

Second list contains 1 additional elements.
First extra element 2:
u'c'

- [u'a', u'b']
+ [u'a', u'b', u'c']
?            ++++++

This is because of:

125         x = f(u('abc'), 16)
126         y = f(u('abc'), 16)
127         self.assertIsInstance(x, unicode)
128         self.assertNotEqual(x,y)
129         self.assertEqual(sorted(set(x)), [u('a'),u('b'),u('c')])

If the random string ``x`` doesn't contain a certain character, the
assertion will fail. By my quick calculation this will happen with a
probability of (2/3)^(16-1).

It will also FTBFS if it generates the exact same random string, but
that seems a little more unlikely.

 [0] https://reproducible-builds.org/


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby at debian.org / chris-lamb.co.uk
       `-