[Debian-med-packaging] Bug#938501: Please port smalt to Python3 (Was: Bug#938501: smalt: Python2 removal in sid/bullseye) [EXT]

Tue Oct 8 17:24:47 BST 2019

Hi all,

Hannes is still around, but has moved jobs.  I'm including Zemin Ning
in this reply whose team Hannes was in when he wrote Smalt.

I don't know the extent of the issue having not used Smalt myself, but
the main tool itself is C so I doubt the python bit is a major
component.

Regardless, it's not something I'm involved with personally.

James

On Tue, Oct 08, 2019 at 05:33:42PM +0200, Andreas Tille wrote:
> Hi again,
> 
> I just took James Bonfield in CC since I'm unsure whether my mail might
> have reached someone who is currently working at sanger.ac.uk.  Just to
> stretch the importance of the issue have a look at the Python2 EOL clock:
> 
>    https://urldefense.proofpoint.com/v2/url?u=https-3A__pythonclock.org_&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=1ojoaFsu0HVyuhxGkiUBSnPjQJLSXJQyv1jrE-FfnMw&e= 
> 
> In short: Please be so kind and verify the patch linked below and fix
> the remaining issues with your deeper knowledge of the code.
> 
> Thanks a lot
> 
>       Andreas.
> 
> 
> On Thu, Sep 05, 2019 at 03:39:35PM +0200, Andreas Tille wrote:
> > Control: tags -1 upsteam
> > Control: forwarded -1 Hannes Ponstingl <hp3 at sanger.ac.uk>
> > 
> > Hi Hannes,
> > 
> > as you can read below Debian will remove Python2 since it is EOL.
> > I tried to port your test scripts using the 2to3 tool.  The result
> > of the automatic conversion can be found here:
> > 
> >     https://urldefense.proofpoint.com/v2/url?u=https-3A__salsa.debian.org_med-2Dteam_smalt_blob_master_debian_patches_2to3.patch&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=j7oR-RSaMB5DM2SvBCK63XeKcftyATF9QB_A56HxSCs&e= 
> > 
> > Unfortunately there are some remaining issues in the test suite which
> > are probably not very hard to fix.  I wonder whether you might want
> > to have a look at these and could prepare an official Python3 release
> > of your nice tool.
> > 
> > Here is the output of the test suite:
> > 
> > 
> > PASS: splitReads_test.py
> > PASS: results_split_test.py
> > PASS: ouform_cigar_test.py
> > Traceback (most recent call last):
> >   File "./sample_test.py", line 154, in <module>
> >     compare_mapping(oufilnam1, oufilnam3)
> >   File "./sample_test.py", line 108, in compare_mapping
> >     if cmp(cig1.qnam, cig2.qnam):
> > NameError: name 'cmp' is not defined
> > FAIL: sample_test.py
> > PASS: cigar_test.py
> > Discrepancy:
> > cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
> > cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
> > FAIL: mthread_test.py
> > Traceback (most recent call last):
> >   File "./ioform_test.py", line 45, in <module>
> >     samnam = df.unpack(READ_PREFIX + ".sam")
> >   File "/build/smalt-0.7.6/test/testdata.py", line 71, in unpack
> >     oufil.write(lin)
> > TypeError: write() argument must be str, not bytes
> > FAIL: ioform_test.py
> > PASS: xali_test.py
> > Traceback (most recent call last):
> >   File "./bam_cigar_test.py", line 254, in <module>
> >     isOK = testSAMfilesAreIdentical(sambamnam, samoufilnam)
> >   File "./bam_cigar_test.py", line 149, in testSAMfilesAreIdentical
> >     linA = infilA.readline()
> >   File "/usr/lib/python3.7/codecs.py", line 322, in decode
> >     (result, consumed) = self._buffer_decode(data, self.errors, final)
> > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 260: invalid start byte
> > FAIL: bam_cigar_test.py
> > =================================
> > 4 of 9 tests failed
> > Please report to hp3 at sanger.ac.uk
> > =================================
> > 
> > 
> > Kind regards
> > 
> >       Andreas.
> > 
> > 
> > On Fri, Aug 30, 2019 at 07:52:50AM +0000, Matthias Klose wrote:
> > > Package: src:smalt
> > > Version: 0.7.6-8
> > > Severity: normal
> > > Tags: sid bullseye
> > > User: debian-python at lists.debian.org
> > > Usertags: py2removal
> > > 
> > > Python2 becomes end-of-live upstream, and Debian aims to remove
> > > Python2 from the distribution, as discussed in
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.debian.org_debian-2Dpython_2019_07_msg00080.html&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=Xgujf6sRlZuNDRpgPdw-HM2EVf1K2YRkzazDPAt9sK4&e= 
> > > 
> > > Your package either build-depends, depends on Python2, or uses Python2
> > > in the autopkg tests.  Please stop using Python2, and fix this issue
> > > by one of the following actions.
> > > 
> > > - Convert your Package to Python3. This is the preferred option.  In
> > >   case you are providing a Python module foo, please consider dropping
> > >   the python-foo package, and only build a python3-foo package.  Please
> > >   don't drop Python2 modules, which still have reverse dependencies,
> > >   just document them.
> > >   
> > >   This is the preferred option.
> > > 
> > > - If the package is dead upstream, cannot be converted or maintained
> > >   in Debian, it should be removed from the distribution.  If the
> > >   package still has reverse dependencies, raise the severity to
> > >   "serious" and document the reverse dependencies with the BTS affects
> > >   command.  If the package has no reverse dependencies, confirm that
> > >   the package can be removed, reassign this issue to ftp.debian.org,
> > >   make sure that the bug priority is set to normal and retitle the
> > >   issue to "RM: PKG -- removal triggered by the Python2 removal".
> > > 
> > > - If the package has still many users (popcon >= 300), or is needed to
> > >   build another package which cannot be removed, document that by
> > >   adding the "py2keep" user tag (not replacing the py2remove tag),
> > >   using the debian-python at lists.debian.org user.  Also any
> > >   dependencies on an unversioned python package (python, python-dev)
> > >   must not be used, same with the python shebang.  These have to be
> > >   replaced by python2/python2.7 dependencies and shebang.
> > > 
> > >   This is the least preferred option.
> > > 
> > > If the conversion or removal needs action on another package first,
> > > please document the blocking by using the BTS affects command, like
> > > 
> > >   affects <bug number of blocking py2removal bug> + src:smalt
> > > 
> > > If there is no py2removal bug for that reverse-dependency, please file
> > > a bug on this package (similar to this bug report).
> > > 
> > > If there are questions, please refer to the wiki page for the removal:
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.debian.org_Python_2Removal&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=LtGkMxuth36_KHtGeql2HwKH7xEIKZ6BNAP0GOH9aDE&e= , or ask for help on IRC
> > > #debian-python, or the debian-python at lists.debian.org mailing list.
> > > 
> > > _______________________________________________
> > > Debian-med-packaging mailing list
> > > Debian-med-packaging at alioth-lists.debian.net
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__alioth-2Dlists.debian.net_cgi-2Dbin_mailman_listinfo_debian-2Dmed-2Dpackaging&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=QNdw7jPPPozYeLfZMk5O-mjcTGurrKIidxWrdwjOlao&e= 
> > 
> > -- 
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=vbfIEwXiPAwzyQefpXbHmZpdmW1CI-xSYZm96PkCsmw&e= 
> > 
> > _______________________________________________
> > Debian-med-packaging mailing list
> > Debian-med-packaging at alioth-lists.debian.net
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__alioth-2Dlists.debian.net_cgi-2Dbin_mailman_listinfo_debian-2Dmed-2Dpackaging&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=QNdw7jPPPozYeLfZMk5O-mjcTGurrKIidxWrdwjOlao&e= 
> 
> -- 
> https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=vbfIEwXiPAwzyQefpXbHmZpdmW1CI-xSYZm96PkCsmw&e= 

-- 
James Bonfield (jkb at sanger.ac.uk)
The Sanger Institute, Hinxton, Cambs, CB10 1SA

-- 
 The Wellcome Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 