[med-svn] [Git][med-team/mapdamage][upstream] New upstream version 2.2.0+dfsg
Andreas Tille
gitlab at salsa.debian.org
Fri Dec 13 13:06:28 GMT 2019
Andreas Tille pushed to branch upstream at Debian Med / mapdamage
Commits:
2d75da74 by Andreas Tille at 2019-12-13T10:54:26Z
New upstream version 2.2.0+dfsg
- - - - -
7 changed files:
- README.md
- mapdamage/mp_test.py
- mapdamage/rescale.py
- + mapdamage/tests/probs/Stats_out_MCMC_correct_prob.csv
- + mapdamage/tests/test.rescaled.correct.sam
- mapdamage/version.py
- setup.py
Changes:
=====================================
README.md
=====================================
@@ -1,19 +1,35 @@
## mapDamage
-[](http://bioconda.github.io/recipes/mapdamage2/README.html) [](https://anaconda.org/bioconda/mapdamage2/files)
+[](http://bioconda.github.io/recipes/mapdamage2/README.html) [](https://anaconda.org/bioconda/mapdamage2/files)   [](https://www.repostatus.org/#inactive)
+
+
+#### `bioconda` installation
+
+- python3 version **2.2.0**
+```
+conda install -c bioconda mapdamage2=2.2.0
+```
+
+- python3 version **2.2.0** **with** R and 4 mandatory packages for the Bayesian inference:
+```
+conda install -c bioconda mapdamage2=2.2.0=pyr36_1
+```
---
### Important
-Users with versions dating prior to June 12 2013 please update. A nasty bug that caused the statistical part of `mapDamage` to use half of the data for estimation of the damage parameters, sorry for the inconvenience.
+
+- From version `2.2.0` the `master` branch is requiring **python3** as `python2` is not supported from 2020-01-01.
+
+- Users with versions dating prior to June 12 2013 please update. A nasty bug that caused the statistical part of `mapDamage` to use half of the data for estimation of the damage parameters, sorry for the inconvenience.
### Introduction
Complete documentation, instructions, examples, screenshots and FAQ are available at [this address](http://ginolhac.github.io/mapDamage/).
-[mapDamage2](http://geogenetics.ku.dk/publications/mapdamage2.0/) is a computational framework written in **Python3** and **R**, which tracks and quantifies DNA damage patterns
+[mapDamage2](https://geogenetics.ku.dk/publications/mapdamage2.0/) is a computational framework written in **Python3** and **R**, which tracks and quantifies DNA damage patterns
among ancient DNA sequencing reads generated by Next-Generation Sequencing platforms.
-`mapDamage` was developed at the [Centre for GeoGenetics](http://geogenetics.ku.dk/) by the [Orlando Group ](http://geogenetics.ku.dk/research/research_groups/palaeomix_group/).
+`mapDamage` was developed at the [Centre for GeoGenetics](https://geogenetics.ku.dk/) by the [Orlando Group ](https://geogenetics.ku.dk/research_groups/palaeomix_group/).
### Citation
@@ -29,15 +45,30 @@ Ginolhac A, Rasmussen M, Gilbert MT, Willerslev E, Orlando L.
http://bioinformatics.oxfordjournals.org/content/27/15/2153](http://bioinformatics.oxfordjournals.org/content/27/15/2153)
-### Test
+### Test the no-stats part and rescaling
-in the package, you can test `mapDamage` by running:
+you can test `mapDamage` by running:
```
cd mapDamage/mapdamage/
python3 mp_test.py
```
+should return
+```
+Started with the command: /usr/local/bin/mapDamage -i tests/test.bam -r tests/fake1.fasta -d tests/results --no-stats
+ Reading from 'tests/test.bam'
+ Writing results to 'tests/results/'
+pdf tests/results/Fragmisincorporation_plot.pdf generated
+additional tests/results/Length_plot.pdf generated
+Successful run
+.
+----------------------------------------------------------------------
+Ran 2 tests in 3.357s
+
+OK
+```
+
### Contact
Please report bugs and suggest possible improvements to Aurélien Ginolhac, Mikkel Schubert or Hákon Jónsson by email:
=====================================
mapdamage/mp_test.py
=====================================
@@ -1,12 +1,28 @@
import unittest
import subprocess
-
+import pysam
+import mapdamage
+import optparse
+import filecmp
def read_nocomment(dnacomp):
with open(dnacomp, 'r') as f:
a = f.readlines()
return [x for x in a if not x.startswith('#')]
+def mock_options(filename,rescale_out,folder):
+ """Make the options object with nice values for testing"""
+ return optparse.Values({
+ "filename":filename,
+ "rescale_out":rescale_out,
+ "verbose":True,
+ "folder":folder,
+ "rescale_length_5p":12, # default values as in --seq-length
+ "rescale_length_3p":12, # default values as in --seq-length
+ "quiet":True
+ })
+
+
class testCases(unittest.TestCase):
def test_no_stats(self):
@@ -14,5 +30,31 @@ class testCases(unittest.TestCase):
subprocess.run(["mapDamage", "-i", "tests/test.bam", "-r", "tests/fake1.fasta", "-d", "tests/results", "--no-stats"], check = True)
self.assertTrue(read_nocomment("tests/dnacomp.txt") == read_nocomment("tests/results/dnacomp.txt"))
+class testRescaling(unittest.TestCase):
+ def test_single_end_file(self):
+ """Test, rescaling BAM file"""
+ #
+ # The expected substition frequencies before and after scaling using the scaled qualities as probalities:
+ # CT 0.06226411977920493 0.04163524443356556
+ # TC 0.020395286584806528 0.020395286584806528
+ # GA 0.04400459948304954 0.03794905109091021
+ # AG 0.05355350777642983 0.05355350777642983
+ # Quality metrics before and after scaling
+ # CT-Q0 5 5
+ # CT-Q10 5 2
+ # CT-Q20 3 2
+ # CT-Q30 3 2
+ # CT-Q40 0 0
+ # GA-Q0 5 5
+ # GA-Q10 5 4
+ # GA-Q20 1 0
+ # GA-Q30 1 0
+ # GA-Q40 0 0
+ options = mock_options("tests/test.bam","tests/test.rescaled.sam","tests/probs/")
+ ref = pysam.Fastafile("tests/fake1.fasta")
+ mapdamage.rescale.rescale_qual(ref,options,debug=True)
+ self.assertTrue(filecmp.cmp("tests/test.rescaled.sam","tests/test.rescaled.correct.sam"))
+
+
if __name__=='__main__':
unittest.main()
=====================================
mapdamage/rescale.py
=====================================
@@ -19,8 +19,8 @@ def phred_char_to_pval(ch):
def get_corr_prob(folder, rescale_length_5p, rescale_length_3p):
"""
- Reads the damage probability correction file, returns a
- dictionary with this structure
+ Reads the damage probability correction file, returns a
+ dictionary with this structure
position (one based) - CT - probability
- GA - probability
"""
@@ -28,21 +28,21 @@ def get_corr_prob(folder, rescale_length_5p, rescale_length_3p):
if not os.path.isfile(full_path):
sys.exit("Missing file, the file \n\tStats_out_MCMC_correct_prob.csv\nshould be in the folder\n\t"+folder+"\nDid you run the MCMC estimates of the parameters?")
try:
- fi_handle = csv.DictReader(open(full_path))
- corr_prob = {}
- for line in fi_handle:
- if (line["Position"] in corr_prob):
- sys.exit('This file has multiple position definitions %s, line %d: %s' % \
- (folder, fi_handle.line_num, corr_prob[line["Position"]]))
- else:
- corr_prob[int(line["Position"])] = {'C.T':float(line["C.T"]), 'G.A':float(line["G.A"])}
-
- # Exclude probabilities for positions outside of user-specified region
- for key in list(corr_prob.keys()):
- if key < -rescale_length_3p or key > rescale_length_5p:
- corr_prob.pop(key)
+ with open(full_path) as fi:
+ fi_handle = csv.DictReader(fi)
+ corr_prob = {}
+ for line in fi_handle:
+ if (line["Position"] in corr_prob):
+ sys.exit('This file has multiple position definitions %s, line %d: %s' % \
+ (folder, fi_handle.line_num, corr_prob[line["Position"]]))
+ else:
+ corr_prob[int(line["Position"])] = {'C.T':float(line["C.T"]), 'G.A':float(line["G.A"])}
- return corr_prob
+ # Exclude probabilities for positions outside of user-specified region
+ for key in list(corr_prob.keys()):
+ if key < -rescale_length_3p or key > rescale_length_5p:
+ corr_prob.pop(key)
+ return corr_prob
except csv.Error as e:
sys.exit('File %s, line %d: %s' % (os.path.join(folder,"Stats_out_MCMC_correct_prob.csv"), \
fi_handle.line_num, e))
@@ -50,11 +50,11 @@ def get_corr_prob(folder, rescale_length_5p, rescale_length_3p):
def corr_this_base(corr_prob, nt_seq, nt_ref, pos, length,direction="both"):
"""
- The position specific damaging correction, using the input
- corr_prob dictionary holding the damage correcting values
- nt_seq nucleotide in the sequence
+ The position specific damaging correction, using the input
+ corr_prob dictionary holding the damage correcting values
+ nt_seq nucleotide in the sequence
nt_ref nucleotide in the reference
- pos relative position from the 5' end
+ pos relative position from the 5' end
length length of the sequence
direction which end to consider the rescaling
returns the correction probability for this particular set
@@ -71,8 +71,8 @@ def corr_this_base(corr_prob, nt_seq, nt_ref, pos, length,direction="both"):
else:
# other transitions/transversions are not affected by damage
return 0
-
- back_pos = pos-length-1
+
+ back_pos = pos-length-1
# position from 3' end
if pos in corr_prob:
@@ -90,7 +90,7 @@ def corr_this_base(corr_prob, nt_seq, nt_ref, pos, length,direction="both"):
if direction == "forward":
return p5_corr
elif direction == "backward":
- return p3_corr
+ return p3_corr
elif direction == "both":
if pos < abs(back_pos) :
# then we use the forward correction
@@ -151,7 +151,7 @@ def record_subs(subs,nt_seq,nt_ref,nt_qual,nt_newqual,prob_corr):
else:
sub_type = "NN"
if (sub_type != "NN"):
- # record only transitions
+ # record only transitions
subs[sub_type+"-before"][int(ord(nt_qual))-33] += 1
subs[sub_type+"-after"][int(ord(nt_newqual))-33] += 1
if (nt_ref in ["A","C","G","T"]):
@@ -175,7 +175,7 @@ def print_subs(subs):
if subs["C"]!=0:
# the special case of no substitutions
print(("\tCT\t"+str(subs["CT-pvals_before"]/subs["C"])+"\t\t"+str(subs["CT-pvals"]/subs["C"])))
- else:
+ else:
print("\tCT\tNA\t\tNA")
if subs["T"]!=0:
print(("\tTC\t"+str(subs["TC-pvals"]/subs["T"])+"\t\t"+str(subs["TC-pvals"]/subs["T"])))
@@ -200,19 +200,19 @@ def print_subs(subs):
print(("\tGA-Q20 \t"+str(subs["GA-before-Q20"])+"\t\t"+str(subs["GA-after-Q20"])))
print(("\tGA-Q30 \t"+str(subs["GA-before-Q30"])+"\t\t"+str(subs["GA-after-Q30"])))
print(("\tGA-Q40 \t"+str(subs["GA-before-Q40"])+"\t\t"+str(subs["GA-after-Q40"])))
-
+
def rescale_qual_read(bam, read, ref, corr_prob,subs, debug = False,direction="both"):
"""
bam a pysam bam object
read a pysam read object
ref a pysam fasta ref file
- reflengths a dictionary holding the length of the references
- subs a dictionary holding the corrected number of substition before and after scaling
+ reflengths a dictionary holding the length of the references
+ subs a dictionary holding the corrected number of substition before and after scaling
corr_prob dictionary from get_corr_prob
returns a read with rescaled quality score
-
- Iterates through the read and reference, rescales the quality
+
+ Iterates through the read and reference, rescales the quality
according to corr_prob
"""
if not debug:
@@ -238,7 +238,7 @@ def rescale_qual_read(bam, read, ref, corr_prob,subs, debug = False,direction="b
pos_on_read = 0
number_of_rescaled_bases = 0.0
for (i, nt_seq, nt_ref, nt_qual) in zip(range(length_align), seq, refseq, qual):
- # rescale the quality according to the triplet position,
+ # rescale the quality according to the triplet position,
# pair of the reference and the sequence
if ((nt_seq == "T" and nt_ref =="C") or (nt_seq == "A" and nt_ref =="G")):
# need to rescale this subs.
@@ -250,13 +250,13 @@ def rescale_qual_read(bam, read, ref, corr_prob,subs, debug = False,direction="b
else:
# don't rescale, other bases
newp = 1 - phred_char_to_pval(nt_qual)
- newq = nt_qual
+ newq = nt_qual
if pos_on_read < length_read:
- new_qual[pos_on_read] = newq
+ new_qual[pos_on_read] = newq
record_subs(subs,nt_seq,nt_ref,nt_qual,new_qual[pos_on_read],newp)
if nt_seq != "-":
pos_on_read += 1
- # done with the aligned portion of the read
+ # done with the aligned portion of the read
else:
if not debug:
logger.warning("Warning: The aligment of the read is longer than the actual read %s",(read.qname))
@@ -266,7 +266,7 @@ def rescale_qual_read(bam, read, ref, corr_prob,subs, debug = False,direction="b
if read.is_reverse:
new_qual = new_qual[::-1]
if (read.cigar[0][0] == 4):
- # check for soft clipping at forward end
+ # check for soft clipping at forward end
new_qual = read.qual[0:read.cigar[0][1]] + new_qual
if (read.cigar[-1][0] == 4):
# the same backwards
@@ -285,7 +285,7 @@ def rescale_qual_read(bam, read, ref, corr_prob,subs, debug = False,direction="b
def rescale_qual(ref, options,debug=False):
- """
+ """
ref a pysam fasta ref file
bam_filename name of a BAM/SAM file to read
fi file containing the csv with correction probabilities
@@ -319,20 +319,20 @@ def rescale_qual(ref, options,debug=False):
pass
elif not hit.qual and not debug:
logger.warning("Cannot rescale base PHRED scores for read '%s'; no scores assigned." % hit.qname)
- elif hit.is_paired :
+ elif hit.is_paired :
if first_pair and not debug:
- # assuming the ends are non-overlapping
+ # assuming the ends are non-overlapping
logger.warning("Warning! Assuming the pairs are non-overlapping, facing inwards and correctly paired.")
first_pair=False
#5p --------------> 3p
#3p <-------------- 5p
# pair 1 (inwards)
- #5p ---->
+ #5p ---->
# <---- 5p
# A B
- # pair 2 (outwards), this happens if the reference is RC this is not supported
+ # pair 2 (outwards), this happens if the reference is RC this is not supported
# ----> 3p
- #3p <----
+ #3p <----
# A B
# Correct outwards pairs from the 3p and inwards pairs with the 5p end
if ((not hit.is_reverse) and hit.mate_is_reverse and (hit.pnext>hit.pos) and hit.tid==hit.mrnm):
=====================================
mapdamage/tests/probs/Stats_out_MCMC_correct_prob.csv
=====================================
@@ -0,0 +1,25 @@
+"","Position","C.T","G.A"
+"1",1,0.653616777018436,0
+"2",2,0.574821404505089,0
+"3",3,0.524660717195785,0
+"4",4,0.491676371805506,0
+"5",5,0.46919312975022,0
+"6",6,0.453361082263795,0
+"7",7,0.441882878430246,0
+"8",8,0.433345603590333,0
+"9",9,0.426853067214069,0
+"10",10,0.4218187429273,0
+"11",11,0.417847465539491,0
+"12",12,0.414666179579865,0
+"13",-12,0,0.427459648772127
+"14",-11,0,0.430678304009036
+"15",-10,0,0.434694598982698
+"16",-9,0,0.439783285536968
+"17",-8,0,0.446341431282045
+"18",-7,0,0.454957458647015
+"19",-6,0,0.466528745659899
+"20",-5,0,0.482466321045583
+"21",-4,0,0.505055143009322
+"22",-3,0,0.538101706832429
+"23",-2,0,0.588153916474448
+"24",-1,0,0.666277684522864
=====================================
mapdamage/tests/test.rescaled.correct.sam
=====================================
@@ -0,0 +1,11 @@
+ at SQ SN:fake1 LN:201
+ at PG ID:bwa PN:bwa VN:0.7.17-r1188 CL:bwa samse -f input.sam fake1.fasta - input.fastq
+fake1:50-100 0 fake1 69 37 31M * 0 0 CCATGTCGGGCAGGCTGGTCTCGAACTCCTG ////////E6/EE/E/A////E//AAAE/EE XT:A:U NM:i:0 X0:i:1 X1:i:0 XM:i:0 XO:i:0 XG:i:0 MD:Z:31 MR:f:0
+fake1:50-100b 0 fake1 69 37 31M * 0 0 CTATGTCGGGCAGGCTGGTCTCGAACTCCTG /#//////E6/EE/E/A////E//AAAE/EE XT:A:U NM:i:1 X0:i:1 X1:i:0 XM:i:1 XO:i:0 XG:i:0 MD:Z:1C29 MR:f:0.57482
+fake1:50-100c 4 * 0 0 * * 0 0 CGGTAGAGATGGAGTTTCACCATGTCGGGCAAG //////////////A////////////E6/EE/
+fake1:50-100d 4 * 0 0 * * 0 0 TAATAGAGATGGAGTTTCACCATGTCGTGCAAG //////////////A////////////E6/EE/
+fake1:70-120 0 fake1 70 37 38M * 0 0 TATGTCGGGCAGGCTGGTCTCGAACTCCTGACCTCAGA #////6EE//AA6E//E66E/EAEEEEEAEEEAAEEA# XT:A:U NM:i:2 X0:i:1 X1:i:0 XM:i:2 XO:i:0 XG:i:0 MD:Z:0C36G0 MR:f:1.31989
+fake1:80-130 16 fake1 80 25 41M * 0 0 GAGCTGGTCTCGAACTCCTGACCTCAGGCGATCTGCCTGTC AAABBCCDDFEEEEEAEEEEA/EE/E/AAE//A//E/A/// XT:A:U NM:i:3 X0:i:1 X1:i:0 XM:i:3 XO:i:0 XG:i:0 MD:Z:0A0G37C1 MR:f:0
+fake1:80-130b 16 fake1 80 25 51M * 0 0 AGGCCGGTCTCGAACTCCTGACCTCAGGCGATCTGCCTGCCTTAACCTCCC AAABBCCDDFEEEEEAEEEEA/EE/E/AAE//A//E/A///E/A$/EE66/ XT:A:U NM:i:3 X0:i:1 X1:i:0 XM:i:3 XO:i:0 XG:i:0 MD:Z:4T37C1G6 MR:f:0.44188
+fake1:80-130c 16 fake1 80 37 51M * 0 0 AGGCTGGTCTCGAACTCCTGACCTCAGGCGATCTGCCTGCCTTAGCCCCCC AAABBCCDDFEEEEEAEEEEA/EE/E/AAE//A//E/A///E/A//EE66/ XT:A:U NM:i:2 X0:i:1 X1:i:0 XM:i:2 XO:i:0 XG:i:0 MD:Z:42C4T3 MR:f:0
+fake1:80-130d 16 fake1 80 25 51M * 0 0 AGGCTGGTCTCGAACTCCTGACCTCAGACGATCTGCCTGCCTTAGCCCCCC AAABBCCDDFEEEEEAEEEEA/EE/E/AAE//A//E/A///E/A//EE66/ XT:A:U NM:i:3 X0:i:1 X1:i:0 XM:i:3 XO:i:0 XG:i:0 MD:Z:27G14C4T3 MR:f:0
=====================================
mapdamage/version.py
=====================================
@@ -2,4 +2,4 @@
try:
from mapdamage._version import __version__
except ImportError:
- __version__ = "2.1.0"
+ __version__ = "2.2.0"
=====================================
setup.py
=====================================
@@ -4,8 +4,13 @@
from distutils.core import setup
from distutils.command.install import install as DistutilsInstall
import os
+import sys
import subprocess
+if sys.version_info < (3, 5):
+ print("At least Python 3.5 is required.\n", file=sys.stderr)
+ exit(1)
+
def compile_seqtk():
"""Compiling the seqtk toolkit"""
old_wd = os.getcwd()
@@ -55,11 +60,11 @@ class compileInstall(DistutilsInstall):
setup(
cmdclass={'install': compileInstall},
name='mapdamage',
- version='2.1.0',
- author='Aurélien Ginolhac, Mikkel Schubert, Hákon Jónsson',
+ version='2.2.0',
+ author='Aurélien Ginolhac, Mikkel Schubert, Ãkon Jónsson',
author_email='MSchubert at snm.ku.dk, jonsson.hakon at gmail.com',
packages=['mapdamage'],
- package_data={'mapdamage': ['Rscripts/*.R','Rscripts/stats/*.R','seqtk/seqtk']},
+ package_data={'mapdamage': ['Rscripts/*.R','Rscripts/stats/*.R','tests/*','seqtk/seqtk']},
scripts=['bin/mapDamage'],
url='https://github.com/ginolhac/mapDamage',
license='LICENSE.txt',
View it on GitLab: https://salsa.debian.org/med-team/mapdamage/commit/2d75da747584b1166c8a9200b964ea44c346a9ed
--
View it on GitLab: https://salsa.debian.org/med-team/mapdamage/commit/2d75da747584b1166c8a9200b964ea44c346a9ed
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/debian-med-commit/attachments/20191213/2112964d/attachment-0001.html>
More information about the debian-med-commit
mailing list