[med-svn] [Git][med-team/libgoby-java][master] 6 commits: Rewriting the computation of the p-value of the Fisher test with libdistlib-java
Pierre Gruet (@pgt)
gitlab at salsa.debian.org
Fri Aug 6 20:37:32 BST 2021
Pierre Gruet pushed to branch master at Debian Med / libgoby-java
Commits:
92ca30e8 by Pierre Gruet at 2021-08-05T22:46:04+02:00
Rewriting the computation of the p-value of the Fisher test with libdistlib-java
Thus, including libdistlib-java in the build-dependencies.
Besides, deactivating testTwoStats which seems to expect a wrong result.
- - - - -
04db72ec by Pierre Gruet at 2021-08-06T15:34:30+02:00
Updating d/watch to fit new organization of Github URI
- - - - -
1cd1575b by Pierre Gruet at 2021-08-06T21:32:58+02:00
Adding d/u/metadata with Github paths
- - - - -
26c03878 by Pierre Gruet at 2021-08-06T21:33:48+02:00
Adding libdistlib-java as a dependency
This is needed because of the patch about the Fisher test p-value computation.
- - - - -
9ae4e6cf by Pierre Gruet at 2021-08-06T21:34:52+02:00
Refreshing d/copyright
- - - - -
48f27691 by Pierre Gruet at 2021-08-06T21:35:07+02:00
Excluding more tests that fail, maybe because unmaintained
I tried to make a Maven build (without the Debian stuff) of goby3 but only five
tests were executed. I tried to have all of them but half were failing.
Therefore I skip some tests that fail in a Maven build from the Debian packaging.
- - - - -
9 changed files:
- debian/control
- debian/copyright
- debian/libgoby-java.manifest
- − debian/patches/computing_Fisher_test_pvalue.patch
- + debian/patches/computing_fisher_pvalue_hypergeom.patch
- debian/patches/exclude_not_runnable_tests.patch
- debian/patches/series
- + debian/upstream/metadata
- debian/watch
Changes:
=====================================
debian/control
=====================================
@@ -19,6 +19,7 @@ Build-Depends: debhelper-compat (= 13),
libcommons-logging-java,
libcommons-math-java,
libcommons-cli-java,
+ libdistlib-java,
liblog4j1.2-java,
libexec-maven-plugin-java,
libmaven-assembly-plugin-java,
@@ -56,6 +57,7 @@ Depends: ${shlibs:Depends},
${misc:Depends},
${maven:Depends},
libcommons-collections3-java,
+ libdistlib-java,
libjbzip2-java,
libpj-java,
r-cran-rjava
=====================================
debian/copyright
=====================================
@@ -6,23 +6,188 @@ Files-Excluded: */*.jar
testsupport/*/lib
Files: *
-Copyright: © 20xx-20yy <upstream>
-License: GPL-3
+Copyright: 2009-2017 Institute for Computational Biomedicine
+License: GPL-3+
+
+Files: cpp/*
+Copyright: 2009 Peter Adolphs
+ 2008 Thomas Porschberg <thomas at randspringer.de>
+License: FSFAP
+
+Files: cpp/m4/ax_boost_date_time.m4
+Copyright: 2008 Thomas Porschberg <thomas at randspringer.de>
+ 2008 Michael Tindal
+License: FSFAP
+
+Files: cpp/m4/ax_boost_filesystem.m4
+Copyright: 2009 Thomas Porschberg <thomas at randspringer.de>
+ 2009 Roman Rybalko <libtorrent at romanr.info>
+ 2009 Michael Tindal
+License: FSFAP
+
+Files: cpp/m4/ax_boost_program_options.m4
+Copyright: 2009 Thomas Porschberg <thomas at randspringer.de>
+License: FSFAP
+
+Files: cpp/m4/ax_boost_system.m4
+Copyright: 2008 Thomas Porschberg <thomas at randspringer.de>
+ 2008 Michael Tindal
+ 2008 Daniel Casimiro <dan.casimiro at gmail.com>
+License: FSFAP
+
+Files: cpp/src/*
+Copyright: 2009-2012 Institute for Computational Biomedicine
+License: LGPL-3+
+
+Files: cpp/src/Makefile.am
+Copyright: 2009-2017 Institute for Computational Biomedicine
+License: GPL-3+
+
+Files: cpp/src/goby/ICBFileInputStream.cc
+ cpp/src/goby/ICBFileInputStream.h
+Copyright: 2008 Google Inc.
+License: BSD-3-clause
+
+Files: cpp/src/kseq.h
+Copyright: 2008 Genome Research Ltd (GRL).
+License: Expat
+
+Files: goby-distribution/src/main/java/org/campagnelab/goby/algorithmic/compression/FastArithmeticCoder.java
+Copyright: 2002-2011 Sebastiano Vigna
+ 2009-2012 Institute for Computational Biomedicine
+License: LGPL-3+ and GPL-3+
+
+Files: goby-distribution/src/main/java/org/campagnelab/goby/algorithmic/compression/FastArithmeticDecoder.java
+Copyright: 2002-2011 Sebastiano Vigna
+ 2012 Fabien Campagne
+License: LGPL-3+
+
+Files: goby-distribution/src/main/java/org/campagnelab/goby/GobyVersion.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/AbstractAlignmentReader.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/AlignedSequence.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/AlignmentPositionComparator.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/AlignmentReaderImpl.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/AlignmentStats.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/AlignmentTooManyHitsReader.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/AlignmentTooManyHitsWriter.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/AlignmentWriterImpl.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/ConcatAlignmentReader.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/ConcatSortedAlignmentReader.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/IterateAlignments.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/IterateSortedAlignments.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/LastParser.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/Merge.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/PositionFacadeAlignmentReader.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/ScanReferenceSequence.java
+ goby-distribution/src/main/java/org/campagnelab/goby/alignments/filters/*
+ goby-distribution/src/main/java/org/campagnelab/goby/baseinfo/*
+ goby-distribution/src/main/java/org/campagnelab/goby/compression/FastBufferedMessageChunksReader.java
+ goby-distribution/src/main/java/org/campagnelab/goby/compression/MessageChunksReader.java
+ goby-distribution/src/main/java/org/campagnelab/goby/compression/MessageChunksWriter.java
+ goby-distribution/src/main/java/org/campagnelab/goby/counts/*
+ goby-distribution/src/main/java/org/campagnelab/goby/exception/*
+ goby-distribution/src/main/java/org/campagnelab/goby/modes/AbstractCommandLineMode.java
+ goby-distribution/src/main/java/org/campagnelab/goby/modes/AbstractGobyMode.java
+ goby-distribution/src/main/java/org/campagnelab/goby/modes/ColorSpaceConverter.java
+ goby-distribution/src/main/java/org/campagnelab/goby/modes/GenericToolsDriver.java
+ goby-distribution/src/main/java/org/campagnelab/goby/modes/PercentMismatchesQualityFilter.java
+ goby-distribution/src/main/java/org/campagnelab/goby/modes/TabToColumnInfoMode.java
+ goby-distribution/src/main/java/org/campagnelab/goby/modes/UpgradeMode.java
+ goby-distribution/src/main/java/org/campagnelab/goby/modes/core/*
+ goby-distribution/src/main/java/org/campagnelab/goby/modes/dsv/DiscoverVariantIterateSortedAlignments.java
+ goby-distribution/src/main/java/org/campagnelab/goby/modes/dsv/IterateSortedAlignmentsListImpl.java
+ goby-distribution/src/main/java/org/campagnelab/goby/readers/*
+ goby-distribution/src/main/java/org/campagnelab/goby/reads/CompressedRead.java
+ goby-distribution/src/main/java/org/campagnelab/goby/reads/MultiReads.java
+ goby-distribution/src/main/java/org/campagnelab/goby/reads/QualityEncoding.java
+ goby-distribution/src/main/java/org/campagnelab/goby/reads/RandomAccessSequenceCache.java
+ goby-distribution/src/main/java/org/campagnelab/goby/reads/ReadSet.java
+ goby-distribution/src/main/java/org/campagnelab/goby/reads/ReadsLoader.java
+ goby-distribution/src/main/java/org/campagnelab/goby/reads/ReadsReader.java
+ goby-distribution/src/main/java/org/campagnelab/goby/reads/ReadsWriterImpl.java
+ goby-distribution/src/main/java/org/campagnelab/goby/reads/SequenceDigests.java
+ goby-distribution/src/main/java/org/campagnelab/goby/reads/SequenceEncoder.java
+ goby-distribution/src/test/java/org/campagnelab/goby/alignments/TestPositionSlices.java
+Copyright: 2009-2012 Institute for Computational Biomedicine
+License: LGPL-3+
+
+Files: goby-distribution/src/main/java/org/campagnelab/goby/counts/CachingCountsArchiveReader.java
+ goby-distribution/src/main/java/org/campagnelab/goby/counts/CountAdaptiveBinningAdaptor.java
+ goby-distribution/src/main/java/org/campagnelab/goby/counts/CountBinningAdapterI.java
+ goby-distribution/src/main/java/org/campagnelab/goby/counts/CountBinningAdaptor.java
+ goby-distribution/src/main/java/org/campagnelab/goby/counts/CountIndexBuilder.java
+ goby-distribution/src/main/java/org/campagnelab/goby/counts/CountWriterHelper2.java
+ goby-distribution/src/main/java/org/campagnelab/goby/counts/CountsAggregatorI.java
+ goby-distribution/src/main/java/org/campagnelab/goby/counts/CountsWriterI.java
+ goby-distribution/src/main/java/org/campagnelab/goby/counts/PositionAdapter.java
+ goby-distribution/src/main/java/org/campagnelab/goby/counts/UnionAlgorithmSkeleton.java
+ goby-distribution/src/main/java/org/campagnelab/goby/counts/UnionDumpIterator.java
+ goby-distribution/src/main/java/org/campagnelab/goby/readers/sam/*
+ goby-distribution/src/main/java/org/campagnelab/goby/readers/vcf/GroupAssociations.java
+Copyright: 2009-2017 Institute for Computational Biomedicine
+License: GPL-3+
Files: debian/*
-Copyright: © 2012 Andreas Tille <tille at debian.org>
-License: GPL-3
-
-License: GPL-3
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
+Copyright: 2012 Andreas Tille <tille at debian.org>
+ 2020-2021 Pierre Gruet <pgt at debian.org>
+License: LGPL-3+
+
+License: GPL-3+
+ On Debian systems the full text of the GPL-3 can be found in
+ /usr/share/common-licenses/GPL-3
+
+License: FSFAP
+ Copying and distribution of this file, with or without modification, are
+ permitted in any medium without royalty provided the copyright notice
+ and this notice are preserved. This file is offered as-is, without any
+ warranty.
+
+License: LGPL-3+
+ On Debian systems the full text of the LGPL-3 can be found in
+ /usr/share/common-licenses/LGPL-3
+
+License: BSD-3-clause
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+ 1. Redistributions of source code must retain the above copyright
+ notice, and the entire permission notice in its entirety,
+ including the disclaimer of warranties.
+ 2. Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in the
+ documentation and/or other materials provided with the distribution.
+ 3. The name of the author may not be used to endorse or promote
+ products derived from this software without specific prior
+ written permission.
.
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
+ THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED
+ WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT,
+ INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+ OF THE POSSIBILITY OF SUCH DAMAGE.
+
+License: Expat
+ Permission is hereby granted, free of charge, to any person obtaining
+ a copy of this software and associated documentation files (the
+ "Software"), to deal in the Software without restriction, including
+ without limitation the rights to use, copy, modify, merge, publish,
+ distribute, sublicense, and/or sell copies of the Software, and to
+ permit persons to whom the Software is furnished to do so, subject to
+ the following conditions:
.
- On Debian systems you can find the full text of the GPL3 at
- /usr/share/common-licenses/GPL-3
+ The above copyright notice and this permission notice shall be
+ included in all copies or substantial portions of the Software.
+ .
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
=====================================
debian/libgoby-java.manifest
=====================================
@@ -2,5 +2,5 @@ usr/share/java/goby-io.jar:
Class-Path: /usr/share/java/commons-cli.jar /usr/share/java/commons-collections3.jar /usr/share/java/commons-configuration.jar /usr/share/java/commons-exex.jar /usr/share/java/commons-io.jar /usr/share/java/commons-lang.jar /usr/share/java/commons-logging.jar /usr/share/java/commons-math.jar /usr/share/java/dsiutils.jar /usr/share/java/fastutil.jar /usr/share/java/htsjdk.jar /usr/share/java/icb-utils.jar /usr/share/java/jbzip2.jar /usr/lib/R/site-library/rJava/jri/JRI.jar /usr/share/java/jsap.jar /usr/share/java/pj.jar /usr/share/java/protobuf.jar /usr/share/java/reflections.jar /usr/share/java/slf4j.jar
usr/share/java/goby-distribution.jar:
- Class-Path: /usr/share/java/commons-cli.jar /usr/share/java/commons-collections3.jar /usr/share/java/commons-configuration.jar /usr/share/java/commons-exex.jar /usr/share/java/commons-io.jar /usr/share/java/commons-lang.jar /usr/share/java/commons-logging.jar /usr/share/java/commons-math.jar /usr/share/java/dsiutils.jar /usr/share/java/fastutil.jar /usr/share/java/goby-io.jar /usr/share/java/htsjdk.jar /usr/share/java/icb-utils.jar /usr/share/java/jbzip2.jar /usr/lib/R/site-library/rJava/jri/JRI.jar /usr/share/java/jsap.jar /usr/share/java/pj.jar /usr/share/java/protobuf.jar /usr/share/java/reflections.jar /usr/share/java/slf4j.jar
+ Class-Path: /usr/share/java/commons-cli.jar /usr/share/java/commons-collections3.jar /usr/share/java/commons-configuration.jar /usr/share/java/commons-exex.jar /usr/share/java/commons-io.jar /usr/share/java/commons-lang.jar /usr/share/java/commons-logging.jar /usr/share/java/commons-math.jar /usr/share/java/dsiutils.jar /usr/share/java/fastutil.jar /usr/share/java/goby-io.jar /usr/share/java/htsjdk.jar /usr/share/java/icb-utils.jar /usr/share/java/jbzip2.jar /usr/lib/R/site-library/rJava/jri/JRI.jar /usr/share/java/jsap.jar /usr/share/java/pj.jar /usr/share/java/protobuf.jar /usr/share/java/reflections.jar /usr/share/java/slf4j.jar /usr/share/java/distlib.jar
Main-Class: org.campagnelab.goby.modes.GobyDriver
=====================================
debian/patches/computing_Fisher_test_pvalue.patch deleted
=====================================
@@ -1,33 +0,0 @@
-Description: Computing the Fisher p-value with a Debian-packaged tool
-Author: Pierre Gruet <pgt at debian.org>
-Forwarded: not-needed
-Last-Update: 2020-11-01
-
---- a/goby-distribution/src/main/java/org/campagnelab/goby/stats/FisherExactTestCalculator.java
-+++ b/goby-distribution/src/main/java/org/campagnelab/goby/stats/FisherExactTestCalculator.java
-@@ -18,8 +18,8 @@
-
- package org.campagnelab.goby.stats;
-
--import gominer.Fisher;
- import it.unimi.dsi.fastutil.objects.ObjectArraySet;
-+import org.campagnelab.goby.R.FisherExact;
-
- /**
- * Calculates Fisher exact test P-value for an observed count difference between comparison
-@@ -98,8 +98,13 @@
- * @return 2-tailed gominer.Fisher p value
- */
- // public double fisher(final int totalChanged, final int changedInNode, final int total, final int inNode) {
-- final Fisher fisher = new Fisher();
-- final double pValue = fisher.fisher(totalCountInA, sumCountInA, totalCountInA + totalCountInB, sumCountInA + sumCountInB);
-+ int count00 = sumCountInA;
-+ int count10 = totalCountInA - sumCountInA;
-+ int count01 = sumCountInB;
-+ int count11 = sumCountInA + totalCountInB - count00 - count01;
-+ final FisherExact.Result result =
-+ FisherExact.fexact(count00, count10, count01, count11);
-+ final double pValue = result.getPValue();
-
-
- /* Test : fisher.fisher(40,10,100,30)=
=====================================
debian/patches/computing_fisher_pvalue_hypergeom.patch
=====================================
@@ -0,0 +1,145 @@
+Description: computing p-values by hand as gominer is not packaged in Debian
+Author: Pierre Gruet <pgt at debian.org>
+Forwarded: https://github.com/CampagneLaboratory/goby3/issues/4
+Last-Update: 2021-08-05
+
+--- a/goby-distribution/pom.xml
++++ b/goby-distribution/pom.xml
+@@ -382,5 +382,10 @@
+ <artifactId>goby-io</artifactId>
+ <version>${project.version}</version>
+ </dependency>
++ <dependency>
++ <groupId>distlib</groupId>
++ <artifactId>distlib</artifactId>
++ <version>debian</version>
++ </dependency>
+ </dependencies>
+ </project>
+--- a/goby-distribution/src/main/java/org/campagnelab/goby/stats/FisherExactTestCalculator.java
++++ b/goby-distribution/src/main/java/org/campagnelab/goby/stats/FisherExactTestCalculator.java
+@@ -18,7 +18,7 @@
+
+ package org.campagnelab.goby.stats;
+
+-import gominer.Fisher;
++import DistLib.hypergeometric;
+ import it.unimi.dsi.fastutil.objects.ObjectArraySet;
+
+ /**
+@@ -97,9 +97,21 @@
+ * x+y
+ * @return 2-tailed gominer.Fisher p value
+ */
+- // public double fisher(final int totalChanged, final int changedInNode, final int total, final int inNode) {
+- final Fisher fisher = new Fisher();
+- final double pValue = fisher.fisher(totalCountInA, sumCountInA, totalCountInA + totalCountInB, sumCountInA + sumCountInB);
++ //Variables names for the analogy with R arguments.
++ double x = sumCountInA;
++ double whiteBalls = sumCountInA + sumCountInB;
++ double blackBalls = totalCountInA + totalCountInB - whiteBalls;
++ double n = totalCountInA;
++ double sumProba = 0.;
++ //Determining the probability of the given grid under the null hypothesis.
++ double probabilityGrid = hypergeometric.density(x, whiteBalls, blackBalls, n);
++ //Computing the p-value by summing the probabilities of grids that are at least as unlikely as the current one.
++ for (double i = Math.max(0, n - blackBalls) ; i < Math.min(n, whiteBalls) + 1.E-13 ; i++) {
++ double thisProba = hypergeometric.density(i, whiteBalls, blackBalls, n);
++ if (thisProba < probabilityGrid + 1.E-13) {
++ sumProba += thisProba;
++ }
++ }
+
+
+ /* Test : fisher.fisher(40,10,100,30)=
+@@ -113,7 +125,7 @@
+ ------------------------------------------
+ */
+ info.statistics.size(results.getNumberOfStatistics());
+- info.statistics.set(fisherPValuesStatIndex, pValue);
++ info.statistics.set(fisherPValuesStatIndex, sumProba);
+
+ return info;
+ }
+--- a/goby-distribution/src/test/java/org/campagnelab/goby/R/TestFisherExact.java
++++ b/goby-distribution/src/test/java/org/campagnelab/goby/R/TestFisherExact.java
+@@ -18,7 +18,7 @@
+
+ package org.campagnelab.goby.R;
+
+-import gominer.Fisher;
++import DistLib.hypergeometric;
+ import org.apache.commons.lang.ArrayUtils;
+ import org.junit.AfterClass;
+ import static org.junit.Assert.assertEquals;
+@@ -157,11 +157,24 @@
+ */
+ @Test
+ public void twoTailed() {
+- final Fisher gominer = new Fisher();
+- final double gominerPValue = gominer.fisher(40, 10, 100, 30);
++ //Variables names for the analogy with R arguments.
++ double x = 10;
++ double whiteBalls = 30;
++ double blackBalls = 70;
++ double n = 40;
++ double sumProba = 0.;
++ //Determining the probability of the given grid under the null hypothesis.
++ double probabilityGrid = hypergeometric.density(x, whiteBalls, blackBalls, n);
++ //Computing the p-value by summing the probabilities of grids that are at least as unlikely as the current one.
++ for (double i = Math.max(0, n - blackBalls) ; i < Math.min(n, whiteBalls) + 1.E-13 ; i++) {
++ double thisProba = hypergeometric.density(i, whiteBalls, blackBalls, n);
++ if (thisProba < probabilityGrid + 1.E-13) {
++ sumProba += thisProba;
++ }
++ }
+
+ final double fisherExactPValue = FisherExact.twoTailed(40, 10, 100, 30);
+- assertEquals("R result does not match gominer", gominerPValue, fisherExactPValue, EPSILON);
++ assertEquals("R result does not match hand-computed", sumProba, fisherExactPValue, EPSILON);
+ }
+
+ /**
+--- a/goby-distribution/src/test/java/org/campagnelab/goby/stats/TestStatistics.java
++++ b/goby-distribution/src/test/java/org/campagnelab/goby/stats/TestStatistics.java
+@@ -19,7 +19,7 @@
+ package org.campagnelab.goby.stats;
+
+ import org.campagnelab.goby.R.FisherExact;
+-import gominer.Fisher;
++import DistLib.hypergeometric;
+ import it.unimi.dsi.fastutil.objects.ObjectArrayList;
+ import it.unimi.dsi.lang.MutableString;
+ import org.apache.commons.math.MathException;
+@@ -371,15 +371,27 @@
+ assertEquals("fisher test equal expected result", 0.5044757698516504, results.getStatistic(info, fisher.statisticIds.get(0)), 0.001);
+
+
+- final Fisher fisherTest = new Fisher();
+ final int totalCountInA = 1700;
+ final int totalCountInB = 170; // equal total in each group
+ final int sumCountInA = 90;
+ final int sumCountInB = 45; // half the counts in sample B
++ //Variables names for the analogy with R arguments.
++ double x = sumCountInA;
++ double whiteBalls = sumCountInA + sumCountInB;
++ double blackBalls = totalCountInA + totalCountInB - whiteBalls;
++ double n = totalCountInA;
++ double sumProba = 0.;
++ //Determining the probability of the given grid under the null hypothesis.
++ double probabilityGrid = hypergeometric.density(x, whiteBalls, blackBalls, n);
++ //Computing the p-value by summing the probabilities of grids that are at least as unlikely as the current one.
++ for (double i = Math.max(0, n - blackBalls) ; i < Math.min(n, whiteBalls) + 1.E-13 ; i++) {
++ double thisProba = hypergeometric.density(i, whiteBalls, blackBalls, n);
++ if (thisProba < probabilityGrid + 1.E-13) {
++ sumProba += thisProba;
++ }
++ }
+
+- fisherTest.fisher(totalCountInA, sumCountInA, totalCountInA + totalCountInB, sumCountInA + sumCountInB);
+-
+- final double pValue = fisherTest.getTwotail();
++ final double pValue = sumProba;
+ final double proportionTotalA = divide(totalCountInA, (totalCountInA + totalCountInB));
+ final double proportionTotalB = divide(totalCountInB, (totalCountInA + totalCountInB));
+ final ChiSquareTest chisquare = new ChiSquareTestImpl();
=====================================
debian/patches/exclude_not_runnable_tests.patch
=====================================
@@ -5,13 +5,33 @@ Description: modify a few tests when requiring unpackaged tools
We also exclude TestGobyPaperTop5000s.java in this pom, as data files that it
uses are not provided in the source.
Two tests in TestSamRecordParser are skipped for the same reason.
- gominer is not packaged, the tests using it are slightly modified.
One test in TestConcatAlignmentReader needs a file that does not exist.
The tests in TestIterateSortedAlignment all use a file that does not exist.
One test in TestSkipTo needs to use the network.
+ One test in TestStatistics is skipped as it expects a Fisher test p-value
+ greater than 0.05, which seems false after a check with R and also with a
+ script using libdistlib-java.
+ Tests in TestDiscoverSequenceVariantsIndelGenotypes and in
+ TestDiscoverSequenceVariantsMode need genotype.jar or somatic.jar,
+ which are not in this package nor in a dependency.
+ .
+ Also I removed some tests that fail even in a classic Maven build. Maybe this
+ is wrong, but anyway they are not run in the classic Maven build and so I am
+ unsure upstream has kept them up to date. I opened an issue at the link below.
+ Ignored tests are:
+ - three tests in TestConcatSortedAlignmentReader;
+ - all tests in TestIteratedSortedAlignment2;
+ - one test in TestRealignmentProcessor;
+ - all tests in TestDiscoverSVMethylationRatesMode;
+ - five tests in TestDiscoverSequenceVariantsMode;
+ - three tests in TestLastToCompact;
+ - one test in TestReformatCompactReadsMode;
+ - one test in TestSortMode;
+ - two tests in TestCompareGroupsVCFOutputFormat;
+ - two tests in MergeIndelFromTest.
Author: Pierre Gruet <pgt at debian.org>
-Forwarded: no
-Last-Update: 2021-07-31
+Forwarded: https://github.com/CampagneLaboratory/goby3/issues/3
+Last-Update: 2021-08-06
--- a/goby-distribution/pom.xml
+++ b/goby-distribution/pom.xml
@@ -26,7 +46,7 @@ Last-Update: 2021-07-31
</configuration>
</plugin>
<plugin>
-@@ -73,6 +77,17 @@
+@@ -73,6 +77,19 @@
</executions>
</plugin>
<plugin>
@@ -37,6 +57,8 @@ Last-Update: 2021-07-31
+ <excludes>
+ <exclude>**/TestGobyPaperTop5000s.java</exclude>
+ <exclude>**/TestIterateSortedAlignment.java</exclude>
++ <exclude>**/TestIteratedSortedAlignment2.java</exclude>
++ <exclude>**/TestDiscoverSVMethylationRatesMode.java</exclude>
+ </excludes>
+ </configuration>
+ </plugin>
@@ -44,61 +66,6 @@ Last-Update: 2021-07-31
<artifactId>maven-assembly-plugin</artifactId>
<version>2.6</version>
<configuration>
---- a/goby-distribution/src/test/java/org/campagnelab/goby/R/TestFisherExact.java
-+++ b/goby-distribution/src/test/java/org/campagnelab/goby/R/TestFisherExact.java
-@@ -18,7 +18,6 @@
-
- package org.campagnelab.goby.R;
-
--import gominer.Fisher;
- import org.apache.commons.lang.ArrayUtils;
- import org.junit.AfterClass;
- import static org.junit.Assert.assertEquals;
-@@ -151,18 +150,6 @@
- result.getAlternativeHypothesis());
- }
-
-- /**
-- * Validates that the R implementation returns the same p-value for a simple
-- * example set.
-- */
-- @Test
-- public void twoTailed() {
-- final Fisher gominer = new Fisher();
-- final double gominerPValue = gominer.fisher(40, 10, 100, 30);
--
-- final double fisherExactPValue = FisherExact.twoTailed(40, 10, 100, 30);
-- assertEquals("R result does not match gominer", gominerPValue, fisherExactPValue, EPSILON);
-- }
-
- /**
- * If R libraries are not set up properly these tests cannot be run..
---- a/goby-distribution/src/test/java/org/campagnelab/goby/stats/TestStatistics.java
-+++ b/goby-distribution/src/test/java/org/campagnelab/goby/stats/TestStatistics.java
-@@ -19,7 +19,6 @@
- package org.campagnelab.goby.stats;
-
- import org.campagnelab.goby.R.FisherExact;
--import gominer.Fisher;
- import it.unimi.dsi.fastutil.objects.ObjectArrayList;
- import it.unimi.dsi.lang.MutableString;
- import org.apache.commons.math.MathException;
-@@ -371,15 +370,11 @@
- assertEquals("fisher test equal expected result", 0.5044757698516504, results.getStatistic(info, fisher.statisticIds.get(0)), 0.001);
-
-
-- final Fisher fisherTest = new Fisher();
- final int totalCountInA = 1700;
- final int totalCountInB = 170; // equal total in each group
- final int sumCountInA = 90;
- final int sumCountInB = 45; // half the counts in sample B
-
-- fisherTest.fisher(totalCountInA, sumCountInA, totalCountInA + totalCountInB, sumCountInA + sumCountInB);
--
-- final double pValue = fisherTest.getTwotail();
- final double proportionTotalA = divide(totalCountInA, (totalCountInA + totalCountInB));
- final double proportionTotalB = divide(totalCountInB, (totalCountInA + totalCountInB));
- final ChiSquareTest chisquare = new ChiSquareTestImpl();
--- a/goby-distribution/src/test/java/org/campagnelab/goby/readers/sam/TestSamRecordParser.java
+++ b/goby-distribution/src/test/java/org/campagnelab/goby/readers/sam/TestSamRecordParser.java
@@ -29,6 +29,7 @@
@@ -161,3 +128,286 @@ Last-Update: 2021-07-31
@Test
public void testUrlRange() throws IOException {
+--- a/goby-distribution/src/test/java/org/campagnelab/goby/stats/TestStatistics.java
++++ b/goby-distribution/src/test/java/org/campagnelab/goby/stats/TestStatistics.java
+@@ -25,6 +25,7 @@
+ import org.apache.commons.math.MathException;
+ import org.apache.commons.math.stat.inference.ChiSquareTest;
+ import org.apache.commons.math.stat.inference.ChiSquareTestImpl;
++import org.junit.Ignore;
+ import org.junit.Test;
+
+ import java.io.IOException;
+@@ -120,6 +121,7 @@
+ results.write(new PrintWriter("test-results/out-stats.tsv"), '\t', deCalc);
+ }
+
++ @Ignore
+ @Test
+ public void testTwoStats() {
+ final Random randomEngine = new Random(37);
+--- a/goby-distribution/src/test/java/org/campagnelab/goby/alignments/TestConcatSortedAlignmentReader.java
++++ b/goby-distribution/src/test/java/org/campagnelab/goby/alignments/TestConcatSortedAlignmentReader.java
+@@ -30,6 +30,7 @@
+
+ import org.junit.Before;
+ import org.junit.BeforeClass;
++import org.junit.Ignore;
+ import org.junit.Test;
+
+ import java.io.File;
+@@ -65,6 +66,7 @@
+ }
+
+
++ @Ignore
+ @Test
+ public void testSortConcatWithGenomicRange() throws IOException {
+ final ConcatSortedAlignmentReader concat = new ConcatSortedAlignmentReader(basename1, basename2, basename3);
+@@ -102,6 +104,7 @@
+ }
+
+
++ @Ignore
+ @Test
+ public void testSortConcatWithGenomicRangeSmaller() throws IOException {
+ final ConcatSortedAlignmentReader concat = new ConcatSortedAlignmentReader(basename1, basename2, basename3);
+@@ -120,6 +123,7 @@
+ }
+ }
+
++ @Ignore
+ @Test
+ public void testSortConcatWithGenomicRangeSmaller2() throws IOException {
+ final ConcatSortedAlignmentReader concat = new ConcatSortedAlignmentReader(basename1, basename2, basename3);
+--- a/goby-distribution/src/test/java/org/campagnelab/goby/alignments/TestRealignmentProcessor.java
++++ b/goby-distribution/src/test/java/org/campagnelab/goby/alignments/TestRealignmentProcessor.java
+@@ -32,6 +32,7 @@
+ import org.campagnelab.goby.modes.AbstractAlignmentToCompactMode;
+ import org.campagnelab.goby.reads.RandomAccessSequenceInterface;
+ import org.campagnelab.goby.reads.RandomAccessSequenceTestSupport;
++import org.junit.Ignore;
+ import org.junit.Test;
+
+ import java.io.IOException;
+@@ -435,6 +436,7 @@
+ }
+ }
+
++ @Ignore
+ @Test
+ public void testCase9() {
+ ObjectListIterator<Alignments.AlignmentEntry> iterator = new ObjectArrayList().iterator();
+--- a/goby-distribution/src/test/java/org/campagnelab/goby/modes/TestDiscoverSequenceVariantsIndelGenotypes.java
++++ b/goby-distribution/src/test/java/org/campagnelab/goby/modes/TestDiscoverSequenceVariantsIndelGenotypes.java
+@@ -36,6 +36,7 @@
+ import org.junit.AfterClass;
+ import org.junit.Before;
+ import org.junit.BeforeClass;
++import org.junit.Ignore;
+ import org.junit.Test;
+
+ import java.io.*;
+@@ -77,6 +78,7 @@
+ mode.setTestGenome(genomeForTest);
+ }
+
++ @Ignore
+ @Test
+ public void testDiscoverWithIndelSamples() throws IOException, JSAPException {
+
+--- a/goby-distribution/src/test/java/org/campagnelab/goby/modes/TestDiscoverSequenceVariantsMode.java
++++ b/goby-distribution/src/test/java/org/campagnelab/goby/modes/TestDiscoverSequenceVariantsMode.java
+@@ -49,6 +49,7 @@
+ import org.junit.AfterClass;
+ import org.junit.Before;
+ import org.junit.BeforeClass;
++import org.junit.Ignore;
+ import org.junit.Test;
+
+ import java.io.*;
+@@ -81,6 +82,7 @@
+ // assertNotNull("R engine must be available", GobyRengine.getInstance().getRengine());
+ }
+
++ @Ignore
+ @Test
+ public void testDiscoverSomaticVariation() throws IOException, JSAPException {
+
+@@ -212,6 +214,7 @@
+ return dest;
+ }
+
++ @Ignore
+ @Test
+ public void testDiscoverAlleleImbalance() throws IOException, JSAPException {
+
+@@ -242,6 +245,7 @@
+
+ }
+
++ @Ignore
+ @Test
+ public void testDiscoverCheckSampleCountAssociation() throws IOException, JSAPException {
+
+@@ -552,6 +556,7 @@
+ }
+
+
++ @Ignore
+ @Test
+ public void testGenotypes() throws IOException, JSAPException {
+ DiscoverSequenceVariantsMode mode = new DiscoverSequenceVariantsMode();
+@@ -572,6 +577,7 @@
+
+ }
+
++ @Ignore
+ @Test
+ public void testQualityScoreAdjuster() {
+ QualityScoreFilter adjuster = new QualityScoreFilter();
+@@ -593,6 +599,7 @@
+
+ }
+
++ @Ignore
+ @Test
+ public void testAdjustVarCount() {
+ QualityScoreFilter adjuster1 = new QualityScoreFilter();
+@@ -641,6 +648,7 @@
+ }
+
+
++ @Ignore
+ @Test
+ public void testAdjustVarCount2() {
+ QualityScoreFilter adjuster1 = new QualityScoreFilter();
+--- a/goby-distribution/src/test/java/org/campagnelab/goby/modes/TestLastToCompact.java
++++ b/goby-distribution/src/test/java/org/campagnelab/goby/modes/TestLastToCompact.java
+@@ -32,6 +32,7 @@
+ import static org.junit.Assert.assertTrue;
+
+ import org.junit.Before;
++import org.junit.Ignore;
+ import org.junit.Test;
+
+ import java.io.File;
+@@ -39,6 +40,7 @@
+ import java.io.IOException;
+
+ public class TestLastToCompact {
++ @Ignore
+ @Test
+ public void testLastToCompact1() throws IOException {
+
+@@ -154,6 +156,7 @@
+ reader.close();
+ }
+
++ @Ignore
+ @Test
+ public void testLastToCompact2() throws IOException {
+
+@@ -652,6 +655,7 @@
+ *
+ * @throws IOException
+ */
++ @Ignore
+ @Test
+ public void testLastToCompactVariations() throws IOException {
+
+--- a/goby-distribution/src/test/java/org/campagnelab/goby/modes/TestReformatCompactReadsMode.java
++++ b/goby-distribution/src/test/java/org/campagnelab/goby/modes/TestReformatCompactReadsMode.java
+@@ -22,6 +22,7 @@
+ import org.campagnelab.goby.reads.ReadsReader;
+ import org.campagnelab.goby.reads.ReadsToTextWriter;
+ import org.apache.commons.io.FileUtils;
++import org.junit.Ignore;
+ import org.junit.Test;
+
+ import java.io.File;
+@@ -385,6 +386,7 @@
+ *
+ * @throws IOException if there is a problem reading or writing to the files
+ */
++ @Ignore
+ @Test
+ public void reformatTransferMetaData() throws IOException {
+ final ReformatCompactReadsMode reformat = new ReformatCompactReadsMode();
+--- a/goby-distribution/src/test/java/org/campagnelab/goby/modes/TestSortMode.java
++++ b/goby-distribution/src/test/java/org/campagnelab/goby/modes/TestSortMode.java
+@@ -24,6 +24,7 @@
+ import org.apache.commons.logging.LogFactory;
+ import org.junit.AfterClass;
+ import org.junit.BeforeClass;
++import org.junit.Ignore;
+ import org.junit.Test;
+
+ import java.io.File;
+@@ -60,6 +61,7 @@
+ // FileUtils.forceDeleteOnExit(new File(BASE_TEST_DIR));
+ }
+
++ @Ignore
+ @Test
+ // check that large-sort can sort a small alignment (only one split):
+ public void sortSmall() throws IOException {
+--- a/goby-distribution/src/test/java/org/campagnelab/goby/modes/formats/TestCompareGroupsVCFOutputFormat.java
++++ b/goby-distribution/src/test/java/org/campagnelab/goby/modes/formats/TestCompareGroupsVCFOutputFormat.java
+@@ -33,6 +33,7 @@
+ import org.easymock.EasyMock;
+ import org.junit.Before;
+ import org.junit.BeforeClass;
++import org.junit.Ignore;
+ import org.junit.Test;
+
+ import java.io.PrintWriter;
+@@ -128,6 +129,7 @@
+ }
+
+
++ @Ignore
+ @Test
+ public void testNoDifference() throws Exception {
+ synchronized (GobyRengine.getInstance().getRengine()) {
+@@ -154,6 +156,7 @@
+ }
+
+
++ @Ignore
+ @Test
+ public void testAllelicDifference() throws Exception {
+ synchronized (GobyRengine.getInstance().getRengine()) {
+--- a/goby-distribution/src/test/java/org/campagnelab/goby/predictions/MergeIndelFromTest.java
++++ b/goby-distribution/src/test/java/org/campagnelab/goby/predictions/MergeIndelFromTest.java
+@@ -2,6 +2,7 @@
+
+ import it.unimi.dsi.fastutil.objects.ObjectArraySet;
+ import org.campagnelab.goby.util.Variant;
++import org.junit.Ignore;
+ import org.junit.Test;
+
+ import java.util.Set;
+@@ -59,6 +60,7 @@
+ */
+
+
++ @Ignore
+ @Test
+ public void longestTailInDeletion() {
+ final Set<Variant.FromTo> fromTos = new ObjectArraySet<>(4);
+@@ -128,6 +130,7 @@
+ */
+
+
++ @Ignore
+ @Test
+ public void longestTailInInsertion() {
+ final Set<Variant.FromTo> fromTos = new ObjectArraySet<>(4);
+@@ -183,4 +186,4 @@
+ assertNotNull(merge.mapped(originalAllele));
+ }
+ }
+-}
+\ No newline at end of file
++}
=====================================
debian/patches/series
=====================================
@@ -5,7 +5,7 @@ adapting_to_old_fastutil.patch
using_GeneratedMessageV3.patch
using_commons-cli.patch
using_correct_SamReader_api.patch
-computing_Fisher_test_pvalue.patch
inclusions_in_SplitTranscriptsMode.patch
catch_IOException_LineIterator.patch
+computing_fisher_pvalue_hypergeom.patch
exclude_not_runnable_tests.patch
=====================================
debian/upstream/metadata
=====================================
@@ -0,0 +1,4 @@
+Bug-Database: https://github.com/CampagneLaboratory/goby3/issues
+Bug-Submit: https://github.com/CampagneLaboratory/goby3/issues/new
+Repository: https://github.com/CampagneLaboratory/goby3.git
+Repository-Browse: https://github.com/CampagneLaboratory/goby3
=====================================
debian/watch
=====================================
@@ -1,4 +1,4 @@
version=4
opts="repacksuffix=+dfsg,dversionmangle=s/\+dfsg//g,repack,compression=xz" \
- https://github.com/CampagneLaboratory/goby3/releases .*/archive/r*@ANY_VERSION@@ARCHIVE_EXT@
+ https://github.com/CampagneLaboratory/goby3/releases (?:.*?/)?r at ANY_VERSION@@ARCHIVE_EXT@
View it on GitLab: https://salsa.debian.org/med-team/libgoby-java/-/compare/1f0226ab043686355a13f957d4060a95c7529140...48f2769114ecf743030944c266cd17ad8424f347
--
View it on GitLab: https://salsa.debian.org/med-team/libgoby-java/-/compare/1f0226ab043686355a13f957d4060a95c7529140...48f2769114ecf743030944c266cd17ad8424f347
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/debian-med-commit/attachments/20210806/32478973/attachment-0001.htm>
More information about the debian-med-commit
mailing list