[med-svn] [Git][med-team/snakemake][debian_for_upstream_5.14.0_which_requires_conda] 3 commits: fix link_security.patch (needs both upstream commits,
Rebecca N. Palmer
gitlab at salsa.debian.org
Sat Apr 25 20:11:53 BST 2020
Rebecca N. Palmer pushed to branch debian_for_upstream_5.14.0_which_requires_conda at Debian Med / snakemake
Commits:
ae057049 by Rebecca N. Palmer at 2020-04-25T15:53:53+01:00
fix link_security.patch (needs both upstream commits,
and overlaps with local_javascript.patch)
- - - - -
d5c191a1 by Rebecca N. Palmer at 2020-04-25T17:35:43+01:00
Remove inconveniently licensed (CC-BY-SA) snippets.
- - - - -
ff24eb24 by Rebecca N. Palmer at 2020-04-25T19:01:32+01:00
Install examples.
- - - - -
7 changed files:
- debian/changelog
- debian/copyright
- debian/patches/link_security.patch
- debian/patches/local_javascript.patch
- + debian/patches/remove_ccbysa_snippets.patch
- debian/patches/series
- + debian/snakemake.examples
Changes:
=====================================
debian/changelog
=====================================
@@ -22,6 +22,8 @@ snakemake (5.14.0-1) UNRELEASED; urgency=medium
* Privacy: use local dagre-d3, add missing Javascript dependencies,
don't appear to show live status when we don't.
* Use https for links where available.
+ * Remove inconveniently licensed (CC-BY-SA) snippets.
+ * Install examples.
-- Rebecca N. Palmer <rebecca_palmer at zoho.com> Thu, 09 Apr 2020 14:57:00 +0100
=====================================
debian/copyright
=====================================
@@ -6,6 +6,7 @@ Source: https://github.com/snakemake/snakemake
Files: *
Copyright: 2015-2020 Johannes Köster <johannes.koester at tu-dortmund.de>, Manuel Holtgrewe, Christopher Tomkins-Tinch, Sven Nahnsen, Sebastian Kurscheid and contributors
License: Expat
+Comment: The original source contains some CC-BY-SA snippets by Stack Overflow users (https://stackoverflow.com/users/2012682/brice-m-dempsey, https://stackoverflow.com/users/650222/coldfix, https://stackoverflow.com/users/14343/ned-batchelder, https://stackoverflow.com/users/5299236/kevin-guan, https://stackoverflow.com/users/20713/ephemient, https://stackoverflow.com/users/34910/nate-parsons, https://stackoverflow.com/users/4785947/christian-zielinski, https://stackoverflow.com/users/3743213/seth127, https://stackoverflow.com/users/2838606/amir, https://stackoverflow.com/users/4159907/schoyen, https://stackoverflow.com/users/95810/alex-martelli, https://stackoverflow.com/users/1723886/alex-telon, https://stackoverflow.com/users/1417016/wpoely86, https://stackoverflow.com/users/97828/nadia-alramli, https://stackoverflow.com/users/35070/phihag), but these may be too small to be copyrightable, and the less trivial ones are patched out in this package
Files: tests/test_script/expected-results/test.html
tests/test_issue635/expected-results/report.html
=====================================
debian/patches/link_security.patch
=====================================
@@ -81,7 +81,8 @@ index 164f6a3e..a448f1f0 100644
+* Park et al. 2015. `Ebola Virus Epidemiology, Transmission, and Evolution during Seven Months in Sierra Leone <https://doi.org/10.1016/j.cell.2015.06.007>`_. Cell.
+* Břinda et al. 2015. `RNF: a general framework to evaluate NGS read mappers <https://bioinformatics.oxfordjournals.org/content/early/2015/09/30/bioinformatics.btv524>`_. Bioinformatics.
+* Břinda et al. 2015. `Spaced seeds improve k-mer-based metagenomic classification <https://bioinformatics.oxfordjournals.org/content/early/2015/08/10/bioinformatics.btv419>`_. Bioinformatics.
- * Spjuth et al. 2015. `Experiences with workflows for automating data-intensive bioinformatics <https://biologydirect.biomedcentral.com/articles/10.1186/s13062-015-0071-8>`_. Biology Direct.
+-* Spjuth et al. 2015. `Experiences with workflows for automating data-intensive bioinformatics <http://www.biologydirect.com/content/10/1/43>`_. Biology Direct.
++* Spjuth et al. 2015. `Experiences with workflows for automating data-intensive bioinformatics <https://biologydirect.biomedcentral.com/articles/10.1186/s13062-015-0071-8>`_. Biology Direct.
-* Schramm et al. 2015. `Mutational dynamics between primary and relapse neuroblastomas <http://www.nature.com/ng/journal/v47/n8/full/ng.3349.html>`_. Nature Genetics.
-* Berulava et al. 2015. `N6-Adenosine Methylation in MiRNAs <http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0118438>`_. PLOS ONE.
-* The Genome of the Netherlands Consortium 2014. `Whole-genome sequence variation, population structure and demographic history of the Dutch population <http://www.nature.com/ng/journal/v46/n8/full/ng.3021.html>`_. Nature Genetics.
@@ -346,21 +347,24 @@ index 54d1178e..af3973c7 100644
.. _Atom: https://atom.io
.. _Anaconda: https://anaconda.org
-.. _Graphviz: http://www.graphviz.org
+-.. _RestructuredText: http://docutils.sourceforge.net/rst.html
+.. _Graphviz: https://www.graphviz.org
- .. _RestructuredText: https://docutils.sourceforge.io/docs/user/rst/quickstart.html
++.. _RestructuredText: https://docutils.sourceforge.io/docs/user/rst/quickstart.html
.. _data URI: https://developer.mozilla.org/en-US/docs/Web/HTTP/data_URIs
-.. _JSON: http://json.org
-.. _YAML: http://yaml.org
-.. _DRMAA: http://www.drmaa.org
+-.. _rpy2: http://rpy.sourceforge.net
+.. _JSON: https://json.org
+.. _YAML: https://yaml.org
+.. _DRMAA: https://www.drmaa.org
- .. _rpy2: https://rpy2.github.io
++.. _rpy2: https://rpy2.github.io
.. _R: https://www.r-project.org
.. _Rscript: https://stat.ethz.ch/R-manual/R-devel/library/utils/html/Rscript.html
-.. _PyYAML: http://pyyaml.org
+-.. _Docutils: http://docutils.sourceforge.net
+.. _PyYAML: https://pyyaml.org
- .. _Docutils: https://docutils.sourceforge.io
++.. _Docutils: https://docutils.sourceforge.io
.. _Bioconda: https://bioconda.github.io
.. _Vagrant: https://www.vagrantup.com
.. _Vagrant Documentation: https://docs.vagrantup.com
@@ -406,21 +410,24 @@ index f7ead578..03a91988 100644
.. _Atom: https://atom.io
.. _Anaconda: https://anaconda.org
-.. _Graphviz: http://www.graphviz.org
+-.. _RestructuredText: http://docutils.sourceforge.net/rst.html
+.. _Graphviz: https://www.graphviz.org
- .. _RestructuredText: https://docutils.sourceforge.io/docs/user/rst/quickstart.html
++.. _RestructuredText: https://docutils.sourceforge.io/docs/user/rst/quickstart.html
.. _data URI: https://developer.mozilla.org/en-US/docs/Web/HTTP/data_URIs
-.. _JSON: http://json.org
-.. _YAML: http://yaml.org
-.. _DRMAA: http://www.drmaa.org
+-.. _rpy2: http://rpy.sourceforge.net
+.. _JSON: https://json.org
+.. _YAML: https://yaml.org
+.. _DRMAA: https://www.drmaa.org
- .. _rpy2: https://rpy2.github.io
++.. _rpy2: https://rpy2.github.io
.. _R: https://www.r-project.org
.. _Rscript: https://stat.ethz.ch/R-manual/R-devel/library/utils/html/Rscript.html
-.. _PyYAML: http://pyyaml.org
+-.. _Docutils: http://docutils.sourceforge.net
+.. _PyYAML: https://pyyaml.org
- .. _Docutils: https://docutils.sourceforge.io
++.. _Docutils: https://docutils.sourceforge.io
.. _Bioconda: https://bioconda.github.io
.. _Vagrant: https://www.vagrantup.com
.. _Vagrant Documentation: https://docs.vagrantup.com
@@ -457,21 +464,24 @@ index cc1cfb45..fe97f775 100644
.. _Atom: https://atom.io
.. _Anaconda: https://anaconda.org
-.. _Graphviz: http://www.graphviz.org
+-.. _RestructuredText: http://docutils.sourceforge.net/rst.html
+.. _Graphviz: https://www.graphviz.org
- .. _RestructuredText: https://docutils.sourceforge.io/docs/user/rst/quickstart.html
++.. _RestructuredText: https://docutils.sourceforge.io/docs/user/rst/quickstart.html
.. _data URI: https://developer.mozilla.org/en-US/docs/Web/HTTP/data_URIs
-.. _JSON: http://json.org
-.. _YAML: http://yaml.org
-.. _DRMAA: http://www.drmaa.org
+-.. _rpy2: http://rpy.sourceforge.net
+.. _JSON: https://json.org
+.. _YAML: https://yaml.org
+.. _DRMAA: https://www.drmaa.org
- .. _rpy2: https://rpy2.github.io
++.. _rpy2: https://rpy2.github.io
.. _R: https://www.r-project.org
.. _Rscript: https://stat.ethz.ch/R-manual/R-devel/library/utils/html/Rscript.html
-.. _PyYAML: http://pyyaml.org
+-.. _Docutils: http://docutils.sourceforge.net
+.. _PyYAML: https://pyyaml.org
- .. _Docutils: https://docutils.sourceforge.io
++.. _Docutils: https://docutils.sourceforge.io
.. _Bioconda: https://bioconda.github.io
.. _Vagrant: https://www.vagrantup.com
.. _Vagrant Documentation: https://docs.vagrantup.com
@@ -486,7 +496,7 @@ diff --git a/docs/tutorial/setup.rst b/docs/tutorial/setup.rst
index 32ff8ca3..a098bc28 100644
--- a/docs/tutorial/setup.rst
+++ b/docs/tutorial/setup.rst
-@@ -4,20 +4,20 @@
+@@ -4,29 +4,29 @@
Setup
-----
@@ -513,12 +523,15 @@ index 32ff8ca3..a098bc28 100644
.. _Atom: https://atom.io
-.. _Graphviz: http://www.graphviz.org
-.. _PyYAML: http://pyyaml.org
+-.. _Docutils: http://docutils.sourceforge.net
+-.. _Jinja2: http://jinja.pocoo.org
+.. _Graphviz: https://www.graphviz.org
+.. _PyYAML: https://pyyaml.org
- .. _Docutils: https://docutils.sourceforge.io
- .. _Jinja2: https://jinja.palletsprojects.com
++.. _Docutils: https://docutils.sourceforge.io
++.. _Jinja2: https://jinja.palletsprojects.com
.. _NetworkX: https://networkx.github.io
-@@ -26,7 +26,7 @@ Setup
+ .. _Matplotlib: https://matplotlib.org
+ .. _Pysam: https://pysam.readthedocs.io
.. _Bioconda: https://bioconda.github.io
.. _Vagrant: https://www.vagrantup.com
.. _Vagrant Documentation: https://docs.vagrantup.com
=====================================
debian/patches/local_javascript.patch
=====================================
@@ -26,18 +26,18 @@ Forwarded: no
<html lang="en">
<head>
<title>Snakemake</title>
-- <script type="text/javascript" src="http://cdnjs.cloudflare.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
+- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
+ <script type="text/javascript" src="file:///usr/share/javascript/jquery/jquery.min.js"></script>
-- <script type="text/javascript" src="http://cdnjs.cloudflare.com/ajax/libs/d3/3.4.6/d3.min.js"></script>
+- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.4.6/d3.min.js"></script>
+ <script type="text/javascript" src="file:///usr/share/javascript/d3/d3.min.js"></script>
-- <script type="text/javascript" src="http://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.1.1/js/bootstrap.min.js"></script>
+- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.1.1/js/bootstrap.min.js"></script>
+ <script type="text/javascript" src="file:///usr/share/javascript/bootstrap/js/bootstrap.min.js"></script>
-- <script type="text/javascript" src="http://cpettitt.github.io/project/dagre-d3/v0.1.5/dagre-d3.js"></script>
+- <script type="text/javascript" src="https://cpettitt.github.io/project/dagre-d3/v0.1.5/dagre-d3.js"></script>
+ <script type="text/javascript" src="file:///usr/share/nodejs/dagre-d3/dist/dagre-d3.min.js"></script>
- <script type="text/javascript" src="http://cdnjs.cloudflare.com/ajax/libs/bootstrap-select/1.5.4/bootstrap-select.min.js"></script>
-- <link rel="stylesheet" type="text/css" href="http://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.1.1/css/bootstrap.min.css"/>
+ <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/bootstrap-select/1.5.4/bootstrap-select.min.js"></script>
+- <link rel="stylesheet" type="text/css" href="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.1.1/css/bootstrap.min.css"/>
+ <link rel="stylesheet" type="text/css" href="file:///usr/share/javascript/bootstrap/css/bootstrap.min.css"/>
- <link rel="stylesheet" type="text/css" href="http://cdnjs.cloudflare.com/ajax/libs/bootstrap-select/1.5.4/bootstrap-select.min.css"/>
+ <link rel="stylesheet" type="text/css" href="https://cdnjs.cloudflare.com/ajax/libs/bootstrap-select/1.5.4/bootstrap-select.min.css"/>
<style>
body {
--- a/docs/index.rst
=====================================
debian/patches/remove_ccbysa_snippets.patch
=====================================
@@ -0,0 +1,193 @@
+Description: Remove code from Stack Overflow
+
+Stack Overflow content is CC-BY-SA licensed,
+which this package is not supposed to be. These snippets may be
+too small to be copyrightable, but removing them to be safe.
+
+Author: Rebecca N. Palmer <rebecca_palmer at zoho.com>
+Forwarded: no - one part needs Python 3.6+
+
+diff --git a/snakemake/io.py b/snakemake/io.py
+index ff149c3..8fd767b 100755
+--- a/snakemake/io.py
++++ b/snakemake/io.py
+@@ -1355,18 +1355,7 @@ def _load_configfile(configpath, filetype="Config"):
+ except ValueError:
+ f.seek(0) # try again
+ try:
+- # From https://stackoverflow.com/a/21912744/84349
+- class OrderedLoader(yaml.Loader):
+- pass
+-
+- def construct_mapping(loader, node):
+- loader.flatten_mapping(node)
+- return collections.OrderedDict(loader.construct_pairs(node))
+-
+- OrderedLoader.add_constructor(
+- yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG, construct_mapping
+- )
+- return yaml.load(f, Loader=OrderedLoader)
++ return yaml.full_load(f)
+ except yaml.YAMLError:
+ raise WorkflowError(
+ "Config file is not valid JSON or YAML. "
+diff --git a/snakemake/remote/S3Mocked.py b/snakemake/remote/S3Mocked.py
+index 81391ea..1b7646e 100644
+--- a/snakemake/remote/S3Mocked.py
++++ b/snakemake/remote/S3Mocked.py
+@@ -10,6 +10,7 @@ import pickle
+ import time
+ import threading
+ import functools
++import pathlib
+
+ # intra-module
+ from snakemake.remote.S3 import (
+@@ -122,26 +123,19 @@ class RemoteObject(S3RemoteObject):
+ # ====== Helpers =====
+
+
+-def touch(fname, mode=0o666, dir_fd=None, **kwargs):
+- # create lock file faster
+- # https://stackoverflow.com/a/1160227
+- flags = os.O_CREAT | os.O_APPEND
+- with os.fdopen(os.open(fname, flags=flags, mode=mode, dir_fd=dir_fd)) as f:
+- os.utime(
+- f.fileno() if os.utime in os.supports_fd else fname,
+- dir_fd=None if os.supports_fd else dir_fd,
+- **kwargs
+- )
+-
++def touch(fname, mode=0o666):
++ pathlib.Path(fname).touch(mode=mode, exist_ok=True)
+
+ @contextmanager
+ def file_lock(filepath):
+ lock_file = filepath + ".lock"
+
+- while os.path.isfile(lock_file):
+- time.sleep(2)
+-
+- touch(lock_file)
++ while True:
++ try:
++ pathlib.Path(lock_file).touch(exist_ok=False)
++ break
++ except FileExistsError:
++ time.sleep(2)
+
+ try:
+ yield
+diff --git a/snakemake/utils.py b/snakemake/utils.py
+index 451c25c..5750a7a 100644
+--- a/snakemake/utils.py
++++ b/snakemake/utils.py
+@@ -449,25 +449,23 @@ def min_version(version):
+ def update_config(config, overwrite_config):
+ """Recursively update dictionary config with overwrite_config.
+
+- See
+- https://stackoverflow.com/questions/3232943/update-value-of-a-nested-dictionary-of-varying-depth
+- for details.
+-
+ Args:
+ config (dict): dictionary to update
+ overwrite_config (dict): dictionary whose items will overwrite those in config
+
+ """
+
+- def _update(d, u):
+- for (key, value) in u.items():
+- if isinstance(value, collections.abc.Mapping):
+- d[key] = _update(d.get(key, {}), value)
++ def nestedupdate(old_d, new_d):
++ for k, v in new_d.items():
++ if isinstance(v, collections.abc.Mapping):
++ if k not in old_d:
++ old_d[k] = {}
++ nestedupdate(old_d[k], v)
+ else:
+- d[key] = value
+- return d
++ old_d[k] = v
++
+
+- _update(config, overwrite_config)
++ nestedupdate(config, overwrite_config)
+
+
+ def available_cpu_count():
+@@ -476,22 +472,16 @@ def available_cpu_count():
+ The number of available CPUs can be smaller than the total number of CPUs
+ when the cpuset(7) mechanism is in use, as is the case on some cluster
+ systems.
+-
+- Adapted from https://stackoverflow.com/a/1006301/715090
+ """
+ try:
+- with open("/proc/self/status") as f:
+- status = f.read()
+- m = re.search(r"(?m)^Cpus_allowed:\s*(.*)$", status)
+- if m:
+- res = bin(int(m.group(1).replace(",", ""), 16)).count("1")
+- if res > 0:
+- return min(res, multiprocessing.cpu_count())
+- except IOError:
+- pass
+-
+- return multiprocessing.cpu_count()
+-
++ cpu_count = len(os.sched_getaffinity(0))
++ except Exception:
++ cpu_count = 0
++ if cpu_count == 0:
++ cpu_count = os.cpu_count()
++ if not cpu_count: # None if unknown
++ cpu_count = 1
++ return cpu_count
+
+ def argvquote(arg, force=True):
+ """ Returns an argument quoted in such a way that that CommandLineToArgvW
+diff --git a/tests/test_static_remote/S3MockedForStaticTest.py b/tests/test_static_remote/S3MockedForStaticTest.py
+index 4f6c343..42403f4 100644
+--- a/tests/test_static_remote/S3MockedForStaticTest.py
++++ b/tests/test_static_remote/S3MockedForStaticTest.py
+@@ -10,6 +10,7 @@ import pickle
+ import time
+ import threading
+ import functools
++import pathlib
+
+ # intra-module
+ from snakemake.remote.S3 import RemoteObject as S3RemoteObject, RemoteProvider as S3RemoteProvider
+@@ -116,22 +117,19 @@ class RemoteObject(S3RemoteObject):
+
+ # ====== Helpers =====
+
+-def touch(fname, mode=0o666, dir_fd=None, **kwargs):
+- # create lock file faster
+- # https://stackoverflow.com/a/1160227
+- flags = os.O_CREAT | os.O_APPEND
+- with os.fdopen(os.open(fname, flags=flags, mode=mode, dir_fd=dir_fd)) as f:
+- os.utime(f.fileno() if os.utime in os.supports_fd else fname,
+- dir_fd=None if os.supports_fd else dir_fd, **kwargs)
++def touch(fname, mode=0o666):
++ pathlib.Path(fname).touch(mode=mode, exist_ok=True)
+
+ @contextmanager
+ def file_lock(filepath):
+ lock_file = filepath + ".lock"
+
+- while os.path.isfile(lock_file):
+- time.sleep(2)
+-
+- touch(lock_file)
++ while True:
++ try:
++ pathlib.Path(lock_file).touch(exist_ok=False)
++ break
++ except FileExistsError:
++ time.sleep(2)
+
+ try:
+ yield
=====================================
debian/patches/series
=====================================
@@ -1,4 +1,5 @@
0001-Use-the-inbuild-sphinx.ext.napoleon.patch
+link_security.patch
local_javascript.patch
0003-Compat-fix.patch
0006-restore-bin.patch
@@ -6,5 +7,5 @@ local_javascript.patch
0009-skip-test-without-google-cloud-sdk.patch
python3.patch
workaround_sphinx_issue.patch
-link_security.patch
bug322_cluster_sync.patch
+remove_ccbysa_snippets.patch
=====================================
debian/snakemake.examples
=====================================
@@ -0,0 +1 @@
+examples/*
View it on GitLab: https://salsa.debian.org/med-team/snakemake/-/compare/7d50f8d318478b6bb767b008c540da954f4fc92a...ff24eb24855ae1cf2e52bd66f4a0d41d66858d01
--
View it on GitLab: https://salsa.debian.org/med-team/snakemake/-/compare/7d50f8d318478b6bb767b008c540da954f4fc92a...ff24eb24855ae1cf2e52bd66f4a0d41d66858d01
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/debian-med-commit/attachments/20200425/eb575f59/attachment-0001.html>
More information about the debian-med-commit
mailing list