[med-svn] [Git][med-team/snakemake][master] 3 commits: Merge branch 'upstream' of salsa.debian.org:med-team/snakemake into upstream
Rebecca N. Palmer (@rnpalmer-guest)
gitlab at salsa.debian.org
Sun Feb 20 13:45:57 GMT 2022
Rebecca N. Palmer pushed to branch master at Debian Med / snakemake
Commits:
bebee42a by Rebecca N. Palmer at 2022-02-05T21:21:48+00:00
Merge branch 'upstream' of salsa.debian.org:med-team/snakemake into upstream
- - - - -
4e0aafe4 by Rebecca N. Palmer at 2022-02-20T13:29:46+00:00
New upstream version 6.15.5
- - - - -
1f41139d by Rebecca N. Palmer at 2022-02-20T13:30:09+00:00
Update upstream source from tag 'upstream/6.15.5'
Update to upstream version '6.15.5'
with Debian dir 3a45910786b1d5df9c0a34cc55776e6cbae9db3c
- - - - -
30 changed files:
- CHANGELOG.md
- docs/getting_started/installation.rst
- docs/snakefiles/deployment.rst
- docs/tutorial/setup.rst
- snakemake/_version.py
- snakemake/deployment/conda.py
- snakemake/linting/__init__.py
- snakemake/logging.py
- snakemake/modules.py
- snakemake/parser.py
- snakemake/remote/XRootD.py
- snakemake/report/__init__.py
- snakemake/rules.py
- snakemake/unit_tests/__init__.py
- snakemake/unit_tests/templates/ruletest.py.jinja2
- snakemake/workflow.py
- tests/test_conda/Snakefile
- + tests/test_github_issue1384/Snakefile
- + tests/test_github_issue1384/expected-results/.gitempty
- tests/test_io.py
- + tests/test_modules_peppy/Snakefile
- + tests/test_modules_peppy/config/config.yaml
- + tests/test_modules_peppy/expected-results/foo/results/test.out
- + tests/test_modules_peppy/module-test/Snakefile
- + tests/test_modules_peppy/pep/config.yaml
- + tests/test_modules_peppy/pep/sample_table.csv
- tests/test_peppy/workflow/Snakefile → tests/test_peppy/Snakefile
- + tests/test_peppy/expected-results/a.txt
- + tests/test_peppy/expected-results/b.txt
- tests/tests.py
Changes:
=====================================
CHANGELOG.md
=====================================
@@ -1,5 +1,49 @@
# Changelog
+### [6.15.5](https://www.github.com/snakemake/snakemake/compare/v6.15.4...v6.15.5) (2022-02-09)
+
+
+### Bug Fixes
+
+* convert conda env to string before checks ([#1382](https://www.github.com/snakemake/snakemake/issues/1382)) ([7a8da9f](https://www.github.com/snakemake/snakemake/commit/7a8da9fbf01a037a99ebaa3732fe25e87a96fcd2))
+* fix pepfile handling in case of module usage ([#1387](https://www.github.com/snakemake/snakemake/issues/1387)) ([f097a76](https://www.github.com/snakemake/snakemake/commit/f097a761472248d779113cdb22b5274395828bcb))
+
+### [6.15.4](https://www.github.com/snakemake/snakemake/compare/v6.15.3...v6.15.4) (2022-02-09)
+
+
+### Bug Fixes
+
+* fix issue when generating unit tests for rules with directory output ([#1385](https://www.github.com/snakemake/snakemake/issues/1385)) ([7db614f](https://www.github.com/snakemake/snakemake/commit/7db614fa1753179d2cdc20095df17d5ac2885ad0))
+
+
+### Documentation
+
+* fix tutorial setup instructions for MacOS. ([#1383](https://www.github.com/snakemake/snakemake/issues/1383)) ([b57b749](https://www.github.com/snakemake/snakemake/commit/b57b7493d372605323204122af859ede38864e4d))
+
+### [6.15.3](https://www.github.com/snakemake/snakemake/compare/v6.15.2...v6.15.3) (2022-02-07)
+
+
+### Bug Fixes
+
+* skip global report caption when using a module ([#1379](https://www.github.com/snakemake/snakemake/issues/1379)) ([a755cee](https://www.github.com/snakemake/snakemake/commit/a755ceefa478d51070f926beed9090067771edf1))
+
+### [6.15.2](https://www.github.com/snakemake/snakemake/compare/v6.15.1...v6.15.2) (2022-02-05)
+
+
+### Bug Fixes
+
+* avoid mutable default argument ([#1330](https://www.github.com/snakemake/snakemake/issues/1330)) ([978cc93](https://www.github.com/snakemake/snakemake/commit/978cc9327ce7deb517ad609977e1ce432c58c5e2))
+* don't raise WorkflowError when entry is empty ([#1368](https://www.github.com/snakemake/snakemake/issues/1368)) ([1fc6f7b](https://www.github.com/snakemake/snakemake/commit/1fc6f7b5d7e7d7f40baab961db89c4b59c950bf7))
+* fix assertion error in conda env file spec when applying wildcards (thanks [@ddesvillechabrol](https://www.github.com/ddesvillechabrol)) ([#1377](https://www.github.com/snakemake/snakemake/issues/1377)) ([6200652](https://www.github.com/snakemake/snakemake/commit/6200652b9aff2362a63581cee58eb9f9cae189da))
+* fix None type error when invoking Workflow object manually ([#1366](https://www.github.com/snakemake/snakemake/issues/1366)) ([fca3895](https://www.github.com/snakemake/snakemake/commit/fca3895430c206fc159e71622ee567f77566980d))
+* XRootDHelper.exists supports non posix filesystem (object store) ([#1348](https://www.github.com/snakemake/snakemake/issues/1348)) ([7a3ad2f](https://www.github.com/snakemake/snakemake/commit/7a3ad2f438586690dd40e4c8ec591d8c10b22b00))
+
+
+### Documentation
+
+* add sentence about workflow template to docs ([#1369](https://www.github.com/snakemake/snakemake/issues/1369)) ([5fabffb](https://www.github.com/snakemake/snakemake/commit/5fabffbb4af8e9e122677e5adeaebf2d6bd0eeb3))
+* fix typo in installation.rst ([#1344](https://www.github.com/snakemake/snakemake/issues/1344)) ([c45d47a](https://www.github.com/snakemake/snakemake/commit/c45d47a79b78a1afed3b1319e6cafd1b2525fe43))
+
### [6.15.1](https://www.github.com/snakemake/snakemake/compare/v6.15.0...v6.15.1) (2022-01-31)
=====================================
docs/getting_started/installation.rst
=====================================
@@ -22,7 +22,7 @@ This is the **recommended** way to install Snakemake,
because it also enables Snakemake to :ref:`handle software dependencies of your
workflow <integrated_package_management>`.
-First, you have install a Conda-based Python3 distribution.
+First, you need to install a Conda-based Python3 distribution.
The recommended choice is Mambaforge_ which not only provides the required Python and Conda commands,
but also includes Mamba_ an extremely fast and robust replacement for the Conda_ package manager which is highly recommended.
The default conda solver is a bit slow and sometimes has issues with `selecting the latest package releases <https://github.com/conda/conda/issues/9905>`_.
=====================================
docs/snakefiles/deployment.rst
=====================================
@@ -35,15 +35,18 @@ following structure:
├── results
└── resources
-In other words, the workflow code goes into a subfolder ``workflow``, while the configuration is stored in a subfolder ``config``.
+In other words, the workflow code goes into a subfolder ``workflow``, while the configuration is stored in a subfolder ``config``.
Inside of the ``workflow`` subfolder, the central ``Snakefile`` marks the entrypoint of the workflow (it will be automatically discovered when running snakemake from the root of above structure.
-In addition to the central ``Snakefile``, rules can be stored in a modular way, using the optional subfolder ``workflow/rules``. Such modules should end with ``.smk`` the recommended file extension of Snakemake.
+This main structure and the recommendations below are implemented in `this Snakemake workflow template <https://github.com/snakemake-workflows/snakemake-workflow-template>`_ that you can use to `create your own workflow repository with a single click on "Use this template" <https://github.com/snakemake-workflows/snakemake-workflow-template/generate>_.
+In addition to the central ``Snakefile``, rules can be stored in a modular way, using the optional subfolder ``workflow/rules``.
+Such modules should end with ``.smk``, the recommended file extension of Snakemake.
Further, :ref:`scripts <snakefiles-external_scripts>` should be stored in a subfolder ``workflow/scripts`` and notebooks in a subfolder ``workflow/notebooks``.
Conda environments (see :ref:`integrated_package_management`) should be stored in a subfolder ``workflow/envs`` (make sure to keep them as finegrained as possible to improve transparency and maintainability).
Finally, :ref:`report caption files <snakefiles-reports>` should be stored in ``workflow/report``.
-All output files generated in the workflow should be stored under ``results``, unless they are rather retrieved resources, in which case they should be stored under ``resources``. The latter subfolder may also contain small resources that shall be delivered along with the workflow via git (although it might be tempting, please refrain from trying to generate output file paths with string concatenation of a central ``outdir`` variable or so, as this hampers readability).
+All output files generated in the workflow should be stored under ``results``, unless they are rather retrieved resources, in which case they should be stored under ``resources``.
+The latter subfolder may also contain small resources that shall be delivered along with the workflow via git (although it might be tempting, please refrain from trying to generate output file paths with string concatenation of a central ``outdir`` variable or so, as this hampers readability).
-Workflows setup in above structure can be easily used and combined via :ref:`the Snakemake module system <use_with_modules>`.
+Workflows set up in above structure can be easily used and combined via :ref:`the Snakemake module system <use_with_modules>`.
Such deployment can even be automated via `Snakedeploy <https://snakedeploy.readthedocs.io>`_.
Moreover, by publishing a workflow on `Github <https://github.com>`_ and following a set of additional `rules <https://snakemake.github.io/snakemake-workflow-catalog/?rules=true>`_ the workflow will be automatically included in the `Snakemake workflow catalog <https://snakemake.github.io/snakemake-workflow-catalog>`_, thereby easing discovery and even automating its usage documentation.
For an example of such automated documentation, see `here <https://snakemake.github.io/snakemake-workflow-catalog/?usage=snakemake-workflows%2Fdna-seq-varlociraptor>`_.
=====================================
docs/tutorial/setup.rst
=====================================
@@ -120,7 +120,7 @@ Assuming that you have a 64-bit system, on Linux, download and install Miniconda
.. code:: console
- $ wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh
+ $ curl -L https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh -o Mambaforge-Linux-x86_64.sh
$ bash Mambaforge-Linux-x86_64.sh
On MacOS with x86_64 architecture, download and install with
@@ -165,11 +165,19 @@ First, we download some example data on which the workflow shall be executed:
.. code:: console
- $ wget https://github.com/snakemake/snakemake-tutorial-data/archive/v5.24.1.tar.gz
- $ tar --wildcards -xf v5.24.1.tar.gz --strip 1 "*/data" "*/environment.yaml"
+ $ curl -L https://github.com/snakemake/snakemake-tutorial-data/archive/v5.24.1.tar.gz -o snakemake-tutorial-data.tar.gz
+
+Next we extract the data. On Linux, run
+
+.. code:: console
+ $ tar --wildcards -xf snakemake-tutorial-data.tar.gz --strip 1 "*/data" "*/environment.yaml"
+
+On MacOS, run
+
+.. code:: console
+ $ tar -xf snakemake-tutorial-data.tar.gz --strip 1 "*/data" "*/environment.yaml"
This will create a folder ``data`` and a file ``environment.yaml`` in the working directory.
-If your tar command does not provide a ``--wildcards`` flag, you can also just unpack the file without it (which will just leave some more unneeded files in the working directory).
Step 3: Creating an environment with the required software
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
=====================================
snakemake/_version.py
=====================================
@@ -22,9 +22,9 @@ def get_keywords():
# setup.py/versioneer.py will grep for the variable names, so they must
# each be defined on a line of their own. _version.py will just call
# get_keywords().
- git_refnames = " (HEAD -> main, tag: v6.15.1)"
- git_full = "8c5c5e8261ae12fc40c3c293045dea6af7df0e01"
- git_date = "2022-01-31 12:10:44 +0100"
+ git_refnames = " (tag: v6.15.5)"
+ git_full = "0246bc91f3fd9440ec6a4d6f322d3305c8f7d489"
+ git_date = "2022-02-09 23:29:28 +0100"
keywords = {"refnames": git_refnames, "full": git_full, "date": git_date}
return keywords
=====================================
snakemake/deployment/conda.py
=====================================
@@ -704,12 +704,12 @@ class CondaEnvFileSpec(CondaEnvSpec):
else:
self.file = IOFile(filepath, rule=rule)
- def apply_wildcards(self, wildcards):
+ def apply_wildcards(self, wildcards, rule):
filepath = self.file.apply_wildcards(wildcards)
if is_local_file(filepath):
# Normalize 'file:///my/path.yml' to '/my/path.yml'
filepath = parse_uri(filepath).uri_path
- return CondaEnvFileSpec(filepath)
+ return CondaEnvFileSpec(filepath, rule)
def check(self):
self.file.check()
@@ -736,7 +736,7 @@ class CondaEnvNameSpec(CondaEnvSpec):
def __init__(self, name: str):
self.name = name
- def apply_wildcards(self, wildcards):
+ def apply_wildcards(self, wildcards, _):
return CondaEnvNameSpec(apply_wildcards(self.name, wildcards))
def get_conda_env(self, workflow, env_dir=None, container_img=None, cleanup=None):
=====================================
snakemake/linting/__init__.py
=====================================
@@ -61,10 +61,10 @@ class Linter(ABC):
class Lint:
- def __init__(self, title, body, links=[]):
+ def __init__(self, title, body, links=None):
self.title = title
self.body = body
- self.links = links
+ self.links = links or []
def __str__(self):
width, _ = shutil.get_terminal_size()
=====================================
snakemake/logging.py
=====================================
@@ -597,9 +597,12 @@ class Logger:
self.last_msg_was_job_info = False
-def format_dict(dict_like, omit_keys=[], omit_values=[]):
+def format_dict(dict_like, omit_keys=None, omit_values=None):
from snakemake.io import Namedlist
+ omit_keys = omit_keys or []
+ omit_values = omit_values or []
+
if isinstance(dict_like, Namedlist):
items = dict_like.items()
elif isinstance(dict_like, dict):
=====================================
snakemake/modules.py
=====================================
@@ -69,7 +69,13 @@ class ModuleInfo:
self.replace_prefix = replace_prefix
self.prefix = prefix
- def use_rules(self, rules=None, name_modifier=None, ruleinfo=None):
+ def use_rules(
+ self,
+ rules=None,
+ name_modifier=None,
+ ruleinfo=None,
+ skip_global_report_caption=False,
+ ):
snakefile = self.get_snakefile()
with WorkflowModifier(
self.workflow,
@@ -77,6 +83,7 @@ class ModuleInfo:
base_snakefile=snakefile,
skip_configfile=self.config is not None,
skip_validation=self.skip_validation,
+ skip_global_report_caption=skip_global_report_caption,
rule_whitelist=self.get_rule_whitelist(rules),
rulename_modifier=get_name_modifier_func(rules, name_modifier),
ruleinfo_overwrite=ruleinfo,
@@ -129,6 +136,7 @@ class WorkflowModifier:
base_snakefile=None,
skip_configfile=False,
skip_validation=False,
+ skip_global_report_caption=False,
rulename_modifier=None,
rule_whitelist=None,
ruleinfo_overwrite=None,
@@ -150,6 +158,7 @@ class WorkflowModifier:
self.skip_configfile = skip_configfile
self.rulename_modifier = rulename_modifier
self.skip_validation = skip_validation
+ self.skip_global_report_caption = skip_global_report_caption
self.rule_whitelist = rule_whitelist
self.ruleinfo_overwrite = ruleinfo_overwrite
self.allow_rule_overwrite = allow_rule_overwrite
=====================================
snakemake/parser.py
=====================================
@@ -248,7 +248,9 @@ class Configfile(GlobalKeywordState):
class Pepfile(GlobalKeywordState):
- pass
+ @property
+ def keyword(self):
+ return "set_pepfile"
class Pepschema(GlobalKeywordState):
=====================================
snakemake/remote/XRootD.py
=====================================
@@ -7,6 +7,7 @@ import os
from os.path import abspath, join, normpath
import re
+from stat import S_ISREG
from snakemake.remote import AbstractRemoteObject, AbstractRemoteProvider
from snakemake.exceptions import WorkflowError, XRootDFileException
@@ -142,23 +143,30 @@ class XRootDHelper(object):
return domain, dirname, filename
def exists(self, url):
+
domain, dirname, filename = self._parse_url(url)
- status, dirlist = self.get_client(domain).dirlist(dirname)
+
+ status, statInfo = self.get_client(domain).stat(os.path.join(dirname, filename))
+
if not status.ok:
if status.errno == 3011:
return False
- else:
- raise XRootDFileException(
- "Error listing directory "
- + dirname
- + " on domain "
- + domain
- + "\n"
- + repr(status)
- + "\n"
- + repr(dirlist)
- )
- return filename in [f.name for f in dirlist.dirlist]
+ raise XRootDFileException(
+ "Error stating URL "
+ + os.path.join(dirname, filename)
+ + " on domain "
+ + domain
+ + "\n"
+ + repr(status)
+ + "\n"
+ + repr(statInfo)
+ )
+
+ return True
+ # return not (
+ # (statInfo.flags & StatInfoFlags.IS_DIR)
+ # or (statInfo.flags & StatInfoFlags.OTHER)
+ # )
def _get_statinfo(self, url):
domain, dirname, filename = self._parse_url(url)
=====================================
snakemake/report/__init__.py
=====================================
@@ -506,7 +506,10 @@ class FileRecord:
)
except Exception as e:
raise WorkflowError(
- "Error loading caption file of output marked for report.", e
+ "Error loading caption file {} of output marked for report.".format(
+ self.raw_caption.get_path_or_uri()
+ ),
+ e,
)
@property
@@ -602,7 +605,7 @@ def get_resource_as_string(url):
def auto_report(dag, path, stylesheet=None):
try:
- from jinja2 import Template, Environment, PackageLoader
+ from jinja2 import Template, Environment, PackageLoader, UndefinedError
except ImportError as e:
raise WorkflowError(
"Python package jinja2 must be installed to create reports."
@@ -848,12 +851,20 @@ def auto_report(dag, path, stylesheet=None):
config = dag.workflow.config
text = f.read() + rst_links
- text = publish_parts(
- env.from_string(text).render(
- snakemake=Snakemake, categories=results, files=files
- ),
- writer_name="html",
- )["body"]
+ try:
+ text = publish_parts(
+ env.from_string(text).render(
+ snakemake=Snakemake, categories=results, files=files
+ ),
+ writer_name="html",
+ )["body"]
+ except UndefinedError as e:
+ raise WorkflowError(
+ "Error rendering global report caption {}:".format(
+ dag.workflow.report_text.get_path_or_uri()
+ ),
+ e,
+ )
# record time
now = "{} {}".format(datetime.datetime.now().ctime(), time.tzname[0])
=====================================
snakemake/rules.py
=====================================
@@ -434,7 +434,7 @@ class Rule:
def check_output_duplicates(self):
"""Check ``Namedlist`` for duplicate entries and raise a ``WorkflowError``
- on problems.
+ on problems. Does not raise if the entry is empty.
"""
seen = dict()
idx = None
@@ -444,10 +444,10 @@ class Rule:
idx = 0
else:
idx += 1
- if value in seen:
+ if value and value in seen:
raise WorkflowError(
"Duplicate output file pattern in rule {}. First two "
- "duplicate for entries {} and {}".format(
+ "duplicate for entries {} and {}.".format(
self.name, seen[value], name or idx
)
)
@@ -1050,7 +1050,9 @@ class Rule:
def expand_conda_env(self, wildcards):
try:
conda_env = (
- self.conda_env.apply_wildcards(wildcards) if self.conda_env else None
+ self.conda_env.apply_wildcards(wildcards, self)
+ if self.conda_env
+ else None
)
except WildcardError as e:
raise WildcardError(
=====================================
snakemake/unit_tests/__init__.py
=====================================
@@ -54,6 +54,15 @@ def generate(dag, path, deploy=["conda", "singularity"], configfiles=None):
)
for rulename, jobs in groupby(dag.jobs, key=lambda job: job.rule.name):
+ jobs = list(jobs)
+ if jobs[0].rule.norun:
+ logger.info(
+ "Skipping rule {} because it does not execute anything.".format(
+ rulename
+ )
+ )
+ continue
+
testpath = path / "test_{}.py".format(rulename)
if testpath.exists():
@@ -79,12 +88,11 @@ def generate(dag, path, deploy=["conda", "singularity"], configfiles=None):
if parent.is_absolute():
root = str(f.parents[len(f.parents) - 1])
parent = str(parent)[len(root) :]
- print(parent)
target = path / rulename / content_type / parent
- os.makedirs(target, exist_ok=True)
if f.is_dir():
- shutil.copytree(f, target)
+ shutil.copytree(f, target / f.name)
else:
+ os.makedirs(target, exist_ok=True)
shutil.copy(f, target)
if not files:
os.makedirs(path / rulename / content_type, exist_ok=True)
=====================================
snakemake/unit_tests/templates/ruletest.py.jinja2
=====================================
@@ -30,7 +30,7 @@ def test_{{ ruletest.name }}():
"-m",
"snakemake",
"{{ ruletest.target }}",
- "-F",
+ "-f",
"-j1",
"--keep-target-files",
{% if configfiles %}
=====================================
snakemake/workflow.py
=====================================
@@ -204,7 +204,9 @@ class Workflow:
self.attempt = attempt
self.default_remote_provider = default_remote_provider
self.default_remote_prefix = default_remote_prefix
- self.configfiles = list(overwrite_configfiles) or []
+ self.configfiles = (
+ [] if overwrite_configfiles is None else list(overwrite_configfiles)
+ )
self.run_local = run_local
self.report_text = None
self.conda_cleanup_pkgs = conda_cleanup_pkgs
@@ -1239,19 +1241,18 @@ class Workflow:
def configfile(self, fp):
"""Update the global config with data from the given file."""
- global config
if not self.modifier.skip_configfile:
if os.path.exists(fp):
self.configfiles.append(fp)
c = snakemake.io.load_configfile(fp)
- update_config(config, c)
+ update_config(self.config, c)
if self.overwrite_config:
logger.info(
"Config file {} is extended by additional config specified via the command line.".format(
fp
)
)
- update_config(config, self.overwrite_config)
+ update_config(self.config, self.overwrite_config)
elif not self.overwrite_configfiles:
raise WorkflowError(
"Workflow defines configfile {} but it is not present or accessible.".format(
@@ -1259,8 +1260,7 @@ class Workflow:
)
)
- def pepfile(self, path):
- global pep
+ def set_pepfile(self, path):
try:
import peppy
@@ -1268,11 +1268,9 @@ class Workflow:
raise WorkflowError("For PEP support, please install peppy.")
self.pepfile = path
- pep = peppy.Project(self.pepfile)
+ self.globals["pep"] = peppy.Project(self.pepfile)
def pepschema(self, schema):
- global pep
-
try:
import eido
except ImportError:
@@ -1283,11 +1281,14 @@ class Workflow:
schema = self.current_basedir.join(schema).get_path_or_uri()
if self.pepfile is None:
raise WorkflowError("Please specify a PEP with the pepfile directive.")
- eido.validate_project(project=pep, schema=schema, exclude_case=True)
+ eido.validate_project(
+ project=self.globals["pep"], schema=schema, exclude_case=True
+ )
def report(self, path):
"""Define a global report description in .rst format."""
- self.report_text = self.current_basedir.join(path)
+ if not self.modifier.skip_global_report_caption:
+ self.report_text = self.current_basedir.join(path)
@property
def config(self):
@@ -1481,6 +1482,9 @@ class Workflow:
rule=rule,
)
+ if isinstance(ruleinfo.conda_env, Path):
+ ruleinfo.conda_env = str(ruleinfo.conda_env)
+
if (
ruleinfo.conda_env is not None
and is_conda_env_file(ruleinfo.conda_env)
@@ -1841,6 +1845,8 @@ class Workflow:
rules,
name_modifier,
ruleinfo=None if callable(maybe_ruleinfo) else maybe_ruleinfo,
+ skip_global_report_caption=self.report_text
+ is not None, # do not overwrite existing report text via module
)
else:
# local inheritance
=====================================
tests/test_conda/Snakefile
=====================================
@@ -1,4 +1,5 @@
shell.executable("bash")
+conda_env = Path("test-env.yaml")
rule all:
input:
@@ -19,6 +20,6 @@ rule b:
output:
"test{i}.out2"
conda:
- "test-env.yaml"
+ conda_env
shell:
"Tm -h > {output} || true"
=====================================
tests/test_github_issue1384/Snakefile
=====================================
@@ -0,0 +1,20 @@
+rule all:
+ input:
+ "results/checksum.txt"
+
+rule foo:
+ output:
+ directory("results/dir")
+ shell:
+ """
+ mkdir results/dir
+ echo hi >> results/dir/something
+ """
+
+rule bar:
+ input:
+ rules.foo.output
+ output:
+ "results/checksum.txt"
+ shell:
+ "ls {input}/* >> {output}"
=====================================
tests/test_github_issue1384/expected-results/.gitempty
=====================================
=====================================
tests/test_io.py
=====================================
@@ -104,11 +104,8 @@ def test_expand():
) == sorted(["a: aa + b: b", "a: aa + b: bb", "c: c", "c: cc"])
# expand on pathlib.Path objects
- assert (
- expand(
- PosixPath() / "{x}" / "{y}",
- x="Hello",
- y="world",
- )
- == ["Hello/world"]
- )
+ assert expand(
+ PosixPath() / "{x}" / "{y}",
+ x="Hello",
+ y="world",
+ ) == ["Hello/world"]
=====================================
tests/test_modules_peppy/Snakefile
=====================================
@@ -0,0 +1,15 @@
+shell.executable("bash")
+
+configfile: "config/config.yaml"
+
+
+module test:
+ snakefile:
+ "module-test/Snakefile"
+ config:
+ config
+ prefix:
+ "foo"
+
+
+use rule * from test
=====================================
tests/test_modules_peppy/config/config.yaml
=====================================
@@ -0,0 +1,2 @@
+test: 1
+pepfile: "pep/config.yaml"
\ No newline at end of file
=====================================
tests/test_modules_peppy/expected-results/foo/results/test.out
=====================================
@@ -0,0 +1 @@
+1
=====================================
tests/test_modules_peppy/module-test/Snakefile
=====================================
@@ -0,0 +1,14 @@
+configfile: "config.yaml" # does not exist, but this statement should be ignored on module import
+pepfile: config["pepfile"]
+
+def some_func():
+ return 15
+
+print(pep)
+
+rule a:
+ output:
+ "results/test.out",
+ "/tmp/foo.txt"
+ shell:
+ "echo {config[test]} > {output[0]}; touch {output[1]}"
\ No newline at end of file
=====================================
tests/test_modules_peppy/pep/config.yaml
=====================================
@@ -0,0 +1,2 @@
+pep_version: "2.0.0"
+sample_table: sample_table.csv
=====================================
tests/test_modules_peppy/pep/sample_table.csv
=====================================
@@ -0,0 +1,3 @@
+sample_name,protocol
+a,test
+b,test2
=====================================
tests/test_peppy/workflow/Snakefile → tests/test_peppy/Snakefile
=====================================
@@ -1,5 +1,7 @@
pepfile: "pep/config.yaml"
-pepschema: "schemas/pep.yaml"
+pepfile: "pep/config.yaml" # test overwriting
+
+pepschema: "workflow/schemas/pep.yaml"
rule all:
input:
=====================================
tests/test_peppy/expected-results/a.txt
=====================================
=====================================
tests/test_peppy/expected-results/b.txt
=====================================
=====================================
tests/tests.py
=====================================
@@ -1289,6 +1289,11 @@ def test_modules_prefix():
run(dpath("test_modules_prefix"), targets=["a"])
+ at skip_on_windows
+def test_modules_peppy():
+ run(dpath("test_modules_peppy"), targets=["a"])
+
+
def test_modules_specific():
run(dpath("test_modules_specific"), targets=["test_a"])
@@ -1412,3 +1417,23 @@ def test_default_target():
def test_cache_multioutput():
run(dpath("test_cache_multioutput"), shouldfail=True)
+
+
+ at skip_on_windows
+def test_github_issue1384():
+ try:
+ tmpdir = run(dpath("test_github_issue1384"), cleanup=False)
+ shell(
+ """
+ cd {tmpdir}
+ python -m snakemake --generate-unit-tests
+ pytest -v .tests/unit
+ """
+ )
+ finally:
+ shutil.rmtree(tmpdir)
+
+
+ at skip_on_windows
+def test_peppy():
+ run(dpath("test_peppy"))
View it on GitLab: https://salsa.debian.org/med-team/snakemake/-/compare/5b3400faa42ed4871e062695a4e177b5122f93be...1f41139d20e5c7adba461d7970dcd4ecfee997e3
--
View it on GitLab: https://salsa.debian.org/med-team/snakemake/-/compare/5b3400faa42ed4871e062695a4e177b5122f93be...1f41139d20e5c7adba461d7970dcd4ecfee997e3
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/debian-med-commit/attachments/20220220/c41fe342/attachment-0001.htm>
More information about the debian-med-commit
mailing list