[med-svn] [Git][med-team/q2-demux][master] 6 commits: Bumped versioned Depends
Andreas Tille (@tille)
gitlab at salsa.debian.org
Thu Jan 12 20:47:34 GMT 2023
Andreas Tille pushed to branch master at Debian Med / q2-demux
Commits:
d06cbe66 by Andreas Tille at 2023-01-12T21:19:48+01:00
Bumped versioned Depends
- - - - -
c532c12f by Andreas Tille at 2023-01-12T21:20:04+01:00
routine-update: New upstream version
- - - - -
606ce49b by Andreas Tille at 2023-01-12T21:20:05+01:00
New upstream version 2022.11.1+dfsg
- - - - -
5826449f by Andreas Tille at 2023-01-12T21:20:05+01:00
Update upstream source from tag 'upstream/2022.11.1+dfsg'
Update to upstream version '2022.11.1+dfsg'
with Debian dir 6d9a31b97288d21a00a6624025d6847e60270a71
- - - - -
825471b5 by Andreas Tille at 2023-01-12T21:20:05+01:00
routine-update: Standards-Version: 4.6.2
- - - - -
cf00cb86 by Andreas Tille at 2023-01-12T21:42:34+01:00
Upload to experimental
- - - - -
20 changed files:
- + .github/CONTRIBUTING.md
- + .github/ISSUE_TEMPLATE/1-user-need-help.md
- + .github/ISSUE_TEMPLATE/2-dev-need-help.md
- + .github/ISSUE_TEMPLATE/3-found-bug.md
- + .github/ISSUE_TEMPLATE/4-make-better.md
- + .github/ISSUE_TEMPLATE/5-make-new.md
- + .github/ISSUE_TEMPLATE/6-where-to-go.md
- + .github/SUPPORT.md
- + .github/pull_request_template.md
- + .github/rubric.png
- + .github/workflows/add-to-project-ci.yml
- + .github/workflows/ci.yml
- debian/changelog
- debian/control
- q2_demux/_demux.py
- + q2_demux/_examples.py
- q2_demux/_version.py
- q2_demux/plugin_setup.py
- q2_demux/tests/test_demux.py
- setup.py
Changes:
=====================================
.github/CONTRIBUTING.md
=====================================
@@ -0,0 +1,23 @@
+# Contributing to this project
+
+Thanks for thinking of us :heart: :tada: - we would love a helping hand!
+
+## I just have a question
+
+> Note: Please don't file an issue to ask a question. You'll get faster results
+> by using the resources below.
+
+### QIIME 2 Users
+
+Check out the [User Docs](https://docs.qiime2.org) - there are many tutorials,
+walkthroughs, and guides available. If you still need help, please visit us at
+the [QIIME 2 Forum](https://forum.qiime2.org/c/user-support).
+
+### QIIME 2 Developers
+
+Check out the [Developer Docs](https://dev.qiime2.org) - there are many
+tutorials, walkthroughs, and guides available. If you still need help, please
+visit us at the [QIIME 2 Forum](https://forum.qiime2.org/c/dev-discussion).
+
+This document is based heavily on the following:
+https://github.com/atom/atom/blob/master/CONTRIBUTING.md
=====================================
.github/ISSUE_TEMPLATE/1-user-need-help.md
=====================================
@@ -0,0 +1,14 @@
+---
+name: I am a user and I need help with QIIME 2...
+about: I am using QIIME 2 and have a question or am experiencing a problem
+
+---
+
+Have you had a chance to check out the docs?
+https://docs.qiime2.org
+There are many tutorials, walkthroughs, and guides available.
+
+If you still need help, please visit:
+https://forum.qiime2.org/c/user-support
+
+Help requests filed here will not be answered.
=====================================
.github/ISSUE_TEMPLATE/2-dev-need-help.md
=====================================
@@ -0,0 +1,12 @@
+---
+name: I am a developer and I need help with QIIME 2...
+about: I am developing a QIIME 2 plugin or interface and have a question or a problem
+
+---
+
+Have you had a chance to check out the developer docs?
+https://dev.qiime2.org
+There are many tutorials, walkthroughs, and guides available.
+
+If you still need help, please visit:
+https://forum.qiime2.org/c/dev-discussion
=====================================
.github/ISSUE_TEMPLATE/3-found-bug.md
=====================================
@@ -0,0 +1,36 @@
+---
+name: I am a developer and I found a bug...
+about: I am a developer and I found a bug that I can describe
+
+---
+
+**Bug Description**
+A clear and concise description of what the bug is.
+
+**Steps to reproduce the behavior**
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Computation Environment**
+- OS: [e.g. macOS High Sierra]
+- QIIME 2 Release [e.g. 2018.6]
+
+**Questions**
+1. An enumerated list with any questions about the problem here.
+2. If not applicable, please delete this section.
+
+**Comments**
+1. An enumerated list with any other context or comments about the problem here.
+2. If not applicable, please delete this section.
+
+**References**
+1. An enumerated list of links to relevant references, including forum posts, stack overflow, etc.
+2. If not applicable, please delete this section.
=====================================
.github/ISSUE_TEMPLATE/4-make-better.md
=====================================
@@ -0,0 +1,26 @@
+---
+name: I am a developer and I have an idea for an improvement...
+about: I am a developer and I have an idea for an improvement to existing functionality
+
+---
+
+**Improvement Description**
+A clear and concise description of what the improvement is.
+
+**Current Behavior**
+Please provide a brief description of the current behavior.
+
+**Proposed Behavior**
+Please provide a brief description of the proposed behavior.
+
+**Questions**
+1. An enumerated list of questions related to the proposal.
+2. If not applicable, please delete this section.
+
+**Comments**
+1. An enumerated list of comments related to the proposal that don't fit anywhere else.
+2. If not applicable, please delete this section.
+
+**References**
+1. An enumerated list of links to relevant references, including forum posts, stack overflow, etc.
+2. If not applicable, please delete this section.
=====================================
.github/ISSUE_TEMPLATE/5-make-new.md
=====================================
@@ -0,0 +1,26 @@
+---
+name: I am a developer and I have an idea for a new feature...
+about: I am a developer and I have an idea for new functionality
+
+---
+
+**Addition Description**
+A clear and concise description of what the addition is.
+
+**Current Behavior**
+Please provide a brief description of the current behavior, if applicable.
+
+**Proposed Behavior**
+Please provide a brief description of the proposed behavior.
+
+**Questions**
+1. An enumerated list of questions related to the proposal.
+2. If not applicable, please delete this section.
+
+**Comments**
+1. An enumerated list of comments related to the proposal that don't fit anywhere else.
+2. If not applicable, please delete this section.
+
+**References**
+1. An enumerated list of links to relevant references, including forum posts, stack overflow, etc.
+2. If not applicable, please delete this section.
=====================================
.github/ISSUE_TEMPLATE/6-where-to-go.md
=====================================
@@ -0,0 +1,147 @@
+---
+name: I don't know where to file my issue...
+about: I am a developer and I don't know which repo to file this in
+
+---
+
+The repos within the QIIME 2 GitHub Organization are listed below, with a brief description about the repo.
+
+Sorted alphabetically by repo name.
+
+- The CI automation engine that builds and distributes QIIME 2
+ https://github.com/qiime2/busywork/issues
+
+- A Concourse resource for working with conda
+ https://github.com/qiime2/conda-channel-resource/issues
+
+- Web app for vanity URLs for QIIME 2 data assets
+ https://github.com/qiime2/data.qiime2.org/issues
+
+- The Developer Documentation
+ https://github.com/qiime2/dev-docs/issues
+
+- A discourse plugin for handling queued/unqueued topics
+ https://github.com/qiime2/discourse-unhandled-tagger/issues
+
+- The User Documentation
+ https://github.com/qiime2/docs/issues
+
+- Rendered QIIME 2 environment files for conda
+ https://github.com/qiime2/environment-files/issues
+
+- Google Sheets Add-On for validating tabular data
+ https://github.com/qiime2/Keemei/issues
+
+- A docker image for linux-based busywork workers
+ https://github.com/qiime2/linux-worker-docker/issues
+
+- Official project logos
+ https://github.com/qiime2/logos/issues
+
+- The q2-alignment plugin
+ https://github.com/qiime2/q2-alignment/issues
+
+- The q2-composition plugin
+ https://github.com/qiime2/q2-composition/issues
+
+- The q2-cutadapt plugin
+ https://github.com/qiime2/q2-cutadapt/issues
+
+- The q2-dada2 plugin
+ https://github.com/qiime2/q2-dada2/issues
+
+- The q2-deblur plugin
+ https://github.com/qiime2/q2-deblur/issues
+
+- The q2-demux plugin
+ https://github.com/qiime2/q2-demux/issues
+
+- The q2-diversity plugin
+ https://github.com/qiime2/q2-diversity/issues
+
+- The q2-diversity-lib plugin
+ https://github.com/qiime2/q2-diversity-lib/issues
+
+- The q2-emperor plugin
+ https://github.com/qiime2/q2-emperor/issues
+
+- The q2-feature-classifier plugin
+ https://github.com/qiime2/q2-feature-classifier/issues
+
+- The q2-feature-table plugin
+ https://github.com/qiime2/q2-feature-table/issues
+
+- The q2-fragment-insertion plugin
+ https://github.com/qiime2/q2-fragment-insertion/issues
+
+- The q2-gneiss plugin
+ https://github.com/qiime2/q2-gneiss/issues
+
+- The q2-longitudinal plugin
+ https://github.com/qiime2/q2-longitudinal/issues
+
+- The q2-metadata plugin
+ https://github.com/qiime2/q2-metadata/issues
+
+- The q2-phylogeny plugin
+ https://github.com/qiime2/q2-phylogeny/issues
+
+- The q2-quality-control plugin
+ https://github.com/qiime2/q2-quality-control/issues
+
+- The q2-quality-filter plugin
+ https://github.com/qiime2/q2-quality-filter/issues
+
+- The q2-sample-classifier plugin
+ https://github.com/qiime2/q2-sample-classifier/issues
+
+- The q2-shogun plugin
+ https://github.com/qiime2/q2-shogun/issues
+
+- The q2-taxa plugin
+ https://github.com/qiime2/q2-taxa/issues
+
+- The q2-types plugin
+ https://github.com/qiime2/q2-types/issues
+
+- The q2-vsearch plugin
+ https://github.com/qiime2/q2-vsearch/issues
+
+- The CLI interface
+ https://github.com/qiime2/q2cli/issues
+
+- The prototype CWL interface
+ https://github.com/qiime2/q2cwl/issues
+
+- The prototype Galaxy interface
+ https://github.com/qiime2/q2galaxy/issues
+
+- An internal tool for ensuring header text and copyrights are present
+ https://github.com/qiime2/q2lint/issues
+
+- The prototype GUI interface
+ https://github.com/qiime2/q2studio/issues
+
+- A base template for use in official QIIME 2 plugins
+ https://github.com/qiime2/q2templates/issues
+
+- The read-only web interface at view.qiime2.org
+ https://github.com/qiime2/q2view/issues
+
+- The QIIME 2 homepage at qiime2.org
+ https://github.com/qiime2/qiime2.github.io/issues
+
+- The QIIME 2 framework
+ https://github.com/qiime2/qiime2/issues
+
+- Centralized templates for repo assets
+ https://github.com/qiime2/template-repo/issues
+
+- Scripts for building QIIME 2 VMs
+ https://github.com/qiime2/vm-playbooks/issues
+
+- Scripts for building QIIME 2 workshop clusters
+ https://github.com/qiime2/workshop-playbooks/issues
+
+- The web app that runs workshops.qiime2.org
+ https://github.com/qiime2/workshops.qiime2.org/issues
=====================================
.github/SUPPORT.md
=====================================
@@ -0,0 +1,112 @@
+# QIIME 2 Users
+
+Check out the [User Docs](https://docs.qiime2.org) - there are many tutorials,
+walkthroughs, and guides available. If you still need help, please visit us at
+the [QIIME 2 Forum](https://forum.qiime2.org/c/user-support).
+
+# QIIME 2 Developers
+
+Check out the [Developer Docs](https://dev.qiime2.org) - there are many
+tutorials, walkthroughs, and guides available. If you still need help, please
+visit us at the [QIIME 2 Forum](https://forum.qiime2.org/c/dev-discussion).
+
+# General Bug/Issue Triage Discussion
+
+
+
+# Projects/Repositories in the QIIME 2 GitHub Organization
+
+Sorted alphabetically by repo name.
+
+- [busywork](https://github.com/qiime2/busywork/issues)
+ | The CI automation engine that builds and distributes QIIME 2
+- [conda-channel-resource](https://github.com/qiime2/conda-channel-resource/issues)
+ | A Concourse resource for working with conda
+- [data.qiime2.org](https://github.com/qiime2/data.qiime2.org/issues)
+ | Web app for vanity URLs for QIIME 2 data assets
+- [dev-docs](https://github.com/qiime2/dev-docs/issues)
+ | The Developer Documentation
+- [discourse-unhandled-tagger](https://github.com/qiime2/discourse-unhandled-tagger/issues)
+ | A discourse plugin for handling queued/unqueued topics
+- [docs](https://github.com/qiime2/docs/issues)
+ | The User Documentation
+- [environment-files](https://github.com/qiime2/environment-files/issues)
+ | Rendered QIIME 2 environment files for conda
+- [Keemei](https://github.com/qiime2/Keemei/issues)
+ | Google Sheets Add-On for validating tabular data
+- [linux-worker-docker](https://github.com/qiime2/linux-worker-docker/issues)
+ | A docker image for linux-based busywork workers
+- [logos](https://github.com/qiime2/logos/issues)
+ | Official project logos
+- [q2-alignment](https://github.com/qiime2/q2-alignment/issues)
+ | The q2-alignment plugin
+- [q2-composition](https://github.com/qiime2/q2-composition/issues)
+ | The q2-composition plugin
+- [q2-cutadapt](https://github.com/qiime2/q2-cutadapt/issues)
+ | The q2-cutadapt plugin
+- [q2-dada2](https://github.com/qiime2/q2-dada2/issues)
+ | The q2-dada2 plugin
+- [q2-deblur](https://github.com/qiime2/q2-deblur/issues)
+ | The q2-deblur plugin
+- [q2-demux](https://github.com/qiime2/q2-demux/issues)
+ | The q2-demux plugin
+- [q2-diversity](https://github.com/qiime2/q2-diversity/issues)
+ | The q2-diversity plugin
+- [q2-diversity-lib](https://github.com/qiime2/q2-diversity-lib/issues)
+ | The q2-diversity-lib plugin
+- [q2-emperor](https://github.com/qiime2/q2-emperor/issues)
+ | The q2-emperor plugin
+- [q2-feature-classifier](https://github.com/qiime2/q2-feature-classifier/issues)
+ | The q2-feature-classifier plugin
+- [q2-feature-table](https://github.com/qiime2/q2-feature-table/issues)
+ | The q2-feature-table plugin
+- [q2-fragment-insertion](https://github.com/qiime2/q2-fragment-insertion/issues)
+ | The q2-fragment-insertion plugin
+- [q2-gneiss](https://github.com/qiime2/q2-gneiss/issues)
+ | The q2-gneiss plugin
+- [q2-longitudinal](https://github.com/qiime2/q2-longitudinal/issues)
+ | The q2-longitudinal plugin
+- [q2-metadata](https://github.com/qiime2/q2-metadata/issues)
+ | The q2-metadata plugin
+- [q2-phylogeny](https://github.com/qiime2/q2-phylogeny/issues)
+ | The q2-phylogeny plugin
+- [q2-quality-control](https://github.com/qiime2/q2-quality-control/issues)
+ | The q2-quality-control plugin
+- [q2-quality-filter](https://github.com/qiime2/q2-quality-filter/issues)
+ | The q2-quality-filter plugin
+- [q2-sample-classifier](https://github.com/qiime2/q2-sample-classifier/issues)
+ | The q2-sample-classifier plugin
+- [q2-shogun](https://github.com/qiime2/q2-shogun/issues)
+ | The q2-shogun plugin
+- [q2-taxa](https://github.com/qiime2/q2-taxa/issues)
+ | The q2-taxa plugin
+- [q2-types](https://github.com/qiime2/q2-types/issues)
+ | The q2-types plugin
+- [q2-vsearch](https://github.com/qiime2/q2-vsearch/issues)
+ | The q2-vsearch plugin
+- [q2cli](https://github.com/qiime2/q2cli/issues)
+ | The CLI interface
+- [q2cwl](https://github.com/qiime2/q2cwl/issues)
+ | The prototype CWL interface
+- [q2galaxy](https://github.com/qiime2/q2galaxy/issues)
+ | The prototype Galaxy interface
+- [q2lint](https://github.com/qiime2/q2lint/issues)
+ | An internal tool for ensuring header text and copyrights are present
+- [q2studio](https://github.com/qiime2/q2studio/issues)
+ | The prototype GUI interface
+- [q2templates](https://github.com/qiime2/q2templates/issues)
+ | A base template for use in official QIIME 2 plugins
+- [q2view](https://github.com/qiime2/q2view/issues)
+ | The read-only web interface at view.qiime2.org
+- [qiime2.github.io](https://github.com/qiime2/qiime2.github.io/issues)
+ | The QIIME 2 homepage at qiime2.org
+- [qiime2](https://github.com/qiime2/qiime2/issues)
+ | The QIIME 2 framework
+- [template-repo](https://github.com/qiime2/template-repo/issues)
+ | Centralized templates for repo assets
+- [vm-playbooks](https://github.com/qiime2/vm-playbooks/issues)
+ | Scripts for building QIIME 2 VMs
+- [workshop-playbooks](https://github.com/qiime2/workshop-playbooks/issues)
+ | Scripts for building QIIME 2 workshop clusters
+- [workshops.qiime2.org](https://github.com/qiime2/workshops.qiime2.org/issues)
+ | The web app that runs workshops.qiime2.org
=====================================
.github/pull_request_template.md
=====================================
@@ -0,0 +1,11 @@
+Brief summary of the Pull Request, including any issues it may fix using the GitHub closing syntax:
+
+https://help.github.com/articles/closing-issues-using-keywords/
+
+Also, include any co-authors or contributors using the GitHub coauthor tag:
+
+https://help.github.com/articles/creating-a-commit-with-multiple-authors/
+
+---
+
+Include any questions for reviewers, screenshots, sample outputs, etc.
=====================================
.github/rubric.png
=====================================
Binary files /dev/null and b/.github/rubric.png differ
=====================================
.github/workflows/add-to-project-ci.yml
=====================================
@@ -0,0 +1,21 @@
+name: Add new issues and PRs to triage project board
+
+on:
+ issues:
+ types:
+ - opened
+ pull_request_target:
+ types:
+ - opened
+
+jobs:
+ add-to-project:
+ name: Add issue to project
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/add-to-project at v0.3.0
+ with:
+ project-url: https://github.com/orgs/qiime2/projects/36
+ github-token: ${{ secrets.ADD_TO_PROJECT_PAT }}
+ labeled: skip-triage
+ label-operator: NOT
=====================================
.github/workflows/ci.yml
=====================================
@@ -0,0 +1,55 @@
+# This file is automatically generated by busywork.qiime2.org and
+# template-repos - any manual edits made to this file will be erased when
+# busywork performs maintenance updates.
+
+name: ci
+
+on:
+ pull_request:
+ push:
+ branches:
+ - master
+
+jobs:
+ lint:
+ runs-on: ubuntu-latest
+ steps:
+ - name: checkout source
+ uses: actions/checkout at v2
+
+ - name: set up python 3.8
+ uses: actions/setup-python at v1
+ with:
+ python-version: 3.8
+
+ - name: install dependencies
+ run: python -m pip install --upgrade pip
+
+ - name: lint
+ run: |
+ pip install -q https://github.com/qiime2/q2lint/archive/master.zip
+ q2lint
+ pip install -q flake8
+ flake8
+
+ build-and-test:
+ needs: lint
+ strategy:
+ matrix:
+ os: [ubuntu-latest, macos-latest]
+ runs-on: ${{ matrix.os }}
+ steps:
+ - name: checkout source
+ uses: actions/checkout at v2
+ with:
+ fetch-depth: 0
+
+ - name: set up git repo for versioneer
+ run: git fetch --depth=1 origin +refs/tags/*:refs/tags/*
+
+ - uses: qiime2/action-library-packaging at alpha1
+ with:
+ package-name: q2-demux
+ build-target: dev
+ additional-tests: py.test --pyargs q2_demux
+ library-token: ${{ secrets.LIBRARY_TOKEN }}
=====================================
debian/changelog
=====================================
@@ -1,3 +1,12 @@
+q2-demux (2022.11.1+dfsg-1) experimental; urgency=medium
+
+ * Team upload.
+ * New upstream version
+ * Bumped versioned Depends
+ * Standards-Version: 4.6.2 (routine-update)
+
+ -- Andreas Tille <tille at debian.org> Thu, 12 Jan 2023 21:21:28 +0100
+
q2-demux (2022.8.0+dfsg-1) unstable; urgency=medium
* Team Upload.
=====================================
debian/control
=====================================
@@ -18,10 +18,10 @@ Build-Depends: debhelper-compat (= 13),
python3-skbio,
python3-psutil,
python3-seaborn,
- qiime (>= 2022.8.1),
- q2-types (>= 2022.8.0),
- q2templates (>= 2022.8.0)
-Standards-Version: 4.6.1
+ qiime (>= 2022.11.1),
+ q2-types (>= 2022.11.1),
+ q2templates (>= 2022.11.1)
+Standards-Version: 4.6.2
Vcs-Browser: https://salsa.debian.org/med-team/q2-demux
Vcs-Git: https://salsa.debian.org/med-team/q2-demux.git
Homepage: https://qiime2.org/
@@ -40,9 +40,9 @@ Depends: ${shlibs:Depends},
python3-yaml,
python3-psutil,
python3-ipywidgets,
- qiime (>= 2022.8.1),
- q2-types (>= 2022.8.0),
- q2templates (>= 2022.8.0),
+ qiime (>= 2022.11.1),
+ q2-types (>= 2022.11.1),
+ q2templates (>= 2022.11.1),
node-d3
Description: QIIME 2 plugin for demultiplexing of sequence reads
QIIME 2 is a powerful, extensible, and decentralized microbiome analysis
=====================================
q2_demux/_demux.py
=====================================
@@ -42,6 +42,9 @@ class ECDetails:
self._fp = open(str(fmt), 'w')
self._write_header()
+ def __enter__(self):
+ return self
+
def write(self, parts):
self._fp.write('\t'.join([str(part) for part in parts]))
self._fp.write('\n')
@@ -49,6 +52,9 @@ class ECDetails:
def _write_header(self):
self.write(self.COLUMNS)
+ def __exit__(self, *args):
+ self._fp.close()
+
def _read_fastq_seqs(filepath):
# This function is adapted from @jairideout's SO post:
@@ -309,64 +315,65 @@ def emp_single(seqs: BarcodeSequenceFastqIterator,
per_sample_fastqs = {}
ec_details_fmt = ErrorCorrectionDetailsFmt()
- ec_details = ECDetails(ec_details_fmt)
-
- for i, (barcode_record, sequence_record) in enumerate(seqs, start=1):
- raw_barcode_read = barcode_record[1][:barcode_len]
- if rev_comp_barcodes:
- barcode_as_DNA = skbio.DNA(raw_barcode_read)
- raw_barcode_read = str(barcode_as_DNA.reverse_complement())
-
- if golay_error_correction:
- # A three bit filter is implicitly used by the decoder. See Hamady
- # and Knight 2009 Genome Research for the justification:
- #
- # https://genome.cshlp.org/content/19/7/1141.full
- #
- # Specifically that "...Golay codes of 12 bases can correct all
- # triple-bit errors and detect all quadruple-bit errors."
- barcode_read, ecc_errors = decoder.decode(raw_barcode_read)
- golay_stats = [barcode_read, ecc_errors]
- else:
- barcode_read = raw_barcode_read
- golay_stats = [None, None]
-
- sample_id = barcode_map.get(barcode_read)
-
- record = [
- f'record-{i}',
- sample_id,
- barcode_record[0],
- raw_barcode_read,
- ]
- ec_details.write(record + golay_stats)
-
- if sample_id is None:
- continue
-
- if sample_id not in per_sample_fastqs:
- # The barcode id, lane number and read number are not relevant
- # here. We might ultimately want to use a dir format other than
- # SingleLanePerSampleSingleEndFastqDirFmt which doesn't care
- # about this information. Similarly, the direction of the read
- # isn't relevant here anymore.
- barcode_id = len(per_sample_fastqs) + 1
- path = result.sequences.path_maker(sample_id=sample_id,
- barcode_id=barcode_id,
- lane_number=1,
- read_number=1)
- _maintain_open_fh_count(per_sample_fastqs)
- per_sample_fastqs[sample_id] = gzip.open(str(path), mode='a')
- manifest_fh.write('%s,%s,%s\n' % (sample_id, path.name, 'forward'))
-
- if per_sample_fastqs[sample_id].closed:
- _maintain_open_fh_count(per_sample_fastqs)
- per_sample_fastqs[sample_id] = gzip.open(
- per_sample_fastqs[sample_id].name, mode='a')
-
- fastq_lines = '\n'.join(sequence_record) + '\n'
- fastq_lines = fastq_lines.encode('utf-8')
- per_sample_fastqs[sample_id].write(fastq_lines)
+ with ECDetails(ec_details_fmt) as ec_details:
+
+ for i, (barcode_record, sequence_record) in enumerate(seqs, start=1):
+ raw_barcode_read = barcode_record[1][:barcode_len]
+ if rev_comp_barcodes:
+ barcode_as_DNA = skbio.DNA(raw_barcode_read)
+ raw_barcode_read = str(barcode_as_DNA.reverse_complement())
+
+ if golay_error_correction:
+ # A three bit filter is implicitly used by the decoder. See
+ # Hamady and Knight 2009 Genome Research for the justification:
+ #
+ # https://genome.cshlp.org/content/19/7/1141.full
+ #
+ # Specifically that "...Golay codes of 12 bases can correct all
+ # triple-bit errors and detect all quadruple-bit errors."
+ barcode_read, ecc_errors = decoder.decode(raw_barcode_read)
+ golay_stats = [barcode_read, ecc_errors]
+ else:
+ barcode_read = raw_barcode_read
+ golay_stats = [None, None]
+
+ sample_id = barcode_map.get(barcode_read)
+
+ record = [
+ f'record-{i}',
+ sample_id,
+ barcode_record[0],
+ raw_barcode_read,
+ ]
+ ec_details.write(record + golay_stats)
+
+ if sample_id is None:
+ continue
+
+ if sample_id not in per_sample_fastqs:
+ # The barcode id, lane number and read number are not relevant
+ # here. We might ultimately want to use a dir format other than
+ # SingleLanePerSampleSingleEndFastqDirFmt which doesn't care
+ # about this information. Similarly, the direction of the read
+ # isn't relevant here anymore.
+ barcode_id = len(per_sample_fastqs) + 1
+ path = result.sequences.path_maker(sample_id=sample_id,
+ barcode_id=barcode_id,
+ lane_number=1,
+ read_number=1)
+ _maintain_open_fh_count(per_sample_fastqs)
+ per_sample_fastqs[sample_id] = gzip.open(str(path), mode='a')
+ manifest_fh.write('%s,%s,%s\n' % (sample_id, path.name,
+ 'forward'))
+
+ if per_sample_fastqs[sample_id].closed:
+ _maintain_open_fh_count(per_sample_fastqs)
+ per_sample_fastqs[sample_id] = gzip.open(
+ per_sample_fastqs[sample_id].name, mode='a')
+
+ fastq_lines = '\n'.join(sequence_record) + '\n'
+ fastq_lines = fastq_lines.encode('utf-8')
+ per_sample_fastqs[sample_id].write(fastq_lines)
if len(per_sample_fastqs) == 0:
raise ValueError('No sequences were mapped to samples. Check that '
@@ -410,74 +417,74 @@ def emp_paired(seqs: BarcodePairedSequenceFastqIterator,
per_sample_fastqs = {}
ec_details_fmt = ErrorCorrectionDetailsFmt()
- ec_details = ECDetails(ec_details_fmt)
-
- for i, record in enumerate(seqs, start=1):
- barcode_record, forward_record, reverse_record = record
- raw_barcode_read = barcode_record[1][:barcode_len]
- if rev_comp_barcodes:
- barcode_as_DNA = skbio.DNA(raw_barcode_read)
- raw_barcode_read = str(barcode_as_DNA.reverse_complement())
-
- if golay_error_correction:
- # A three bit filter is implicitly used by the decoder. See Hamady
- # and Knight 2009 Genome Research for the justification:
- #
- # https://genome.cshlp.org/content/19/7/1141.full
- #
- # Specifically that "...Golay codes of 12 bases can correct all
- # triple-bit errors and detect all quadruple-bit errors."
- barcode_read, ecc_errors = decoder.decode(raw_barcode_read)
- golay_stats = [barcode_read, ecc_errors]
- else:
- barcode_read = raw_barcode_read
- golay_stats = [None, None]
-
- sample_id = barcode_map.get(barcode_read)
-
- record = [
- f'record-{i}',
- sample_id,
- barcode_record[0],
- raw_barcode_read,
- ]
- ec_details.write(record + golay_stats)
-
- if sample_id is None:
- continue
+ with ECDetails(ec_details_fmt) as ec_details:
+
+ for i, record in enumerate(seqs, start=1):
+ barcode_record, forward_record, reverse_record = record
+ raw_barcode_read = barcode_record[1][:barcode_len]
+ if rev_comp_barcodes:
+ barcode_as_DNA = skbio.DNA(raw_barcode_read)
+ raw_barcode_read = str(barcode_as_DNA.reverse_complement())
+
+ if golay_error_correction:
+ # A three bit filter is implicitly used by the decoder. See
+ # Hamady and Knight 2009 Genome Research for the justification:
+ #
+ # https://genome.cshlp.org/content/19/7/1141.full
+ #
+ # Specifically that "...Golay codes of 12 bases can correct all
+ # triple-bit errors and detect all quadruple-bit errors."
+ barcode_read, ecc_errors = decoder.decode(raw_barcode_read)
+ golay_stats = [barcode_read, ecc_errors]
+ else:
+ barcode_read = raw_barcode_read
+ golay_stats = [None, None]
+
+ sample_id = barcode_map.get(barcode_read)
+
+ record = [
+ f'record-{i}',
+ sample_id,
+ barcode_record[0],
+ raw_barcode_read,
+ ]
+ ec_details.write(record + golay_stats)
+
+ if sample_id is None:
+ continue
+
+ if sample_id not in per_sample_fastqs:
+ barcode_id = len(per_sample_fastqs) + 1
+ fwd_path = result.sequences.path_maker(sample_id=sample_id,
+ barcode_id=barcode_id,
+ lane_number=1,
+ read_number=1)
+ rev_path = result.sequences.path_maker(sample_id=sample_id,
+ barcode_id=barcode_id,
+ lane_number=1,
+ read_number=2)
+
+ _maintain_open_fh_count(per_sample_fastqs, paired=True)
+ per_sample_fastqs[sample_id] = (
+ gzip.open(str(fwd_path), mode='a'),
+ gzip.open(str(rev_path), mode='a')
+ )
+ manifest_fh.write('%s,%s,%s\n' % (sample_id, fwd_path.name,
+ 'forward'))
+ manifest_fh.write('%s,%s,%s\n' % (sample_id, rev_path.name,
+ 'reverse'))
+
+ if per_sample_fastqs[sample_id][0].closed:
+ _maintain_open_fh_count(per_sample_fastqs, paired=True)
+ fwd, rev = per_sample_fastqs[sample_id]
+ per_sample_fastqs[sample_id] = (
+ gzip.open(fwd.name, mode='a'),
+ gzip.open(rev.name, mode='a')
+ )
- if sample_id not in per_sample_fastqs:
- barcode_id = len(per_sample_fastqs) + 1
- fwd_path = result.sequences.path_maker(sample_id=sample_id,
- barcode_id=barcode_id,
- lane_number=1,
- read_number=1)
- rev_path = result.sequences.path_maker(sample_id=sample_id,
- barcode_id=barcode_id,
- lane_number=1,
- read_number=2)
-
- _maintain_open_fh_count(per_sample_fastqs, paired=True)
- per_sample_fastqs[sample_id] = (
- gzip.open(str(fwd_path), mode='a'),
- gzip.open(str(rev_path), mode='a')
- )
- manifest_fh.write('%s,%s,%s\n' % (sample_id, fwd_path.name,
- 'forward'))
- manifest_fh.write('%s,%s,%s\n' % (sample_id, rev_path.name,
- 'reverse'))
-
- if per_sample_fastqs[sample_id][0].closed:
- _maintain_open_fh_count(per_sample_fastqs, paired=True)
fwd, rev = per_sample_fastqs[sample_id]
- per_sample_fastqs[sample_id] = (
- gzip.open(fwd.name, mode='a'),
- gzip.open(rev.name, mode='a')
- )
-
- fwd, rev = per_sample_fastqs[sample_id]
- fwd.write(('\n'.join(forward_record) + '\n').encode('utf-8'))
- rev.write(('\n'.join(reverse_record) + '\n').encode('utf-8'))
+ fwd.write(('\n'.join(forward_record) + '\n').encode('utf-8'))
+ rev.write(('\n'.join(reverse_record) + '\n').encode('utf-8'))
if len(per_sample_fastqs) == 0:
raise ValueError('No sequences were mapped to samples. Check that '
=====================================
q2_demux/_examples.py
=====================================
@@ -0,0 +1,57 @@
+# ----------------------------------------------------------------------------
+# Copyright (c) 2016-2022, QIIME 2 development team.
+#
+# Distributed under the terms of the Modified BSD License.
+#
+# The full license is in the file LICENSE, distributed with this software.
+# ----------------------------------------------------------------------------
+
+import qiime2
+
+
+emp_seq_url = ('https://data.qiime2.org/usage-examples/'
+ 'moving-pictures/emp-single-end-sequences.qza')
+
+demux_url = 'https://data.qiime2.org/usage-examples/moving-pictures/demux.qza'
+
+metadata_url = (f'https://data.qiime2.org/{qiime2.__release__}/tutorials/'
+ 'moving-pictures/sample_metadata.tsv')
+
+
+def emp_single(use):
+ sequences = use.init_artifact_from_url('sequences', emp_seq_url)
+ metadata = use.init_metadata_from_url('sample_metadata', metadata_url)
+ metadata_column = use.get_metadata_column('metadata_column',
+ 'barcode-sequence',
+ metadata)
+
+ demux, correction_details = use.action(
+ use.UsageAction('demux', 'emp_single'),
+ use.UsageInputs(
+ seqs=sequences,
+ barcodes=metadata_column
+ ),
+ use.UsageOutputNames(
+ per_sample_sequences='demux',
+ error_correction_details='demux-details'
+ )
+ )
+
+ demux.assert_output_type('SampleData[SequencesWithQuality]')
+ correction_details.assert_output_type('ErrorCorrectionDetails')
+
+
+def summarize(use):
+ demux = use.init_artifact_from_url('demux', demux_url)
+
+ viz, = use.action(
+ use.UsageAction('demux', 'summarize'),
+ use.UsageInputs(
+ data=demux
+ ),
+ use.UsageOutputNames(
+ visualization='visualization'
+ )
+ )
+
+ viz.assert_output_type('Visualization')
=====================================
q2_demux/_version.py
=====================================
@@ -23,9 +23,9 @@ def get_keywords():
# setup.py/versioneer.py will grep for the variable names, so they must
# each be defined on a line of their own. _version.py will just call
# get_keywords().
- git_refnames = " (tag: 2022.8.0)"
- git_full = "d5a13a4137d2f7c358d7fa992f7ca504f29620fe"
- git_date = "2022-08-23 17:11:46 +0000"
+ git_refnames = " (HEAD -> master, tag: 2022.11.1)"
+ git_full = "ff7e334fc199c01ec17962dcec829836cf62a073"
+ git_date = "2022-12-21 22:00:27 +0000"
keywords = {"refnames": git_refnames, "full": git_full, "date": git_date}
return keywords
=====================================
q2_demux/plugin_setup.py
=====================================
@@ -25,6 +25,8 @@ from ._format import (EMPMultiplexedDirFmt, ErrorCorrectionDetailsDirFmt,
EMPSingleEndDirFmt, EMPSingleEndCasavaDirFmt,
EMPPairedEndDirFmt, EMPPairedEndCasavaDirFmt)
+import q2_demux._examples as ex
+
citations = Citations.load('citations.bib', package='q2_demux')
plugin = Plugin(
@@ -107,6 +109,7 @@ plugin.methods.register_function(
'Project (EMP) amplicon sequencing protocol. Details about '
'this protocol can be found at '
'http://www.earthmicrobiome.org/protocols-and-standards/'),
+ examples={'demux': ex.emp_single},
citations=[
citations['hamady2008'],
citations['hamady2009']]
@@ -178,7 +181,8 @@ plugin.visualizers.register_function(
name='Summarize counts per sample.',
description=('Summarize counts per sample for all samples, and generate '
'interactive positional quality plots based on `n` randomly '
- 'selected sequences.')
+ 'selected sequences.'),
+ examples={'demux': ex.summarize}
)
plugin.methods.register_function(
=====================================
q2_demux/tests/test_demux.py
=====================================
@@ -31,6 +31,13 @@ from q2_demux._summarize._visualizer import (_PlotQualView,
_decode_qual_to_phred33)
+class TestBase(TestPluginBase):
+ package = 'q2_demux.tests'
+
+ def test_examples(self):
+ self.execute_examples()
+
+
class BarcodeSequenceFastqIteratorTests(unittest.TestCase):
def test_valid(self):
=====================================
setup.py
=====================================
@@ -42,7 +42,7 @@ setup(
'data/summarize_empty/empty_forward_in_paired/*',
'data/summarize_empty/empty_reverse_in_paired/*',
'data/summarize_empty/empty_paired_end/*',
- 'data/reverse_only/*'
+ 'data/reverse_only/*',
],
'q2_demux': ['_summarize/assets/*.html',
'_summarize/assets/dist/*',
View it on GitLab: https://salsa.debian.org/med-team/q2-demux/-/compare/baab5a37f3fd538a754e6ac28d79f9ab0f95a1c9...cf00cb86e23b710b661e0969295d7a1fa2804da3
--
View it on GitLab: https://salsa.debian.org/med-team/q2-demux/-/compare/baab5a37f3fd538a754e6ac28d79f9ab0f95a1c9...cf00cb86e23b710b661e0969295d7a1fa2804da3
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/debian-med-commit/attachments/20230112/b0d8d35d/attachment-0001.htm>
More information about the debian-med-commit
mailing list