[med-svn] [Git][med-team/changeo][upstream] New upstream version 1.3.5

Andreas Tille (@tille) gitlab at salsa.debian.org
Sun Nov 15 21:03:44 GMT 2026



Andreas Tille pushed to branch upstream at Debian Med / changeo


Commits:
421d5388 by Andreas Tille at 2026-11-15T17:50:36+01:00
New upstream version 1.3.5
- - - - -


30 changed files:

- INSTALL.rst
- MANIFEST.in
- NEWS.rst
- PKG-INFO
- README.rst
- bin/AssignGenes.py
- bin/BuildTrees.py
- bin/ConvertDb.py
- bin/CreateGermlines.py
- bin/DefineClones.py
- bin/MakeDb.py
- bin/ParseDb.py
- changeo.egg-info/PKG-INFO
- changeo.egg-info/SOURCES.txt
- changeo.egg-info/requires.txt
- changeo/Applications.py
- changeo/Commandline.py
- changeo/Distance.py
- changeo/Gene.py
- changeo/IO.py
- changeo/Multiprocessing.py
- changeo/Version.py
- + pyproject.toml
- requirements.txt
- setup.py
- + tests/test_DefineClones.py
- + tests/test_Distance.py
- + tests/test_IO.py
- + tests/test_ParseDb.py
- + tests/test_Receptor.py


Changes:

=====================================
INSTALL.rst
=====================================
@@ -7,7 +7,7 @@ The simplest way to install the latest stable release of Change-O is via pip::
 
 The current development build can be installed using pip and git in similar fashion::
 
-    > pip3 install git+https://bitbucket.org/kleinstein/changeo@master --user
+    > pip3 install git+https://github.com/immcantation/changeo@master --user
 
 If you currently have a development version installed, then you will likely
 need to add the arguments ``--upgrade --no-deps --force-reinstall`` to the
@@ -18,14 +18,17 @@ Requirements
 
 The minimum dependencies for installation are:
 
-+ `Python 3.4.0 <http://python.org>`__
-+ `setuptools 2.0 <http://bitbucket.org/pypa/setuptools>`__
-+ `NumPy 1.8 <http://numpy.org>`__
-+ `SciPy 0.14 <http://scipy.org>`__
-+ `pandas 0.24 <http://pandas.pydata.org>`__
-+ `Biopython 1.77 <http://biopython.org>`__
-+ `presto 0.7.0 <http://presto.readthedocs.io>`__
++ `Python 3.10.0 <http://python.org>`__
++ `setuptools 65.5 <http://bitbucket.org/pypa/setuptools>`__
++ `NumPy 1.23.2 <http://numpy.org>`__
++ `SciPy 1.9.3 <http://scipy.org>`__
++ `pandas 1.5.0 <http://pandas.pydata.org>`__
++ `Biopython 1.81 <http://biopython.org>`__
++ `presto 0.7.1 <http://presto.readthedocs.io>`__
 + `airr 1.3.1 <https://docs.airr-community.org>`__
++ `PyYAML 6.0 <http://pyyaml.org>`__
++ `packaging 21.3 <https://packaging.pypa.io>`__
++ `importlib-resoures 6.4.0 <https://pypi.org/project/importlib-resources>`__
 
 Some tools wrap external applications that are not required for installation.
 Those tools require minimum versions of:
@@ -34,7 +37,7 @@ Those tools require minimum versions of:
 + ConvertDb-genbank requires `tbl2asn <https://www.ncbi.nlm.nih.gov/genbank/tbl2asn2>`__
 + AssignGenes requires `IgBLAST 1.6 <https://ncbi.github.io/igblast>`__, but
   version 1.11 or higher is strongly recommended.
-+ BuildTrees requires `IgPhyML 1.0.5 <https://bitbucket.org/kbhoehn/igphyml>`_
++ BuildTrees requires `IgPhyML 1.0.5 <https://github.com/immcantation/igphyml>`_
 
 Linux
 --------------------------------------------------------------------------------
@@ -45,9 +48,10 @@ Linux
    Biopython according to its
    `instructions <http://biopython.org/DIST/docs/install/Installation.html>`__.
 
-2. Install `presto 0.6.2 <http://presto.readthedocs.io>`__ or greater.
+2. Install `presto 0.7.0 <http://presto.readthedocs.io>`__ or greater.
 
-3. Download the Change-O bundle and run::
+3. Download the `Change-O bundle <https://github.com/immcantation/changeo/tags>`__
+   and run::
 
    > pip3 install changeo-x.y.z.tar.gz --user
 
@@ -86,10 +90,10 @@ Mac OS X
 
    > pip3 install numpy scipy pandas biopython
 
-8. Install `presto 0.6.2 <http://presto.readthedocs.io>`__ or greater.
+8. Install `presto 0.7.0 <http://presto.readthedocs.io>`__ or greater.
 
-9. Download the Change-O bundle, open a terminal window, change directories
-   to the download folder, and run::
+9. Download the `Change-O bundle <https://github.com/immcantation/changeo/tags>`__, 
+   open a terminal window, change directories to the download folder, and run::
 
    > pip3 install changeo-x.y.z.tar.gz
 
@@ -104,10 +108,10 @@ Windows
    `Unofficial Windows binary <http://www.lfd.uci.edu/~gohlke/pythonlibs>`__
    collection.
 
-3. Install `presto 0.6.2 <http://presto.readthedocs.io>`__ or greater.
+3. Install `presto 0.7.0 <http://presto.readthedocs.io>`__ or greater.
 
-4. Download the Change-O bundle, open a Command Prompt, change directories to
-   the download folder, and run::
+4. Download the `Change-O bundle <https://github.com/immcantation/changeo/tags>`__, 
+   open a Command Prompt, change directories to the download folder, and run::
 
    > pip install changeo-x.y.z.tar.gz
 


=====================================
MANIFEST.in
=====================================
@@ -2,4 +2,5 @@ include requirements.txt
 include README.rst
 include INSTALL.rst
 include NEWS.rst
-include LICENSE
\ No newline at end of file
+include LICENSE
+exclude announcements.html
\ No newline at end of file


=====================================
NEWS.rst
=====================================
@@ -1,6 +1,86 @@
 Release Notes
 ===============================================================================
 
+Version 1.3.5:  September 2, 2026
+-------------------------------------------------------------------------------
+
+General:
+
++ Added support for gzip compressed (``.gz``) repertoire files. Compressed input
+  is detected automatically, and output is compressed to match whenever the
+  input is compressed. A new ``--gzip-output`` argument forces gzip compressed
+  output regardless of the input. Supported by ``AssignGenes.py``,
+  ``BuildTrees.py``, ``ConvertDb.py``, ``CreateGermlines.py``,
+  ``DefineClones.py``, ``MakeDb.py`` and ``ParseDb.py``. ``AssignGenes.py``
+  supports compressed input only, as its output is written directly by the
+  wrapped IgBLAST/IgBLAST-aa executables.
+
+CreateGermlines:
+
++ ``CreateGermlines.py`` now validates the sequence field and raises an
+  informative error if amino acid sequences are detected. Amino acid sequences
+  are not supported; a nucleotide sequence field must be provided.
+
+MakeDb:
+
++ ``MakeDb.py igblast-aa`` now reports all top-scoring ``v_call`` hits in the
+  event of ties. Previously, only the first top-scoring hit was reported. The
+  new behavior aligns with that of the ``igblast`` subcommand.
+
+Gene:
+
++ Added support for dual-locus ``TRA/TRD`` gene detection. ``getLocus`` now
+  correctly identifies genes with the ``TRAV.../DV...`` naming convention
+  (e.g., ``TRAV14/DV4``) and returns ``TRA/TRD`` as the locus.
++ Fixed a bug where ``getLocus`` would raise a ``TypeError`` when the gene
+  call was ``None``.
+
+ParseDb:
+
++ ``ParseDb.py merge`` now accepts ``-f`` and ``-u`` arguments to annotate
+  merged records with a sample identifier field, with one value per input
+  file specified via ``-d``.
++ ``ParseDb.py merge`` now issues a warning when ``-f`` specifies a field that
+  already exists in one or more input files, as its values will be
+  overwritten, and when ``-u`` values are provided without a ``-f`` field to
+  assign them to, in which case the values are ignored.
+
+
+Version 1.3.4:  July 31, 2025
+-------------------------------------------------------------------------------
+
+MakeDb:
+
++ ``MakeDb.py`` is now parallelized across input files. The ``--nproc`` argument 
+   can be used to restrict the resources used.
++ Added the flag ``--partial`` to ``MakeDb igblast-aa`` to pass incomplete alignment
+  results. As ``igblastp`` (as at igblast 1.22.0) only uses the V germline 
+  database, all sequences will be missing the ``junction`` and ``j_call`` fields 
+  and be considered incomplete. Specifying ``--partial`` will allow these 
+  sequences to be processed ignoring the missing fields.
+
+Documentation:
+
++ Added a "Contributing" section to the documentation menu for community guidelines.
++ Updated "Contact".
+
+Version 1.3.3:  May 14, 2025
+-------------------------------------------------------------------------------
+
++ Updated dependencies to address deprecation warnings. Replaced ``pkg_resources``
+  with ``packaging`` and ``importlib``.
++ Bumped minimum Python version to 3.10.0 and updated requirements to: numpy>=1.23.2,
+  scipy>=1.9.3, pandas>=1.5.0, biopython>=1.81, PyYAML>=6.0, setuptools>=65.5.0,
+  presto>=0.7.1, airr>=1.3.1, packaging>=21.3, importlib-resources>=6.4.0.
+
+Version 1.3.1:  March 27, 2025
+-------------------------------------------------------------------------------
+
++ Active development has moved from Bitbucket to GitHub (https://github.com/immcantation/changeo).
++ Documentation updates.
++ Various updates to internals to avoid deprecation warnings.
++ Updates in requirements: Python 3.7.0, biopython>=1.81 and packaging>=23.2.
+
 Version 1.3.0:  December 11, 2022
 -------------------------------------------------------------------------------
 
@@ -58,6 +138,7 @@ Version 1.1.0:  June 21, 2021
 + Updated dependencies to biopython >= v1.77, airr >= v1.3.1, PyYAML>=5.1.
 
 MakeDb:
+
 + Added the ``--imgt-id-len`` argument to accommodate changes introduced in how
   IMGT/HighV-QUEST truncates sequence identifiers as of v1.8.3 (May 7, 2021).
   The header lines in the fasta files are now truncated to 49 characters. In


=====================================
PKG-INFO
=====================================
@@ -1,9 +1,9 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: changeo
-Version: 1.3.0
+Version: 1.3.5
 Summary: A bioinformatics toolkit for processing high-throughput lymphocyte receptor sequencing data.
 Home-page: http://changeo.readthedocs.io
-Download-URL: https://bitbucket.org/kleinstein/changeo/downloads
+Download-URL: https://github.com/immcantation/changeo/tags
 Author: Namita Gupta, Jason Anthony Vander Heiden
 Author-email: immcantation at googlegroups.com
 License: GNU Affero General Public License 3 (AGPL-3)
@@ -13,9 +13,30 @@ Classifier: Environment :: Console
 Classifier: Intended Audience :: Science/Research
 Classifier: Natural Language :: English
 Classifier: Operating System :: OS Independent
-Classifier: Programming Language :: Python :: 3.4
+Classifier: Programming Language :: Python :: 3.10
 Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
 License-File: LICENSE
+Requires-Dist: numpy>=1.23.2
+Requires-Dist: scipy>=1.9.3
+Requires-Dist: pandas>=1.5.0
+Requires-Dist: biopython>=1.81
+Requires-Dist: PyYAML>=6.0
+Requires-Dist: setuptools>=65.5.0
+Requires-Dist: presto>=0.7.1
+Requires-Dist: airr>=1.3.1
+Requires-Dist: packaging>=21.3
+Requires-Dist: importlib-resources>=6.4.0
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: download-url
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: summary
 
 .. image:: https://img.shields.io/pypi/dm/changeo
     :target: https://pypi.org/project/changeo
@@ -28,7 +49,7 @@ Change-O - Repertoire clonal assignment toolkit
 Change-O is a collection of tools for processing the output of V(D)J alignment
 tools, assigning clonal clusters to immunoglobulin (Ig) sequences, and
 reconstructing germline sequences.
- 
+
 Dramatic improvements in high-throughput sequencing technologies now enable 
 large-scale characterization of Ig repertoires, defined as the collection of
 trans-membrane antigen-receptor proteins located on the surface of B cells and
@@ -37,3 +58,6 @@ Ig and TCR sequences following germline segment assignment. Change-O
 handles output from IMGT/HighV-QUEST and IgBLAST, and provides a wide variety of
 clustering methods for assigning clonal groups to Ig sequences. Record sorting, 
 grouping, and various database manipulation operations are also included.
+
+**IMPORTANT!** 
+Active development of Change-O has moved to https://github.com/immcantation/changeo


=====================================
README.rst
=====================================
@@ -9,7 +9,7 @@ Change-O - Repertoire clonal assignment toolkit
 Change-O is a collection of tools for processing the output of V(D)J alignment
 tools, assigning clonal clusters to immunoglobulin (Ig) sequences, and
 reconstructing germline sequences.
- 
+
 Dramatic improvements in high-throughput sequencing technologies now enable 
 large-scale characterization of Ig repertoires, defined as the collection of
 trans-membrane antigen-receptor proteins located on the surface of B cells and
@@ -18,3 +18,6 @@ Ig and TCR sequences following germline segment assignment. Change-O
 handles output from IMGT/HighV-QUEST and IgBLAST, and provides a wide variety of
 clustering methods for assigning clonal groups to Ig sequences. Record sorting, 
 grouping, and various database manipulation operations are also included.
+
+**IMPORTANT!** 
+Active development of Change-O has moved to https://github.com/immcantation/changeo


=====================================
bin/AssignGenes.py
=====================================
@@ -11,7 +11,7 @@ import os
 import shutil
 from argparse import ArgumentParser
 from collections import OrderedDict
-from pkg_resources import parse_version
+from packaging.version import parse as parse_version
 from textwrap import dedent
 from time import time
 import re
@@ -91,8 +91,8 @@ def assignIgBLAST(seq_file, amino_acid=False, igdata=default_igdata, loci='ig',
         out_file = getOutputName(seq_file, out_label='igblast', out_dir=out_args['out_dir'],
                                  out_name=out_args['out_name'], out_type=out_type)
     # convert to FASTA if needed
-    infile = open(seq_file, 'r')
-    test = infile.read()[0]
+    with open(seq_file, 'r') as infile:
+        test = infile.read()[0]
     if test == "@":
         printMessage("Running conversion from FASTQ to FASTA")
         fasta_out_dir, filename = os.path.split(out_file)
@@ -129,7 +129,7 @@ def assignIgBLAST(seq_file, amino_acid=False, igdata=default_igdata, loci='ig',
             while f.read(1) != b'\n':
                 f.seek(-2, os.SEEK_CUR)
             pass_info = f.readline().decode()
-        num_seqs_match = re.search('(# BLAST processed )(\d+)( .*)', pass_info)
+        num_seqs_match = re.search(r'(# BLAST processed )(\d+)( .*)', pass_info)
         num_sequences = num_seqs_match.group(2)
     else:
         f = open(out_file, 'rb')
@@ -183,7 +183,11 @@ def getArgParser():
     subparsers.required = True
 
     # Parent parser
-    parent_parser = getCommonArgParser(db_in=False, log=False, failed=False, format=False, multiproc=True)
+    # gzip_output is intentionally excluded: assignIgBLAST() writes its output through the
+    # igblastn/igblastp executable directly, not through getOutputHandle/openFile, so the
+    # option would otherwise be silently accepted and ignored.
+    parent_parser = getCommonArgParser(db_in=False, log=False, failed=False, format=False,
+                                       multiproc=True, gzip_output=False)
 
     # Subparser to run igblastn
     parser_igblast = subparsers.add_parser('igblast', parents=[parent_parser],


=====================================
bin/BuildTrees.py
=====================================
@@ -23,7 +23,8 @@ from functools import partial
 from presto.Defaults import default_out_args
 from presto.IO import  printLog, printMessage, printWarning, printError, printDebug
 from changeo.Defaults import default_format
-from changeo.IO import splitName, getDbFields, getFormatOperators, getOutputHandle, getOutputName
+from changeo.IO import splitName, getDbFields, getFormatOperators, getOutputHandle, getOutputName, \
+                       openFile
 from changeo.Alignment import RegionDefinition
 from changeo.Commandline import CommonHelpFormatter, checkArgs, getCommonArgParser, parseCommonArgs
 
@@ -1054,7 +1055,8 @@ def buildTrees(db_file, meta_data=None, target_clones=None, collapse=False, ncdr
                                   out_label=out_label,
                                   out_dir=out_args["out_dir"],
                                   out_name= out_args["out_name"],
-                                  out_type="tsv")
+                                  out_type="tsv",
+                                  gzip_output=out_args.get("gzip_output"))
 
     igphyml_out = None
     if igphyml:
@@ -1084,7 +1086,7 @@ def buildTrees(db_file, meta_data=None, target_clones=None, collapse=False, ncdr
     out_fields = getDbFields(db_file, reader=reader)
 
     # open input file
-    handle = open(db_file, "r")
+    handle = openFile(db_file, "r")
     records = reader(handle)
 
     fail_handle, fail_writer = None, None
@@ -1093,7 +1095,8 @@ def buildTrees(db_file, meta_data=None, target_clones=None, collapse=False, ncdr
                                       out_label="lineages-fail",
                                       out_dir=out_args["out_dir"],
                                       out_name=out_args["out_name"],
-                                      out_type=out_args["out_type"])
+                                      out_type=out_args["out_type"],
+                                      gzip_output=out_args.get("gzip_output"))
         fail_writer = writer(fail_handle, fields=out_fields)
 
     cloneseqs = {}


=====================================
bin/ConvertDb.py
=====================================
@@ -29,9 +29,9 @@ from changeo.Defaults import default_id_field, default_seq_field, default_germ_f
                              default_csv_size, default_format, default_out_args
 from changeo.Commandline import CommonHelpFormatter, checkArgs, getCommonArgParser, parseCommonArgs
 from changeo.Gene import getCGene
-from changeo.IO import countDbFile, getFormatOperators, getOutputHandle, AIRRReader, AIRRWriter, \
-                       ChangeoReader, ChangeoWriter, TSVReader, ReceptorData, readGermlines, \
-                       checkFields, yamlDict
+from changeo.IO import countDbFile, getFormatOperators, getOutputHandle, gzipOutputName, openFile, \
+                       AIRRReader, AIRRWriter, ChangeoReader, ChangeoWriter, TSVReader, ReceptorData, \
+                       readGermlines, checkFields, yamlDict
 from changeo.Receptor import AIRRSchema, ChangeoSchema
 
 # System settings
@@ -101,7 +101,7 @@ def convertToAIRR(db_file, format=default_format,
         printError('Invalid format %s.' % format)
 
     # Open input
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = reader(db_handle)
 
     # Set output fields replacing length with end fields
@@ -116,10 +116,11 @@ def convertToAIRR(db_file, format=default_format,
 
     # Open output writer
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(db_file, out_label='airr', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type=AIRRSchema.out_type)
+                                      out_name=out_args['out_name'], out_type=AIRRSchema.out_type,
+                                      gzip_output=out_args.get('gzip_output'))
     pass_writer = AIRRWriter(pass_handle, fields=out_fields)
 
     # Count records
@@ -169,7 +170,7 @@ def convertToChangeo(db_file, out_file=None, out_args=default_out_args):
     printLog(log)
 
     # Open input
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = AIRRReader(db_handle)
 
     # Set output fields replacing length with end fields
@@ -184,10 +185,11 @@ def convertToChangeo(db_file, out_file=None, out_args=default_out_args):
 
     # Open output writer
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(db_file, out_label='changeo', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type=ChangeoSchema.out_type)
+                                      out_name=out_args['out_name'], out_type=ChangeoSchema.out_type,
+                                      gzip_output=out_args.get('gzip_output'))
     pass_writer = ChangeoWriter(pass_handle, fields=out_fields)
 
     # Count records
@@ -252,16 +254,17 @@ def convertToBaseline(db_file, id_field=default_id_field, seq_field=default_seq_
     printLog(log)
     
     # Open input
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = TSVReader(db_handle)
     result_count = countDbFile(db_file)
 
     # Open output
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(db_file, out_label='sequences', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type='clip')
+                                      out_name=out_args['out_name'], out_type='clip',
+                                      gzip_output=out_args.get('gzip_output'))
     # Iterate over records
     start_time = time()
     rec_count, germ_count, pass_count, fail_count = 0, 0, 0, 0
@@ -347,16 +350,17 @@ def convertToFasta(db_file, id_field=default_id_field, seq_field=default_seq_fie
     
     # Open input
     out_type = 'fasta'
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = TSVReader(db_handle)
     result_count = countDbFile(db_file)
 
     # Open output
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(db_file, out_label='sequences', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type=out_type)
+                                      out_name=out_args['out_name'], out_type=out_type,
+                                      gzip_output=out_args.get('gzip_output'))
 
     # Iterate over records
     start_time = time()
@@ -693,7 +697,7 @@ def convertToGenbank(db_file, inference=None, db_xref=None, molecule=default_mol
         printError('Invalid format %s.' % format)
 
     # Open input
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = reader(db_handle)
 
     # Check for required columns


=====================================
bin/CreateGermlines.py
=====================================
@@ -1,6 +1,8 @@
 #!/usr/bin/env python3
 """
-Reconstructs germline sequences from alignment data
+Reconstructs germline sequences from nucleotide alignment data
+
+Note: Amino acid sequences are not currently supported.
 """
 
 # Info
@@ -24,7 +26,7 @@ from changeo.Commandline import CommonHelpFormatter, checkArgs, getCommonArgPars
                                 setDefaultFields
 from changeo.Gene import buildGermline, buildClonalGermline
 from changeo.IO import countDbFile, getDbFields, getFormatOperators, getOutputHandle, readGermlines, \
-                       checkFields
+                       gzipOutputName, openFile, checkFields
 
 # Defaults
 default_germ_types = ['dmask']
@@ -40,7 +42,7 @@ def createGermlines(db_file, references, seq_field=default_seq_field, v_field=de
     Arguments:
       db_file : input tab-delimited database file.
       references : folders and/or files containing germline repertoire data in FASTA format.
-      seq_field : field in which to look for sequence.
+      seq_field : field in which to look for sequence. Must be a nucleotide sequence field.
       v_field : field in which to look for V call.
       d_field : field in which to look for D call.
       j_field : field in which to look for J call.
@@ -53,6 +55,9 @@ def createGermlines(db_file, references, seq_field=default_seq_field, v_field=de
 
     Returns:
       dict: names of the 'pass' and 'fail' output files.
+      
+    Note:
+      Amino acid sequences are not currently supported. The seq_field must contain nucleotide sequences.
     """
     # Print parameter info
     log = OrderedDict()
@@ -92,7 +97,7 @@ def createGermlines(db_file, references, seq_field=default_seq_field, v_field=de
 
     # Get repertoire and open Db reader
     reference_dict = readGermlines(references)
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = reader(db_handle)
 
     # Check for required columns
@@ -115,6 +120,32 @@ def createGermlines(db_file, references, seq_field=default_seq_field, v_field=de
         if f not in db_iter.fields:
             printError('%s field does not exist in input database file.' % f)
 
+    # Check for amino acid sequences
+    aa_fields = ['sequence_aa', 'sequence_alignment_aa', 'SEQUENCE_AA', 'SEQUENCE_ALIGNMENT_AA']
+    if seq_field in aa_fields:
+        printError('Reconstruction of germlines from amino acid sequences is not currently supported. '
+                   'Please use a nucleotide sequence field (e.g., sequence_alignment or SEQUENCE_IMGT).')
+    
+    # Also check actual sequence content to detect amino acids
+    # Sample first few sequences to detect amino acid-specific characters
+    aa_chars = set('EFIKLMPQWY')  # Amino acid characters not valid in nucleotide sequences
+    sample_count = 0
+    max_samples = 10
+    for rec in db_iter:
+        seq = rec.getField(seq_field)
+        if seq and any(c in aa_chars for c in seq.upper().replace('.', '').replace('-', '')):
+            printError('Amino acid sequence detected in %s field. '
+                       'Reconstruction of germlines from amino acid sequences is not currently supported. '
+                       'Please use a nucleotide sequence field.' % seq_field)
+        sample_count += 1
+        if sample_count >= max_samples:
+            break
+    
+    # Reset the file handle after sampling
+    db_handle.close()
+    db_handle = openFile(db_file, 'rt')
+    db_iter = reader(db_handle)
+
     # Translate to Receptor attribute names
     v_field = schema.toReceptor(v_field)
     d_field = schema.toReceptor(d_field)
@@ -186,13 +217,14 @@ def createGermlines(db_file, references, seq_field=default_seq_field, v_field=de
             except AttributeError:
                 # Create output file handle and writer
                 if out_file is not None:
-                    pass_handle = open(out_file, 'w')
+                    pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
                 else:
                     pass_handle = getOutputHandle(db_file,
                                                   out_label='germ-pass',
                                                   out_dir=out_args['out_dir'],
                                                   out_name=out_args['out_name'],
-                                                  out_type=out_args['out_type'])
+                                                  out_type=out_args['out_type'],
+                                                  gzip_output=out_args.get('gzip_output'))
                 pass_writer = writer(pass_handle, fields=out_fields)
                 for r in records:
                     r.setDict(annotations)
@@ -207,7 +239,8 @@ def createGermlines(db_file, references, seq_field=default_seq_field, v_field=de
                                                   out_label='germ-fail',
                                                   out_dir=out_args['out_dir'],
                                                   out_name=out_args['out_name'],
-                                                  out_type=out_args['out_type'])
+                                                  out_type=out_args['out_type'],
+                                                  gzip_output=out_args.get('gzip_output'))
                     fail_writer = writer(fail_handle, fields=out_fields)
                     fail_writer.writeReceptor(records)
 
@@ -299,7 +332,8 @@ def getArgParser():
                              germline call used for the entire clone within the
                              germline_v_call, germline_d_call and germline_j_call fields.''')
     group.add_argument('--sf', action='store', dest='seq_field', default=None,
-                        help='''Field containing the aligned sequence.
+                        help='''Field containing the aligned sequence. Must be a nucleotide sequence field.
+                             Amino acid sequences are not currently supported.
                              Defaults to sequence_alignment (airr) or SEQUENCE_IMGT (changeo).''')
     group.add_argument('--vf', action='store', dest='v_field', default=None,
                         help='''Field containing the germline V segment call.


=====================================
bin/DefineClones.py
=====================================
@@ -28,7 +28,7 @@ from changeo.Commandline import CommonHelpFormatter, checkArgs, getCommonArgPars
                                 setDefaultFields
 from changeo.Distance import distance_models, calcDistances, formClusters
 from changeo.IO import countDbFile, getDbFields, getFormatOperators, getOutputHandle, \
-                       AIRRWriter, checkFields
+                       gzipOutputName, openFile, AIRRWriter, checkFields
 from changeo.Multiprocessing import DbResult, feedDbQueue, processDbQueue
 
 # Defaults
@@ -299,7 +299,7 @@ def distanceClones(result, seq_field=default_junction_field, model=default_dista
     seq_map = {}
     for rec in result.data_pass:
         seq = rec.getField(seq_field)
-        seq = re.sub('[\.-]', 'N', seq)
+        seq = re.sub(r'[\.-]', 'N', seq)
         # Translate sequence for amino acid model
         if model == 'aa':
             # Check for valid translation
@@ -367,13 +367,14 @@ def collectQueue(alive, result_queue, collect_queue, db_file, fields,
     # Wrapper for opening handles and writers
     def _open(x, f, writer=writer, out_file=out_file):
         if out_file is not None and x == 'pass':
-            handle = open(out_file, 'w')
+            handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
         else:
             handle = getOutputHandle(db_file,
                                      out_label='clone-%s' % x,
                                      out_dir=out_args['out_dir'],
                                      out_name=out_args['out_name'],
-                                     out_type=out_args['out_type'])
+                                     out_type=out_args['out_type'],
+                                     gzip_output=out_args.get('gzip_output'))
         return handle, writer(handle, fields=f)
 
     # Open log file


=====================================
bin/MakeDb.py
=====================================
@@ -16,6 +16,8 @@ from collections import OrderedDict
 from textwrap import dedent
 from time import time
 from Bio import SeqIO
+from concurrent.futures import ProcessPoolExecutor
+from concurrent.futures import as_completed
 
 # Presto and changeo imports
 from presto.Annotation import parseAnnotation
@@ -25,8 +27,8 @@ from changeo.Commandline import CommonHelpFormatter, checkArgs, getCommonArgPars
 from changeo.Alignment import RegionDefinition, gapV
 from changeo.Gene import buildGermline
 from changeo.IO import countDbFile, extractIMGT, readGermlines, getFormatOperators, getOutputHandle, \
-                       AIRRWriter, ChangeoWriter, IgBLASTReader, IgBLASTReaderAA, IMGTReader, IHMMuneReader, \
-                       checkFields
+                       gzipOutputName, openFile, AIRRWriter, ChangeoWriter, IgBLASTReader, IgBLASTReaderAA, \
+                       IMGTReader, IHMMuneReader, checkFields
 from changeo.Receptor import ChangeoSchema, AIRRSchema
 
 # 10X Receptor attributes
@@ -121,9 +123,9 @@ def getIDforIMGT(seq_file, imgt_id_len=default_imgt_id_len):
             id_key = rec.description
         else: # truncate and replace characters
             if imgt_id_len == 49: # 28 September 2021 (version 1.8.4)
-                id_key = re.sub('\s|\t', '_', rec.description[:imgt_id_len])
+                id_key = re.sub(r'\s|\t', '_', rec.description[:imgt_id_len])
             else: # older versions
-                id_key = re.sub('\||\s|!|&|\*|<|>|\?', '_', rec.description[:imgt_id_len])
+                id_key = re.sub(r'\||\s|!|&|\*|<|>|\?', '_', rec.description[:imgt_id_len])
         ids.update({id_key: rec.description})
 
     return ids
@@ -233,13 +235,14 @@ def writeDb(records, fields, aligner_file, total_count, id_dict=None, annotation
     # Wrapper for opening handles and writers
     def _open(x, f, writer=writer, out_file=out_file):
         if out_file is not None and x == 'pass':
-            handle = open(out_file, 'w')
+            handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
         else:
             handle = getOutputHandle(aligner_file,
                                      out_label='db-%s' % x,
                                      out_dir=out_args['out_dir'],
                                      out_name=out_args['out_name'],
-                                     out_type=out_args['out_type'])
+                                     out_type=out_args['out_type'],
+                                     gzip_output=out_args.get('gzip_output'))
         return handle, writer(handle, fields=f)
 
     # Function to convert fasta header annotations to changeo columns
@@ -423,7 +426,7 @@ def writeDb(records, fields, aligner_file, total_count, id_dict=None, annotation
 
 def parseIMGT(aligner_file, seq_file=None, repo=None, cellranger_file=None, validate='strict', asis_id=True,
               extended=False, format=default_format, out_file=None, out_args=default_out_args,
-              imgt_id_len=default_imgt_id_len):
+              imgt_id_len=default_imgt_id_len, nproc=None):
     """
     Main for IMGT aligned sample sequences.
 
@@ -438,6 +441,7 @@ def parseIMGT(aligner_file, seq_file=None, repo=None, cellranger_file=None, vali
       out_file (str): output file name. Automatically generated from the input file if None.
       out_args (dict): common output argument dictionary from parseCommonArgs.
       imgt_id_len (int): maximum character length of sequence identifiers reported by IMGT/HighV-QUEST.
+      nproc (int): number of processes to use for parallel processing.
 
     Returns:
       dict: names of the 'pass' and 'fail' output files.
@@ -448,6 +452,7 @@ def parseIMGT(aligner_file, seq_file=None, repo=None, cellranger_file=None, vali
     log['COMMAND'] = 'imgt'
     log['ALIGNER_FILE'] = aligner_file
     log['SEQ_FILE'] = os.path.basename(seq_file) if seq_file else ''
+    log['NPROC'] = nproc
     log['ASIS_ID'] = asis_id
     log['VALIDATE'] = validate
     log['EXTENDED'] = extended
@@ -518,7 +523,7 @@ def parseIMGT(aligner_file, seq_file=None, repo=None, cellranger_file=None, vali
 
 def parseIgBLAST(aligner_file, seq_file, repo, amino_acid=False, cellranger_file=None, validate='strict',
                  asis_id=True, asis_calls=False, extended=False, regions='default', infer_junction=False,
-                 format='changeo', out_file=None, out_args=default_out_args):
+                 format='changeo', out_file=None, out_args=default_out_args, nproc=None):
     """
     Main for IgBLAST aligned sample sequences.
 
@@ -536,6 +541,7 @@ def parseIgBLAST(aligner_file, seq_file, repo, amino_acid=False, cellranger_file
       format (str): output format. one of 'changeo' or 'airr'.
       out_file (str): output file name. Automatically generated from the input file if None.
       out_args (dict): common output argument dictionary from parseCommonArgs.
+      nproc (int): number of processes to use for parallel processing.
 
     Returns:
       dict: names of the 'pass' and 'fail' output files.
@@ -546,6 +552,7 @@ def parseIgBLAST(aligner_file, seq_file, repo, amino_acid=False, cellranger_file
     log['COMMAND'] = 'igblast-aa' if amino_acid else 'igblast'
     log['ALIGNER_FILE'] = os.path.basename(aligner_file)
     log['SEQ_FILE'] = os.path.basename(seq_file)
+    log['NPROC'] = nproc
     log['ASIS_ID'] = asis_id
     log['ASIS_CALLS'] = asis_calls
     log['VALIDATE'] = validate
@@ -600,7 +607,7 @@ def parseIgBLAST(aligner_file, seq_file, repo, amino_acid=False, cellranger_file
         fields.extend(custom)
 
     # Parse and write output
-    with open(aligner_file, 'r') as f:
+    with openFile(aligner_file, 'r') as f:
         parse_iter = parser(f, seq_dict, references, regions=regions, asis_calls=asis_calls, infer_junction=infer_junction)
         germ_iter = (addGermline(x, references, amino_acid=amino_acid) for x in parse_iter)
         output = writeDb(germ_iter, fields=fields, aligner_file=aligner_file, total_count=total_count,
@@ -611,7 +618,7 @@ def parseIgBLAST(aligner_file, seq_file, repo, amino_acid=False, cellranger_file
 
 
 def parseIHMM(aligner_file, seq_file, repo, cellranger_file=None, validate='strict', asis_id=True,
-              extended=False, format=default_format, out_file=None, out_args=default_out_args):
+              extended=False, format=default_format, out_file=None, out_args=default_out_args, nproc=None):
     """
     Main for iHMMuneAlign aligned sample sequences.
 
@@ -625,6 +632,7 @@ def parseIHMM(aligner_file, seq_file, repo, cellranger_file=None, validate='stri
       format (str): output format. One of 'changeo' or 'airr'.
       out_file (str): output file name. Automatically generated from the input file if None.
       out_args (dict): common output argument dictionary from parseCommonArgs.
+      nproc (int): number of processes to use for parallel processing.
 
     Returns:
       dict: names of the 'pass' and 'fail' output files.
@@ -635,6 +643,7 @@ def parseIHMM(aligner_file, seq_file, repo, cellranger_file=None, validate='stri
     log['COMMAND'] = 'ihmm'
     log['ALIGNER_FILE'] = os.path.basename(aligner_file)
     log['SEQ_FILE'] = os.path.basename(seq_file)
+    log['NPROC'] = nproc
     log['ASIS_ID'] = asis_id
     log['VALIDATE'] = validate
     log['EXTENDED'] = extended
@@ -679,7 +688,7 @@ def parseIHMM(aligner_file, seq_file, repo, cellranger_file=None, validate='stri
         fields.extend(custom)
 
     # Parse and write output
-    with open(aligner_file, 'r') as f:
+    with openFile(aligner_file, 'r') as f:
         parse_iter = IHMMuneReader(f, seq_dict, references)
         germ_iter = (addGermline(x, references) for x in parse_iter)
         output = writeDb(germ_iter, fields=fields, aligner_file=aligner_file, total_count=total_count,
@@ -691,7 +700,7 @@ def parseIHMM(aligner_file, seq_file, repo, cellranger_file=None, validate='stri
 
 
 def numberAIRR(aligner_file, repo=None, format=default_format,
-               out_file=None, out_args=default_out_args):
+               out_file=None, out_args=default_out_args, nproc=None):
     """
     Inserts IMGT numbering into V fields
 
@@ -701,6 +710,7 @@ def numberAIRR(aligner_file, repo=None, format=default_format,
       format (str): output format.
       out_file (str): output file name. Automatically generated from the input file if None.
       out_args (dict): common output argument dictionary from parseCommonArgs.
+      nproc (int): number of processes to use for parallel processing.
 
     Returns:
       str: output file name.
@@ -709,6 +719,7 @@ def numberAIRR(aligner_file, repo=None, format=default_format,
     log['START'] = 'MakeDb'
     log['COMMAND'] = 'number'
     log['ALIGNER_FILE'] = os.path.basename(aligner_file)
+    log['NPROC'] = nproc
     printLog(log)
 
     # Define format operators
@@ -718,7 +729,7 @@ def numberAIRR(aligner_file, repo=None, format=default_format,
         printError('Invalid format %s.' % format)
 
     # Open input
-    db_handle = open(aligner_file, 'rt')
+    db_handle = openFile(aligner_file, 'rt')
     db_iter = reader(db_handle)
 
     # Define log handle
@@ -743,15 +754,17 @@ def numberAIRR(aligner_file, repo=None, format=default_format,
 
     # Open output writer
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(aligner_file, out_label='db-pass', out_dir=out_args['out_dir'],
-                                        out_name=out_args['out_name'], out_type=schema.out_type)
+                                        out_name=out_args['out_name'], out_type=schema.out_type,
+                                        gzip_output=out_args.get('gzip_output'))
     pass_writer = writer(pass_handle, fields=db_iter.fields)
 
     if out_args['failed']:
         fail_handle = getOutputHandle(aligner_file, out_label='db-fail', out_dir=out_args['out_dir'],
-                                        out_name=out_args['out_name'], out_type=schema.out_type)
+                                        out_name=out_args['out_name'], out_type=schema.out_type,
+                                        gzip_output=out_args.get('gzip_output'))
         fail_writer = writer(fail_handle, fields=db_iter.fields)
 
     # Count records
@@ -859,7 +872,7 @@ def getArgParser():
     subparsers.required = True
 
     # Parent parser
-    parser_parent = getCommonArgParser(db_in=False)
+    parser_parent = getCommonArgParser(db_in=False, multiproc=True)
 
     # igblastn output parser
     parser_igblast = subparsers.add_parser('igblast', parents=[parser_parent],
@@ -952,6 +965,10 @@ def getArgParser():
     group_igblast_aa.add_argument('--regions', action='store', dest='regions',
                                   choices=('default', 'rhesus-igl'), default='default',
                                   help='''IMGT CDR and FWR boundary definition to use.''')
+    # Add mutually exclusive group for validation, including --partial
+    group_igblast_aa_validate = group_igblast_aa.add_mutually_exclusive_group(required=False)
+    group_igblast_aa_validate.add_argument('--partial', action='store_const', const='partial', dest='validate',
+        help='''If specified, include incomplete V(D)J alignments in the pass file instead of the fail file. An incomplete alignment is defined as a record that is missing a V gene assignment, J gene assignment, junction region, or productivity call.''')
     parser_igblast_aa.set_defaults(func=parseIgBLAST, amino_acid=True, validate='strict')
 
 
@@ -1090,6 +1107,10 @@ if __name__ == "__main__":
     if 'seq_files' in args_dict and not args_dict['seq_files']:
         args_dict['asis_id'] = True
 
+    # Extract nproc before deleting, use parser default if not specified
+    nproc_default = parser.get_default('nproc')
+    nproc = args_dict.get('nproc', nproc_default)
+
     # Delete
     if 'aligner_files' in args_dict: del args_dict['aligner_files']
     if 'seq_files' in args_dict: del args_dict['seq_files']
@@ -1097,16 +1118,30 @@ if __name__ == "__main__":
     if 'out_files' in args_dict: del args_dict['out_files']
     if 'command' in args_dict: del args_dict['command']
     if 'func' in args_dict: del args_dict['func']
+    if 'nproc' in args_dict: del args_dict['nproc']
+
+
+    ds=[]
 
-    # Call main
+    # initialize a dict for each execution, and add to list
     for i, f in enumerate(args.__dict__['aligner_files']):
-        args_dict['aligner_file'] = f
-        args_dict['out_file'] = args.__dict__['out_files'][i] \
+        tmp_dict=args_dict.copy()
+        tmp_dict['aligner_file'] = f
+        tmp_dict['nproc'] = nproc
+        tmp_dict['out_file'] = args.__dict__['out_files'][i] \
                                 if args.__dict__['out_files'] else None
         if 'seq_files' in args.__dict__:
-            args_dict['seq_file'] = args.__dict__['seq_files'][i] \
+            tmp_dict['seq_file'] = args.__dict__['seq_files'][i] \
                                     if args.__dict__['seq_files'] else None
         if 'cellranger_files' in args.__dict__:
-            args_dict['cellranger_file'] = args.__dict__['cellranger_files'][i] \
+            tmp_dict['cellranger_file'] = args.__dict__['cellranger_files'][i] \
                                            if args.__dict__['cellranger_files'] else None
-        args.func(**args_dict)
+        ds.append(tmp_dict)
+
+    # now do the loop in parallel
+    with ProcessPoolExecutor(max_workers=nproc) as executor:
+        futures = [executor.submit(args.func, **d) for d in ds]
+        for future in as_completed(futures):
+            result = future.result()
+        # original code threw away return values, so I am too...
+                       


=====================================
bin/ParseDb.py
=====================================
@@ -18,10 +18,11 @@ from textwrap import dedent
 from time import time
 
 # Presto and changeo imports
-from presto.IO import printLog, printProgress, printMessage
+from presto.IO import printLog, printProgress, printMessage, printWarning
 from changeo.Defaults import default_csv_size, default_out_args
 from changeo.Commandline import CommonHelpFormatter, checkArgs, getCommonArgParser, parseCommonArgs
-from changeo.IO import checkFields, countDbFile, getOutputHandle, splitName, TSVReader, TSVWriter
+from changeo.IO import checkFields, countDbFile, getOutputHandle, gzipOutputName, openFile, \
+                       splitName, TSVReader, TSVWriter
 
 # System settings
 csv.field_size_limit(default_csv_size)
@@ -54,7 +55,7 @@ def splitDbFile(db_file, field, num_split=None, out_args=default_out_args):
     printLog(log)
 
     # Open input
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = TSVReader(db_handle)
     out_fields = db_iter.fields
     __, __, out_args['out_type'] = splitName(db_file)
@@ -73,13 +74,13 @@ def splitDbFile(db_file, field, num_split=None, out_args=default_out_args):
     # Sort records into files based on textual field
     if num_split is None:
         # Create set of unique field tags
-        with open(db_file, 'rt') as tmp_handle:
+        with openFile(db_file, 'rt') as tmp_handle:
             tmp_iter = TSVReader(tmp_handle)
             tag_list = list(set([row[field] for row in tmp_iter]))
 
         # Forbidden characters in filename and replacements
-        no_good = {'\/':'f','\\':'b','?':'q','\%':'p','*':'s',':':'c',
-                   '\|':'pi','\"':'dq','\'':'sq','<':'gt','>':'lt',' ':'_'}
+        no_good = {r'\/':'f',r'\\':'b',r'?':'q',r'\%':'p',r'*':'s',r':':'c',
+                   r'\|':'pi',r'\"':'dq',r'\'':'sq',r'<':'gt',r'>':'lt',r' ':'_'}
         # Replace forbidden characters in tag_list
         tag_dict = {}
         for tag in tag_list:
@@ -92,7 +93,8 @@ def splitDbFile(db_file, field, num_split=None, out_args=default_out_args):
                                              out_label='%s-%s' % (field, label),
                                              out_name=out_args['out_name'],
                                              out_dir=out_args['out_dir'],
-                                             out_type=out_args['out_type'])
+                                             out_type=out_args['out_type'],
+                                             gzip_output=out_args.get('gzip_output'))
                         for tag, label in tag_dict.items()}
 
         # Create Db writer instances
@@ -116,12 +118,14 @@ def splitDbFile(db_file, field, num_split=None, out_args=default_out_args):
                                                  out_label='under-%.1f' % num_split,
                                                  out_name=out_args['out_name'],
                                                  out_dir=out_args['out_dir'],
-                                                 out_type=out_args['out_type']),
+                                                 out_type=out_args['out_type'],
+                                                 gzip_output=out_args.get('gzip_output')),
                         'atleast': getOutputHandle(db_file,
                                                    out_label='atleast-%.1f' % num_split,
                                                    out_name=out_args['out_name'],
                                                    out_dir=out_args['out_dir'],
-                                                   out_type=out_args['out_type'])}
+                                                   out_type=out_args['out_type'],
+                                                   gzip_output=out_args.get('gzip_output'))}
 
         # Create Db writer instances
         writers_dict = {'under': TSVWriter(handles_dict['under'], fields=out_fields),
@@ -178,7 +182,7 @@ def addDbFile(db_file, fields, values, out_file=None, out_args=default_out_args)
     printLog(log)
 
     # Open input
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = TSVReader(db_handle)
     __, __, out_args['out_type'] = splitName(db_file)
 
@@ -188,10 +192,11 @@ def addDbFile(db_file, fields, values, out_file=None, out_args=default_out_args)
 
     # Open output
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(db_file, out_label='parse-add', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type=out_args['out_type'])
+                                      out_name=out_args['out_name'], out_type=out_args['out_type'],
+                                      gzip_output=out_args.get('gzip_output'))
     pass_writer = TSVWriter(pass_handle, out_fields)
 
     # Count records
@@ -247,7 +252,7 @@ def indexDbFile(db_file, field=default_index_field, out_file=None, out_args=defa
     printLog(log)
 
     # Open input
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = TSVReader(db_handle)
     __, __, out_args['out_type'] = splitName(db_file)
 
@@ -257,10 +262,11 @@ def indexDbFile(db_file, field=default_index_field, out_file=None, out_args=defa
 
     # Open output
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(db_file, out_label='parse-index', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type=out_args['out_type'])
+                                      out_name=out_args['out_name'], out_type=out_args['out_type'],
+                                      gzip_output=out_args.get('gzip_output'))
     pass_writer = TSVWriter(pass_handle, out_fields)
 
     # Count records
@@ -314,7 +320,7 @@ def dropDbFile(db_file, fields, out_file=None, out_args=default_out_args):
     printLog(log)
 
     # Open input
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = TSVReader(db_handle)
     __, __, out_args['out_type'] = splitName(db_file)
 
@@ -329,10 +335,11 @@ def dropDbFile(db_file, fields, out_file=None, out_args=default_out_args):
 
     # Open output
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(db_file, out_label='parse-drop', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type=out_args['out_type'])
+                                      out_name=out_args['out_name'], out_type=out_args['out_type'],
+                                      gzip_output=out_args.get('gzip_output'))
     pass_writer = TSVWriter(pass_handle, out_fields)
 
     # Count records
@@ -400,7 +407,7 @@ def deleteDbFile(db_file, fields, values, logic='any', regex=False,
     printLog(log)
 
     # Open input
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = TSVReader(db_handle)
     out_fields = db_iter.fields
     __, __, out_args['out_type'] = splitName(db_file)
@@ -413,10 +420,11 @@ def deleteDbFile(db_file, fields, values, logic='any', regex=False,
 
     # Open output
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(db_file, out_label='parse-delete', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type=out_args['out_type'])
+                                      out_name=out_args['out_name'], out_type=out_args['out_type'],
+                                      gzip_output=out_args.get('gzip_output'))
     pass_writer = TSVWriter(pass_handle, out_fields)
 
     # Count records
@@ -479,7 +487,7 @@ def renameDbFile(db_file, fields, names, out_file=None, out_args=default_out_arg
     printLog(log)
 
     # Open file handles
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = TSVReader(db_handle)
     __, __, out_args['out_type'] = splitName(db_file)
 
@@ -497,10 +505,11 @@ def renameDbFile(db_file, fields, names, out_file=None, out_args=default_out_arg
 
     # Open writer
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(db_file, out_label='parse-rename', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type=out_args['out_type'])
+                                      out_name=out_args['out_name'], out_type=out_args['out_type'],
+                                      gzip_output=out_args.get('gzip_output'))
     pass_writer = TSVWriter(pass_handle, out_fields)
 
     # Count records
@@ -575,7 +584,7 @@ def selectDbFile(db_file, fields, values, logic='any', regex=False,
     printLog(log)
 
     # Open input
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = TSVReader(db_handle)
     out_fields = db_iter.fields
     __, __, out_args['out_type'] = splitName(db_file)
@@ -588,10 +597,11 @@ def selectDbFile(db_file, fields, values, logic='any', regex=False,
 
     # Open output
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(db_file, out_label='parse-select', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type=out_args['out_type'])
+                                      out_name=out_args['out_name'], out_type=out_args['out_type'],
+                                      gzip_output=out_args.get('gzip_output'))
     pass_writer = TSVWriter(pass_handle, out_fields)
 
     # Count records
@@ -659,7 +669,7 @@ def sortDbFile(db_file, field, numeric=False, descend=False,
     printLog(log)
 
     # Open input
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = TSVReader(db_handle)
     out_fields = db_iter.fields
     __, __, out_args['out_type'] = splitName(db_file)
@@ -672,10 +682,11 @@ def sortDbFile(db_file, field, numeric=False, descend=False,
 
     # Open output
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(db_file, out_label='parse-sort', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type=out_args['out_type'])
+                                      out_name=out_args['out_name'], out_type=out_args['out_type'],
+                                      gzip_output=out_args.get('gzip_output'))
     pass_writer = TSVWriter(pass_handle, out_fields)
 
     # Store all records in a dictionary
@@ -741,7 +752,7 @@ def updateDbFile(db_file, field, values, updates, out_file=None, out_args=defaul
     printLog(log)
 
     # Open input
-    db_handle = open(db_file, 'rt')
+    db_handle = openFile(db_file, 'rt')
     db_iter = TSVReader(db_handle)
     out_fields = db_iter.fields
     __, __, out_args['out_type'] = splitName(db_file)
@@ -754,10 +765,11 @@ def updateDbFile(db_file, field, values, updates, out_file=None, out_args=defaul
 
     # Open output
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         pass_handle = getOutputHandle(db_file, out_label='parse-update', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type=out_args['out_type'])
+                                      out_name=out_args['out_name'], out_type=out_args['out_type'],
+                                      gzip_output=out_args.get('gzip_output'))
     pass_writer = TSVWriter(pass_handle, out_fields)
 
     # Count records
@@ -796,28 +808,40 @@ def updateDbFile(db_file, field, values, updates, out_file=None, out_args=defaul
     return pass_handle.name
 
 
-def mergeDbFiles(db_files, drop=False, out_file=None, out_args=default_out_args):
+def mergeDbFiles(db_files, drop=False, field=None, values=None,
+                  out_file=None, out_args=default_out_args):
     """
     Updates field and value pairs to a database file
 
     Arguments:
       db_files : list of database file names.
       drop : if True drop columns not present in all files.
+      field : name of the field to add annotating the source file of each record;
+              if None do not add a sample identifier field.
+      values : list of sample identifiers, one per file in db_files, assigned to
+               field for each record; ignored if field is None.
       out_file : output file name. Automatically generated from the input file if None.
       out_args : common output argument dictionary from parseCommonArgs.
 
     Returns:
       str : output file name.
     """
+    if field is None and values is not None:
+        printWarning('Values (-u) were specified without a field (-f) to assign '
+                     'them to and will be ignored.')
+
     log = OrderedDict()
     log['START'] = 'ParseDb'
     log['COMMAND'] = 'merge'
     log['FILES'] = ','.join([os.path.basename(f) for f in db_files])
     log['DROP'] = drop
+    if field is not None:
+        log['FIELD'] = field
+        log['VALUES'] = ','.join(values)
     printLog(log)
 
     # Open input
-    db_handles = [open(f, 'rt') for f in db_files]
+    db_handles = [openFile(f, 'rt') for f in db_files]
     db_iters = [TSVReader(x) for x in db_handles]
     result_count = sum([countDbFile(f) for f in db_files])
 
@@ -829,25 +853,39 @@ def mergeDbFiles(db_files, drop=False, out_file=None, out_args=default_out_args)
         field_set = set.union(*map(set, field_list))
     field_order = OrderedDict([(f, None) for f in chain(*field_list)])
     out_fields = [f for f in field_order if f in field_set]
+    if field is not None:
+        # Check against all input headers (field_order), not just the post-drop
+        # output header (out_fields), so a field present in only some input files
+        # still triggers the overwrite warning when --drop removes it from out_fields.
+        if field in field_order:
+            printWarning('Field %s already exists in one or more input files and its '
+                         'existing values will be overwritten.' % field)
+        if field not in out_fields:
+            out_fields.append(field)
 
     # Open output file
     if out_file is not None:
-        pass_handle = open(out_file, 'w')
+        pass_handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
     else:
         __, __, out_args['out_type'] = splitName(db_files[0])
         pass_handle = getOutputHandle(db_files[0], out_label='parse-merge', out_dir=out_args['out_dir'],
-                                      out_name=out_args['out_name'], out_type=out_args['out_type'])
+                                      out_name=out_args['out_name'], out_type=out_args['out_type'],
+                                      gzip_output=out_args.get('gzip_output'))
     pass_writer = TSVWriter(pass_handle, out_fields)
 
     # Iterate over records
     start_time = time()
     rec_count = 0
-    for db in db_iters:
+    for i, db in enumerate(db_iters):
         for rec in db:
             # Print progress for previous iteration
             printProgress(rec_count, result_count, 0.05, start_time=start_time)
             rec_count += 1
 
+            # Annotate sample identifier
+            if field is not None:
+                rec[field] = values[i]
+
             # Write records
             pass_writer.writeDict(rec)
 
@@ -1036,6 +1074,14 @@ def getArgParser():
                               help='''If specified, drop fields that do not exist in all input files.
                                    Otherwise, include all columns in all files and fill missing data
                                    with empty strings.''')
+    group_merge.add_argument('-f', action='store', dest='field', default=None,
+                              help='''Name of the field to add to the output file annotating each
+                                   record with a sample identifier. If not specified, no sample
+                                   identifier field is added.''')
+    group_merge.add_argument('-u', nargs='+', action='store', dest='values', default=None,
+                              help='''Sample identifiers to assign to the field specified by -f, one
+                                   per input file (-d) and in the same order. Ignored if -f is not
+                                   specified.''')
     parser_merge.set_defaults(func=mergeDbFiles)
 
     # Subparser to partition files by annotation values
@@ -1078,6 +1124,11 @@ if __name__ == '__main__':
         parser.error('You must specify exactly one new name (-k) per field (-f)')
     elif args.command == 'update' and len(args_dict['values']) != len(args_dict['updates']):
         parser.error('You must specify exactly one value (-u) per replacement (-t)')
+    elif args.command == 'merge' and args_dict['field'] is not None:
+        if args_dict['values'] is None:
+            parser.error('You must specify sample identifiers (-u) when using -f')
+        elif len(args_dict['values']) != len(args_dict['db_files']):
+            parser.error('You must specify exactly one sample identifier (-u) per input file (-d)')
 
     # Call parser function for each database file
     if args.command == 'merge':


=====================================
changeo.egg-info/PKG-INFO
=====================================
@@ -1,9 +1,9 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: changeo
-Version: 1.3.0
+Version: 1.3.5
 Summary: A bioinformatics toolkit for processing high-throughput lymphocyte receptor sequencing data.
 Home-page: http://changeo.readthedocs.io
-Download-URL: https://bitbucket.org/kleinstein/changeo/downloads
+Download-URL: https://github.com/immcantation/changeo/tags
 Author: Namita Gupta, Jason Anthony Vander Heiden
 Author-email: immcantation at googlegroups.com
 License: GNU Affero General Public License 3 (AGPL-3)
@@ -13,9 +13,30 @@ Classifier: Environment :: Console
 Classifier: Intended Audience :: Science/Research
 Classifier: Natural Language :: English
 Classifier: Operating System :: OS Independent
-Classifier: Programming Language :: Python :: 3.4
+Classifier: Programming Language :: Python :: 3.10
 Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
 License-File: LICENSE
+Requires-Dist: numpy>=1.23.2
+Requires-Dist: scipy>=1.9.3
+Requires-Dist: pandas>=1.5.0
+Requires-Dist: biopython>=1.81
+Requires-Dist: PyYAML>=6.0
+Requires-Dist: setuptools>=65.5.0
+Requires-Dist: presto>=0.7.1
+Requires-Dist: airr>=1.3.1
+Requires-Dist: packaging>=21.3
+Requires-Dist: importlib-resources>=6.4.0
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: download-url
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: summary
 
 .. image:: https://img.shields.io/pypi/dm/changeo
     :target: https://pypi.org/project/changeo
@@ -28,7 +49,7 @@ Change-O - Repertoire clonal assignment toolkit
 Change-O is a collection of tools for processing the output of V(D)J alignment
 tools, assigning clonal clusters to immunoglobulin (Ig) sequences, and
 reconstructing germline sequences.
- 
+
 Dramatic improvements in high-throughput sequencing technologies now enable 
 large-scale characterization of Ig repertoires, defined as the collection of
 trans-membrane antigen-receptor proteins located on the surface of B cells and
@@ -37,3 +58,6 @@ Ig and TCR sequences following germline segment assignment. Change-O
 handles output from IMGT/HighV-QUEST and IgBLAST, and provides a wide variety of
 clustering methods for assigning clonal groups to Ig sequences. Record sorting, 
 grouping, and various database manipulation operations are also included.
+
+**IMPORTANT!** 
+Active development of Change-O has moved to https://github.com/immcantation/changeo


=====================================
changeo.egg-info/SOURCES.txt
=====================================
@@ -3,6 +3,7 @@ LICENSE
 MANIFEST.in
 NEWS.rst
 README.rst
+pyproject.toml
 requirements.txt
 setup.py
 bin/AlignRecords.py
@@ -35,4 +36,9 @@ changeo/data/hh_s5f_dist.tsv
 changeo/data/hs1f_compat_dist.tsv
 changeo/data/m1n_compat_dist.tsv
 changeo/data/mk_rs1nf_dist.tsv
-changeo/data/mk_rs5nf_dist.tsv
\ No newline at end of file
+changeo/data/mk_rs5nf_dist.tsv
+tests/test_DefineClones.py
+tests/test_Distance.py
+tests/test_IO.py
+tests/test_ParseDb.py
+tests/test_Receptor.py
\ No newline at end of file


=====================================
changeo.egg-info/requires.txt
=====================================
@@ -1,8 +1,10 @@
-numpy>=1.8
-scipy>=0.14
-pandas>=0.24
-biopython>=1.77
-PyYAML>=5.1
-setuptools>=2.0
-presto>=0.7.0
+numpy>=1.23.2
+scipy>=1.9.3
+pandas>=1.5.0
+biopython>=1.81
+PyYAML>=6.0
+setuptools>=65.5.0
+presto>=0.7.1
 airr>=1.3.1
+packaging>=21.3
+importlib-resources>=6.4.0


=====================================
changeo/Applications.py
=====================================
@@ -8,7 +8,7 @@ __author__ = 'Jason Anthony Vander Heiden'
 # Imports
 import os
 import re
-from pkg_resources import parse_version
+from packaging.version import parse as parse_version
 from subprocess import check_output, STDOUT, CalledProcessError
 
 # Presto and changeo imports
@@ -280,7 +280,7 @@ def getIgBLASTVersion(exec=default_igblastn_exec):
         printError('Running command: %s\n%s' % (' '.join(cmd), e.output))
 
     # Extract version number
-    match = re.search('(?<=Package: igblast )(\d+\.\d+\.\d+)', stdout_str)
+    match = re.search(r'(?<=Package: igblast )(\d+\.\d+\.\d+)', stdout_str)
     version = match.group(0)
 
     return version
\ No newline at end of file


=====================================
changeo/Commandline.py
=====================================
@@ -16,6 +16,7 @@ from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter, \
 # Changeo imports
 from presto.IO import printWarning, printError
 from changeo.Defaults import choices_format, default_format
+from changeo.IO import gzipOutputName
 from changeo.Receptor import AIRRSchema, ChangeoSchema
 
 
@@ -45,7 +46,7 @@ class CommonHelpFormatter(RawDescriptionHelpFormatter, ArgumentDefaultsHelpForma
 
 
 def getCommonArgParser(db_in=True, db_out=True, out_file=True, failed=True, log=True,
-                       format=True, multiproc=False, add_help=True):
+                       format=True, multiproc=False, add_help=True, gzip_output=True):
     """
     Defines an ArgumentParser object with common pRESTO arguments
 
@@ -57,6 +58,11 @@ def getCommonArgParser(db_in=True, db_out=True, out_file=True, failed=True, log=
       log (bool): if True include log arguments.
       format (bool): input and output type arguments.
       multiproc (bool): if True include multiprocessing arguments.
+      gzip_output (bool): if True include the --gzip-output argument. Should be
+                          False for tools whose output is not produced through
+                          getOutputHandle/openFile (e.g. wrapped external
+                          executables), since the flag would otherwise be
+                          silently ignored.
 
     Returns:
       argparse.ArgumentParser : an argument parser.
@@ -96,6 +102,11 @@ def getCommonArgParser(db_in=True, db_out=True, out_file=True, failed=True, log=
                        help='''Changes the prefix of the successfully processed output file
                             to the string specified. May not be specified with multiple
                             input files.''')
+    if gzip_output:
+        group.add_argument('--gzip-output', action='store_true', dest='gzip_output', default=None,
+                           help='''Specify to force gzip compressed output files. Output is
+                                automatically gzip compressed when the input file(s) are
+                                themselves gzip compressed (.gz), even without this argument.''')
 
     # Log arguments
     if log:
@@ -119,11 +130,35 @@ def getCommonArgParser(db_in=True, db_out=True, out_file=True, failed=True, log=
     if multiproc:
         group.add_argument('--nproc', action='store', dest='nproc', type=int, default=mp.cpu_count(),
                            help='''The number of simultaneous computational processes to execute
-                                (CPU cores to utilized).''')
+                                (CPU cores to utilize).''')
 
     return parser
 
 
+def _samePath(path_a, path_b):
+    """
+    Determines whether two paths refer to the same file
+
+    Arguments:
+      path_a (str): first file path.
+      path_b (str): second file path.
+
+    Returns:
+      bool: True if the paths resolve to the same file. Uses os.path.samefile
+           (which follows symlinks and normalizes case/aliasing per the
+           filesystem) when both paths exist; otherwise falls back to
+           comparing normalized absolute paths, since one of the paths
+           (typically the output) may not exist yet.
+    """
+    if os.path.exists(path_a) and os.path.exists(path_b):
+        try:
+            return os.path.samefile(path_a, path_b)
+        except OSError:
+            pass
+
+    return os.path.normpath(os.path.abspath(path_a)) == os.path.normpath(os.path.abspath(path_b))
+
+
 def parseCommonArgs(args, in_arg=None, in_types=None, in_list=False):
     """
     Checks common arguments from getCommonArgParser and transforms output options to a dictionary
@@ -172,8 +207,10 @@ def parseCommonArgs(args, in_arg=None, in_types=None, in_list=False):
         for f in args_dict['db_files']:
             if not os.path.isfile(f):
                 printError('Database file %s does not exist.' % f)
-            if os.path.splitext(f)[-1].lower() not in db_types:
-                printError('Database file %s is not a supported type. Must be one: %s.' \
+            # Strip a trailing .gz extension before checking the underlying file type
+            f_check = f[:-3] if f.lower().endswith('.gz') else f
+            if os.path.splitext(f_check)[-1].lower() not in db_types:
+                printError('Database file %s is not a supported type. Must be one: %s (optionally gzip compressed with a .gz extension).' \
                            % (f, ', '.join(db_types)))
 
     # Verify non-standard input files
@@ -198,17 +235,22 @@ def parseCommonArgs(args, in_arg=None, in_types=None, in_list=False):
     if args_dict.get('out_files', None) is not None:
         if len(args_dict['out_files']) != input_count:
             printError('The -o argument requires one output file name per input file.')
-        for f in args_dict['out_files']:
-            if f in input_files:
+        # Resolve the .gz suffix that gzipOutputName will add at write time, so the
+        # collision and overwrite checks below see the file name that will actually be opened.
+        resolved_out_files = [gzipOutputName(f, args_dict.get('gzip_output', None)) \
+                              for f in args_dict['out_files']]
+        for f in resolved_out_files:
+            if any(_samePath(f, i) for i in input_files):
                 printError('Output files and input files cannot have the same names.')
-        for f in args_dict['out_files']:
+        for f in resolved_out_files:
             if os.path.isfile(f):
                 printWarning('Output file %s already exists and will be overwritten.' % f)
     if args_dict.get('out_file', None) is not None:
-        if args_dict['out_file'] in input_files:
+        resolved_out_file = gzipOutputName(args_dict['out_file'], args_dict.get('gzip_output', None))
+        if any(_samePath(resolved_out_file, i) for i in input_files):
             printError('Output files and input files cannot have the same names.')
-        if os.path.isfile(args_dict['out_file']):
-            printWarning('Output file %s already exists and will be overwritten.' % args_dict['out_file'])
+        if os.path.isfile(resolved_out_file):
+            printWarning('Output file %s already exists and will be overwritten.' % resolved_out_file)
 
     # Exit if output names or log files are specified with multiple input files
     if args_dict.get('out_name', None) is not None \
@@ -224,7 +266,7 @@ def parseCommonArgs(args, in_arg=None, in_types=None, in_list=False):
             printError('Path %s exists but it is not a directory.' % args_dict['out_dir'])
 
     # Redefine common output options as out_args dictionary
-    out_args = ['log_file', 'out_dir', 'out_name', 'out_type', 'failed']
+    out_args = ['log_file', 'out_dir', 'out_name', 'out_type', 'failed', 'gzip_output']
     args_dict['out_args'] = {k:args_dict.setdefault(k, None) for k in out_args}
     for k in out_args: del args_dict[k]
     


=====================================
changeo/Distance.py
=====================================
@@ -9,7 +9,7 @@ __author__ = 'Jason Anthony Vander Heiden, Namita Gupta'
 import numpy as np
 import pandas as pd
 from itertools import combinations, product, zip_longest
-from pkg_resources import resource_stream
+from importlib_resources import open_text
 from scipy.cluster.hierarchy import fcluster, linkage
 from scipy.spatial.distance import squareform
 
@@ -220,22 +220,22 @@ aa_model = getAADistMatrix(mask_dist=0, gap_dist=0)
 ham_model = getDNADistMatrix(mask_dist=0, gap_dist=0)
 
 # Load model data
-with resource_stream(__name__, 'data/hh_s1f_dist.tsv') as f:
+with open_text('changeo', 'data/hh_s1f_dist.tsv') as f:
     hh_s1f_model = pd.read_csv(f, sep='\t', index_col=0)
 
-with resource_stream(__name__, 'data/hh_s5f_dist.tsv') as f:
+with open_text('changeo', 'data/hh_s5f_dist.tsv') as f:
     hh_s5f_model = pd.read_csv(f, sep='\t', index_col=0)
 
-with resource_stream(__name__, 'data/mk_rs1nf_dist.tsv') as f:
+with open_text('changeo', 'data/mk_rs1nf_dist.tsv') as f:
     mk_rs1nf_model = pd.read_csv(f, sep='\t', index_col=0)
 
-with resource_stream(__name__, 'data/mk_rs5nf_dist.tsv') as f:
+with open_text('changeo', 'data/mk_rs5nf_dist.tsv') as f:
     mk_rs5nf_model = pd.read_csv(f, sep='\t', index_col=0)
 
-with resource_stream(__name__, 'data/m1n_compat_dist.tsv') as f:
+with open_text('changeo', 'data/m1n_compat_dist.tsv') as f:
     m1n_compat_model = pd.read_csv(f, sep='\t', index_col=0)
 
-with resource_stream(__name__, 'data/hs1f_compat_dist.tsv') as f:
+with open_text('changeo', 'data/hs1f_compat_dist.tsv') as f:
     hs1f_compat_model = pd.read_csv(f, sep='\t', index_col=0)
 
 distance_models = {'ham': ham_model,


=====================================
changeo/Gene.py
=====================================
@@ -18,6 +18,7 @@ allele_regex = re.compile(r'((IG[HLK]|TR[ABGD])([VDJ][A-R0-9]+[-/\w]*[-\*][\.\w]
 gene_regex = re.compile(r'((IG[HLK]|TR[ABGD])([VDJ][A-R0-9]+[-/\w]*))')
 family_regex = re.compile(r'((IG[HLK]|TR[ABGD])([VDJ][A-R0-9]+))')
 locus_regex = re.compile(r'(IG[HLK]|TR[ABGD])')
+multi_locus_tra_trd = re.compile(r'TRAV\d+(?:-\d+)?/DV\d+')
 
 v_allele_regex = re.compile(r'((IG[HLK]|TR[ABGD])V[A-R0-9]+[-/\w]*[-\*][\.\w]+)')
 d_allele_regex = re.compile(r'((IG[HLK]|TR[ABGD])D[A-R0-9]+[-/\w]*[-\*][\.\w]+)')
@@ -115,6 +116,16 @@ def getLocus(gene, action='first'):
       str: String of the first locus call when action is 'first'.
       tuple: Tuple of locus calls for 'set' or 'list' actions.
     """
+    # Handle None input
+    if gene is None:
+        return None
+    
+    if multi_locus_tra_trd.search(gene):
+        dual = ['TRA', 'TRD']
+        if action == 'first':
+            return 'TRA/TRD'
+        elif action in ('set', 'list'):
+            return tuple(dual)   
     return parseGeneCall(gene, locus_regex, action=action)
 
 


=====================================
changeo/IO.py
=====================================
@@ -6,6 +6,7 @@ __author__ = 'Namita Gupta, Jason Anthony Vander Heiden'
 
 # Imports
 import csv
+import gzip
 import os
 import re
 import tarfile
@@ -29,6 +30,40 @@ from changeo.Alignment import decodeBTOP, encodeCIGAR, padAlignment, gapV, infer
 csv.field_size_limit(default_csv_size)
 
 
+def openFile(filename, mode='r'):
+    """
+    Opens a file with automatic gzip support based on the .gz extension
+
+    Arguments:
+      filename (str): path to the file.
+      mode (str): file opening mode ('r' or 'w'; a trailing 't'/'b' is ignored).
+
+    Returns:
+      file : file handle opened in text mode.
+    """
+    if filename.lower().endswith('.gz'):
+        return gzip.open(filename, mode[0] + 't')
+    else:
+        return open(filename, mode)
+
+
+def gzipOutputName(file, gzip_output):
+    """
+    Appends a .gz extension to a file name if compressed output is requested
+
+    Arguments:
+      file (str): file name.
+      gzip_output (bool): if True force a .gz extension onto file.
+
+    Returns:
+      str: file name, with a .gz extension added if needed.
+    """
+    if gzip_output and not file.lower().endswith('.gz'):
+        file += '.gz'
+
+    return file
+
+
 class TSVReader:
     """
     Simple csv.DictReader wrapper to read format agnostic TSV files.
@@ -526,8 +561,8 @@ class IMGTReader:
         Returns:
           dict : database entries for gene calls.
         """
-        clean_regex = re.compile('(,)|(\(see comment\))')
-        delim_regex = re.compile('\sor\s')
+        clean_regex = re.compile(r'(,)|(\(see comment\))')
+        delim_regex = re.compile(r'\sor\s')
 
         # Gene calls
         v_str = summary['V-GENE and allele']
@@ -953,7 +988,7 @@ class IgBLASTReader:
 
         # Extract column names from comments
         f = next((x for x in chunk if x.startswith('# V-(D)-J rearrangement summary')))
-        f = re.search('summary for query sequence \((.+)\)\.', f).group(1)
+        f = re.search(r'summary for query sequence \((.+)\)\.', f).group(1)
         columns = [summary_map[x.strip()] for x in f.split(',')]
 
         # Extract first row as a list
@@ -987,7 +1022,7 @@ class IgBLASTReader:
 
         # Extract column names from comments
         f = next((x for x in chunk if x.startswith('# Sub-region sequence details')))
-        f = re.search('sequence details \((.+)\)', f).group(1)
+        f = re.search(r'sequence details \((.+)\)', f).group(1)
         columns = [cdr3_map[x.strip()] for x in f.split(',')]
 
         # Extract first CDR3 as a list and remove the CDR3 label
@@ -1608,13 +1643,37 @@ class IgBLASTReaderAA(IgBLASTReader):
         result = {}
         seq_vdj = db['sequence_aa_vdj']
         seq_trim = db['sequence_aa_trim']
-        v_hit = next(x for x in hits if x['segment'] == 'V')
-
+        
+        # Get all V hits
+        v_hits = [x for x in hits if x['segment'] == 'V']
+        
+        # Find the top score
+        top_score = max(float(hit['bit score']) for hit in v_hits)
+        
+        # Get all hits with the top score
+        top_v_hits = [hit for hit in v_hits if float(hit['bit score']) == top_score]
+        
+        # Sort top-scoring hits by subject id and 
+        # use the first hit for positional information and sequence building
+        top_v_hits = sorted(top_v_hits, key=lambda hit: hit['subject id'])
+        v_hit = top_v_hits[0]
+        
         # Alignment positions
         result.update(self._parseVHitPos(v_hit))
 
-        # Assign V gene and update VDJ sequence with and without removing insertions
-        result['v_call'] = v_hit['subject id']
+        # Assign V gene calls from all top-scoring hits (comma-separated) and update VDJ sequence with and without removing insertions
+        if not self.asis_calls:
+            # Parse gene calls and join multiple alleles with commas (consistent with nucleotide parser)
+            v_calls = []
+            for hit in top_v_hits:
+                v_call = getVAllele(hit['subject id'], action='list')
+                if v_call:
+                    v_calls.extend(v_call)
+            result['v_call'] = ','.join(set(v_calls)) if v_calls else None
+        else:
+            # Join raw subject IDs with commas
+            result['v_call'] = ','.join(set(hit['subject id'] for hit in top_v_hits))
+        
         result['sequence_aa_vdj'] = self._appendSeq(seq_vdj, v_hit, 0, trim=False)
         result['sequence_aa_trim'] = self._appendSeq(seq_trim, v_hit, 0, trim=True)
 
@@ -2296,7 +2355,7 @@ def countDbFile(file):
     """
     # Count records and check file
     try:
-        with open(file, 'rt') as db_handle:
+        with openFile(file, 'rt') as db_handle:
             db_records = csv.reader(db_handle, dialect='excel-tab')
             for i, __ in enumerate(db_records):  pass
         db_count = i
@@ -2324,7 +2383,7 @@ def getDbFields(file, add=None, exclude=None, reader=TSVReader):
         list : list of field names
     """
     try:
-        with open(file, 'rt') as handle:
+        with openFile(file, 'rt') as handle:
             fields = reader(handle).fields
     except IOError:
         printError('File %s cannot be read.' % file)
@@ -2384,16 +2443,22 @@ def splitName(file):
       file (str): file name.
 
     Returns:
-      tuple : tuple of the file directory, basename and extension.
+      tuple : tuple of the file directory, basename and extension. A trailing
+              .gz extension is stripped from file before the extension is
+              determined, so a gzip compressed file reports the extension of
+              its underlying (uncompressed) type.
     """
     directory, filename = os.path.split(file)
+    if filename.lower().endswith('.gz'):
+        filename = filename[:-3]
     basename, extension = os.path.splitext(filename)
     extension = extension.lower().lstrip('.')
 
     return directory, basename, extension
 
 
-def getOutputName(file, out_label=None, out_dir=None, out_name=None, out_type=None):
+def getOutputName(file, out_label=None, out_dir=None, out_name=None, out_type=None,
+                  gzip_output=None):
     """
     Creates and output filename from an existing filename
 
@@ -2407,11 +2472,15 @@ def getOutputName(file, out_label=None, out_dir=None, out_name=None, out_type=No
                 if None use directory of input file
       out_name : the short filename to use for the output file;
                  if None use input file short name.
+      gzip_output : if True force a .gz extension onto the output file name;
+                    if None add a .gz extension only when file is itself
+                    gzip compressed (preserve compression by default).
 
     Returns:
       str: file name.
     """
     # Get filename components
+    was_gzipped = os.path.split(file)[1].lower().endswith('.gz')
     directory, basename, extension = splitName(file)
 
     # Define output directory
@@ -2431,11 +2500,16 @@ def getOutputName(file, out_label=None, out_dir=None, out_name=None, out_type=No
     else:
         out_file = os.path.join(out_dir, '%s_%s.%s' % (out_name, out_label, out_type))
 
+    # Preserve or force gzip compression of the output file name
+    if gzip_output or (gzip_output is None and was_gzipped):
+        out_file += '.gz'
+
     # Return file name
     return out_file
 
 
-def getOutputHandle(file, out_label=None, out_dir=None, out_name=None, out_type=None):
+def getOutputHandle(file, out_label=None, out_dir=None, out_name=None, out_type=None,
+                    gzip_output=None):
     """
     Opens an output file handle
 
@@ -2449,16 +2523,20 @@ def getOutputHandle(file, out_label=None, out_dir=None, out_name=None, out_type=
                 if None use directory of input file
       out_name : the short filename to use for the output file;
                  if None use input file short name.
+      gzip_output : if True force a .gz extension onto the output file name
+                    and open it as gzip compressed; if None preserve
+                    compression when file is itself gzip compressed.
 
     Returns:
       file : File handle
     """
     out_file = getOutputName(file, out_label=out_label, out_dir=out_dir,
-                             out_name=out_name, out_type=out_type)
+                             out_name=out_name, out_type=out_type,
+                             gzip_output=gzip_output)
 
     # Open and return handle
     try:
-        return open(out_file, mode='w')
+        return openFile(out_file, mode='w')
     except:
         printError('File %s cannot be opened.' % out_file)
 


=====================================
changeo/Multiprocessing.py
=====================================
@@ -14,7 +14,7 @@ from time import time
 # Presto and changeo imports
 from presto.IO import printProgress, printLog, printError, printWarning
 from changeo.Defaults import default_out_args
-from changeo.IO import countDbFile, getOutputHandle, AIRRReader, AIRRWriter
+from changeo.IO import countDbFile, getOutputHandle, gzipOutputName, openFile, AIRRReader, AIRRWriter
 from changeo.Receptor import Receptor
 
 
@@ -113,7 +113,7 @@ def feedDbQueue(alive, data_queue, db_file, reader=AIRRReader, group_func=None,
     # Open input file and perform grouping
     try:
         # Iterate over records and assign groups
-        db_handle = open(db_file, 'rt')
+        db_handle = openFile(db_file, 'rt')
         db_iter = reader(db_handle)
         if group_func is not None:
             # import cProfile
@@ -224,13 +224,14 @@ def collectDbQueue(alive, result_queue, collect_queue, db_file, label, fields,
     # Wrapper for opening handles and writers
     def _open(x, f, writer=writer, label=label, out_file=out_file):
         if out_file is not None and x == 'pass':
-            handle = open(out_file, 'w')
+            handle = openFile(gzipOutputName(out_file, out_args.get('gzip_output')), 'w')
         else:
             handle = getOutputHandle(db_file,
                                      out_label='%s-%s' % (label, x),
                                      out_dir=out_args['out_dir'],
                                      out_name=out_args['out_name'],
-                                     out_type=out_args['out_type'])
+                                     out_type=out_args['out_type'],
+                                     gzip_output=out_args.get('gzip_output'))
         return handle, writer(handle, fields=f)
 
     try:


=====================================
changeo/Version.py
=====================================
@@ -3,7 +3,7 @@ Version and authorship information
 """
 
 __author__    = 'Namita Gupta, Jason Anthony Vander Heiden'
-__copyright__ = 'Copyright 2021 Kleinstein Lab, Yale University. All rights reserved.'
+__copyright__ = 'Copyright 2026 Kleinstein Lab, Yale University. All rights reserved.'
 __license__   = 'GNU Affero General Public License 3 (AGPL-3)'
-__version__   = '1.3.0'
-__date__      = '2022.12.11'
+__version__   = '1.3.5'
+__date__      = '2026.09.02'


=====================================
pyproject.toml
=====================================
@@ -0,0 +1,3 @@
+[build-system]
+requires = ["setuptools>=40.8.0", "wheel"]
+build-backend = "setuptools.build_meta:__legacy__"


=====================================
requirements.txt
=====================================
@@ -1,8 +1,10 @@
-numpy>=1.8
-scipy>=0.14
-pandas>=0.24
-biopython>=1.77
-PyYAML>=5.1
-setuptools>=2.0
-presto>=0.7.0
+numpy>=1.23.2
+scipy>=1.9.3
+pandas>=1.5.0
+biopython>=1.81
+PyYAML>=6.0
+setuptools>=65.5.0
+presto>=0.7.1
 airr>=1.3.1
+packaging>=21.3
+importlib-resources>=6.4.0


=====================================
setup.py
=====================================
@@ -7,8 +7,8 @@ import os
 import sys
 
 # Check setup requirements
-if sys.version_info < (3,4,0):
-    sys.exit('At least Python 3.4.0 is required.\n')
+if sys.version_info < (3,10,0):
+    sys.exit('At least Python 3.10.0 is required.\n')
 
 try:
     from setuptools import setup
@@ -63,7 +63,7 @@ setup(name='changeo',
       zip_safe=False,
       license=__license__,
       url='http://changeo.readthedocs.io',
-      download_url='https://bitbucket.org/kleinstein/changeo/downloads',
+      download_url='https://github.com/immcantation/changeo/tags',
       keywords=['bioinformatics', 'sequencing', 'immunology', 'adaptive immunity',
                 'immunoglobulin', 'AIRR-seq', 'Rep-Seq',
                 'B cell repertoire analysis', 'adaptive immune receptor repertoires'],
@@ -77,5 +77,5 @@ setup(name='changeo',
                    'Intended Audience :: Science/Research',
                    'Natural Language :: English',
                    'Operating System :: OS Independent',
-                   'Programming Language :: Python :: 3.4',
+                   'Programming Language :: Python :: 3.10',
                    'Topic :: Scientific/Engineering :: Bio-Informatics'])


=====================================
tests/test_DefineClones.py
=====================================
@@ -0,0 +1,253 @@
+"""
+Unit tests for DefineClones
+"""
+# Info
+__author__ = 'Jason Anthony Vander Heiden'
+from changeo import __version__, __date__
+
+# Imports
+import os
+import sys
+import time
+import unittest
+from copy import deepcopy
+from itertools import chain
+
+# Presto and changeo imports
+from changeo.Receptor import Receptor
+from changeo.Multiprocessing import DbResult
+
+# Paths
+test_path = os.path.dirname(os.path.realpath(__file__))
+data_path = os.path.join(test_path, 'data')
+
+# Import script
+sys.path.append(os.path.join(test_path, os.pardir, 'bin'))
+import DefineClones
+
+
+class Test_DefineClones(unittest.TestCase):
+    def setUp(self):
+        print('-> %s()' % self._testMethodName)
+
+        self.mg_ig_file = os.path.join(data_path, 'AB0RF_MG67U_functional-heavy_parse-select_filter-collapse.tab')
+
+        # Define list of preclone properties
+        group_list = [{'V_CALL': 'IGHV1-1*01',
+                       'D_CALL': 'IGHD6-6*01',
+                       'J_CALL': 'IGHJ6*01',
+                       'JUNCTION_LENGTH': 48,
+                       'FIELD1': 'GA',
+                       'FIELD2': '1'},
+                      {'V_CALL': 'IGHV2-1*01',
+                       'D_CALL': 'IGHD6-6*01',
+                       'J_CALL': 'IGHJ6*01',
+                       'JUNCTION_LENGTH': 48,
+                       'FIELD1': 'GA',
+                       'FIELD2': '1'},
+                      {'V_CALL': 'IGHV3-1*01',
+                       'D_CALL': 'IGHD6-6*01',
+                       'J_CALL': 'IGHJ6*01',
+                       'JUNCTION_LENGTH': 48,
+                       'FIELD1': 'GA',
+                       'FIELD2': '2'},
+                      {'V_CALL': 'IGHV1-1*01, IGHV2-1*01, IGHV3-1*01',
+                       'D_CALL': 'IGHD6-6*01',
+                       'J_CALL': 'IGHJ6*01',
+                       'JUNCTION_LENGTH': 48,
+                       'FIELD1': 'GA',
+                       'FIELD2': '2'},
+                      {'V_CALL': 'IGHV4-1*01',
+                       'D_CALL': 'IGHD6-6*01',
+                       'J_CALL': 'IGHJ6*01',
+                       'JUNCTION_LENGTH': 48,
+                       'FIELD1': 'GB',
+                       'FIELD2': '1'},
+                      {'V_CALL': 'IGHV2-1*01, IGHV4-1*01',
+                       'D_CALL': 'IGHD6-6*01',
+                       'J_CALL': 'IGHJ6*01',
+                       'JUNCTION_LENGTH': 48,
+                       'FIELD1': 'GB',
+                       'FIELD2': '2'},
+                      {'V_CALL': 'IGHV5-1*01',
+                       'D_CALL': 'IGHD6-6*01',
+                       'J_CALL': 'IGHJ6*01',
+                       'JUNCTION_LENGTH': 48,
+                       'FIELD1': 'GB',
+                       'FIELD2': None},
+                      {'V_CALL': 'IGHV5-1*01, IGHV6-1*01',
+                       'D_CALL': 'IGHD6-6*01',
+                       'J_CALL': 'IGHJ6*01',
+                       'JUNCTION_LENGTH': 48,
+                       'FIELD1': '',
+                       'FIELD2': '1'}]
+
+        # Define unique sequences
+        seq_list = [{'SEQUENCE_ID': 'A1',
+                     'SEQUENCE_INPUT': 'TGTGCAAGGGGGCCATTGGACTACTTCTACTACGGTGTGGACGTCTGG',
+                     'JUNCTION': 'TGTGCAAGGGGGCCATTGGACTACTTCTACTACGGTGTGGACGTCTGG'},
+                    {'SEQUENCE_ID': 'A2',
+                     'SEQUENCE_INPUT': 'TGTGCAAGGGGGCCATTGGACTACTTCTACTACGGTGTGGACGTCTGG',
+                     'JUNCTION': 'TGTGCAAGGGGGCCATTGGACTACTTCTACTACGGTGTGGACGTCTGG'},
+                    {'SEQUENCE_ID': 'A3',
+                     'SEQUENCE_INPUT': 'TGTGCAAGGGGGCCATTGGACTACTTCTACTACGGTGTGGACGTCTGG',
+                     'JUNCTION': 'TGTGCAAGGGGGCCATTGGACTACTTCTACTACGGTGTGGACGTCTGG'},
+                    {'SEQUENCE_ID': 'A4',
+                     'SEQUENCE_INPUT': 'TGTGCAAGGGGGCCATTGGACTACTTCTACTACGGTGTGGACGNNNNN',
+                     'JUNCTION': 'TGTGCAAGGGGGCCATTGGACTACTTCTACTACGGTGTGGACGNNNNN'},
+                    {'SEQUENCE_ID': 'B1',
+                     'SEQUENCE_INPUT': 'TGTGCAAGATATAGCAGCAGCTACTACTACTACGGTATGGACGTCTGG',
+                     'JUNCTION': 'TGTGCAAGATATAGCAGCAGCTACTACTACTACGGTATGGACGTCTGG'},
+                    {'SEQUENCE_ID': 'B2',
+                     'SEQUENCE_INPUT': 'TGTGNAAGATNTAGCAGCAGCTACTACTACTACGGTATNGACGTCTGG',
+                     'JUNCTION': 'TGTGNAAGATNTAGCAGCAGCTACTACTACTACGGTATNGACGTCTGG'},
+                    {'SEQUENCE_ID': 'B3',
+                     'SEQUENCE_INPUT': 'TGTGCAAGATATAGCAGCAGCTACTACTACTACGGTATGGACGTCTGG',
+                     'JUNCTION': 'TGTGCAAGATATAGCAGCAGCTACTACTACTACGGTATGGACGTCTGG'},
+                    {'SEQUENCE_ID': 'B4',
+                     'SEQUENCE_INPUT': 'TGTGNAAGATNTAGCAGCAGCTACTACTACTACGGTATNGACGTCTGG',
+                     'JUNCTION': 'TGTGNAAGATNTAGCAGCAGCTACTACTACTACGGTATNGACGTCTGG'}]
+
+        # Build preclone Receptor list with unambiguous gene calls
+        seq_copy = deepcopy(seq_list)
+        for x in seq_copy:  x.update(deepcopy(group_list[1]))
+        self.unambig_records = [Receptor(x) for x in seq_copy]
+        self.unambig_data = DbResult('unambig', self.unambig_records)
+        self.unambig_clones = {('A1', 'A2', 'A3', 'A4'),
+                               ('B1', 'B2', 'B3', 'B4')}
+
+        # Build db iterator with ambiguous assignments
+        group_copy = deepcopy(group_list)
+        seq_copy = deepcopy(seq_list)
+        for i, x in enumerate(group_copy):  x.update(seq_copy[i])
+        self.ambig_records = [Receptor(x) for x in group_copy]
+        self.ambig_data = DbResult('ambig', self.ambig_records)
+        self.first_nofields = {('IGHV1-1', 'IGHJ6', '48'): ['A1','A4'],
+                               ('IGHV2-1', 'IGHJ6', '48'): ['A2','B2'],
+                               ('IGHV3-1', 'IGHJ6', '48'): ['A3'],
+                               ('IGHV4-1', 'IGHJ6', '48'): ['B1'],
+                               ('IGHV5-1', 'IGHJ6', '48'): ['B3','B4']}
+        self.set_nofields = {('48', ('IGHJ6',), ('IGHV1-1', 'IGHV2-1', 'IGHV3-1', 'IGHV4-1')):
+                             ['A1', 'A2', 'A3', 'A4', 'B1', 'B2'],
+                             ('48', ('IGHJ6',), ('IGHV5-1', 'IGHV6-1')):
+                             ['B3', 'B4']}
+        self.first_fields = {('IGHV1-1', 'IGHJ6', '48', 'GA', '1'): ['A1'],
+                             ('IGHV2-1', 'IGHJ6', '48', 'GA', '1'): ['A2'],
+                             ('IGHV3-1', 'IGHJ6', '48', 'GA', '2'): ['A3'],
+                             ('IGHV1-1', 'IGHJ6', '48', 'GA', '2'): ['A4'],
+                             ('IGHV4-1', 'IGHJ6', '48', 'GB', '1'): ['B1'],
+                             ('IGHV2-1', 'IGHJ6', '48', 'GB', '2'): ['B2'],
+                             None: ['B3','B4']}
+        self.set_fields = {('48', 'GA', '1', ('IGHJ6',), ('IGHV1-1',)): ['A1'],
+                           ('48', 'GA', '1', ('IGHJ6',), ('IGHV2-1',)): ['A2'],
+                           ('48', 'GA', '2', ('IGHJ6',), ('IGHV1-1', 'IGHV2-1', 'IGHV3-1')): ['A3', 'A4'],
+                           ('48', 'GB', '1', ('IGHJ6',), ('IGHV4-1',)): ['B1'],
+                           ('48', 'GB', '2', ('IGHJ6',), ('IGHV2-1', 'IGHV4-1')): ['B2'],
+                           None: ['B3','B4']}
+        self.start = time.time()
+
+    def tearDown(self):
+        t = time.time() - self.start
+        print("<- %s() %.3f" % (self._testMethodName, t))
+
+    # @unittest.skip("-> groupByGene() skipped\n")
+    def test_groupByGene(self):
+        # Test first grouping without fields
+        results = DefineClones.groupByGene(self.ambig_records, mode='gene', action='first')
+        # Extract nested keys and group lengths for comparison
+        results_dict = dict()
+        print('FIRST>')
+        for k, v in results.items():
+            nest_key = tuple([tuple(sorted(chain(x))) if isinstance(x, tuple) else str(x) for x in k])
+            results_dict[nest_key] = sorted([x.sequence_id for x in v])
+            print('  GROUP>', nest_key, ':', results_dict[nest_key])
+        print('')
+        self.assertDictEqual(self.first_nofields, results_dict)
+
+        # Test ambiguous grouping without fields
+        results = DefineClones.groupByGene(self.ambig_records, mode='gene', action='set')
+        # Extract nested keys and group lengths for comparison
+        results_dict = dict()
+        print('SET>')
+        for k, v in results.items():
+            nest_key = tuple([tuple(sorted(chain(x))) if isinstance(x, tuple) else str(x) for x in k])
+            results_dict[nest_key] = sorted([x.sequence_id for x in v])
+            print('  GROUP>', nest_key, ':', results_dict[nest_key])
+        print('')
+        self.assertDictEqual(self.set_nofields, results_dict)
+
+        # Test first grouping with fields
+        results = DefineClones.groupByGene(self.ambig_records, group_fields=['FIELD1', 'FIELD2'],
+                                           mode='gene', action='first')
+        # Extract nested keys and group lengths for comparison
+        results_dict = dict()
+        print('FIRST>')
+        for k, v in results.items():
+            nest_key = tuple([tuple(sorted(chain(x))) if isinstance(x, tuple) else str(x) for x in k]) if k is not None else None
+            results_dict[nest_key] = sorted([x.sequence_id for x in v])
+            print('  GROUP>', nest_key, ':', results_dict[nest_key])
+        print('')
+        print('RRRR>', self.first_fields)
+        print('QQQQ>', results_dict)
+
+        self.assertDictEqual(self.first_fields, results_dict)
+
+        # Test ambiguous grouping with fields
+        results = DefineClones.groupByGene(self.ambig_records, group_fields=['FIELD1', 'FIELD2'],
+                                              mode='gene', action='set')
+        # Extract nested keys and group lengths for comparison
+        results_dict = dict()
+        print('SET>')
+        for k, v in results.items():
+            nest_key = tuple([tuple(sorted(chain(x))) if isinstance(x, tuple) else str(x) for x in k]) if k is not None else None
+            results_dict[nest_key] = sorted([x.sequence_id for x in v])
+            print('  GROUP>', nest_key, ':', results_dict[nest_key])
+        print('')
+        self.assertDictEqual(self.set_fields, results_dict)
+
+
+    # @unittest.skip("-> distanceClones() skipped\n")
+    def test_distanceClones(self):
+        # import cProfile
+        # prof = cProfile.Profile()
+        # results = prof.runcall(DefineClones.distanceClones, self.unambig_records, model='hs5f', distance=1.0, dist_mat=self.dist_mat)
+        # prof.dump_stats('hs5f-unit-test-dict.prof')
+
+        # ham model
+        results = DefineClones.distanceClones(self.unambig_data, model='ham', distance=9.0, norm='none')
+        results_set = set([tuple(sorted([x.sequence_id for x in v])) for v in results.results.values()])
+        print('MODEL> ham')
+        for i, x in enumerate(results_set):  print('  CLONE-%i> %s' % (i + 1, x))
+        self.assertSetEqual(results_set, self.unambig_clones)
+
+        # m1n_compat model
+        results = DefineClones.distanceClones(self.unambig_data, model='m1n_compat', distance=10.0, norm='none')
+        results_set = set([tuple(sorted([x.sequence_id for x in v])) for v in results.results.values()])
+        print('MODEL> m1n_compat')
+        for i, x in enumerate(results_set):  print('  CLONE-%i> %s' % (i + 1, x))
+        self.assertSetEqual(results_set, self.unambig_clones)
+
+        # hs1f_compat model
+        results = DefineClones.distanceClones(self.unambig_data, model='hs1f_compat', distance=0.25)
+        results_set = set([tuple(sorted([x.sequence_id for x in v])) for v in results.results.values()])
+        print('MODEL> hs1f_compat')
+        for i, x in enumerate(results_set):  print('  CLONE-%i> %s' % (i + 1, x))
+        self.assertSetEqual(results_set, self.unambig_clones)
+
+        # hh_s5f model
+        results = DefineClones.distanceClones(self.unambig_data, model='hh_s5f', distance=0.1)
+        results_set = set([tuple(sorted([x.sequence_id for x in v])) for v in results.results.values()])
+        print('MODEL> hh_s5f')
+        for i, x in enumerate(results_set):  print('  CLONE-%i> %s' % (i + 1, x))
+        self.assertSetEqual(results_set, self.unambig_clones)
+
+        # aa model
+        results = DefineClones.distanceClones(self.unambig_data, model='aa', distance=3.0, norm='none')
+        results_set = set([tuple(sorted([x.sequence_id for x in v])) for v in results.results.values()])
+        print('MODEL> aa')
+        for i, x in enumerate(results_set):  print('  CLONE-%i> %s' % (i + 1, x))
+        self.assertSetEqual(results_set, self.unambig_clones)
+
+
+if __name__ == '__main__':
+    unittest.main()
\ No newline at end of file


=====================================
tests/test_Distance.py
=====================================
@@ -0,0 +1,188 @@
+"""
+Unit tests for Distance module
+"""
+# Info
+__author__ = 'Jason Anthony Vander Heiden'
+from changeo import __version__, __date__
+
+# Imports
+import os
+import time
+import unittest
+import numpy as np
+from Bio.Seq import Seq
+
+# Presto and changeo imports
+import changeo.Distance as Distance
+
+# Paths
+test_path = os.path.dirname(os.path.realpath(__file__))
+data_path = os.path.join(test_path, 'data')
+
+
+class Test_Distance(unittest.TestCase):
+    def setUp(self):
+        print('-> %s()' % self._testMethodName)
+
+        self.nt_seq = ['TGTGCAAGGGGGCCA',
+                       'TGTGCAAGGGGGCCA',
+                       'TGTATTTGGGGGCCA',
+                       'ACACTTGCCACTGAT',
+                       'NNNNNNNNNNNNTGA',
+                       'NNNNNNNNNNNNNNN']
+        self.nt_seq_short = ['TGTGCAAGG',
+                             'TGTGCAAGG',
+                             'TGTATTTGG',
+                             'NNNNNNNNN']
+        self.aa_seq = [str(Seq(x).translate()) for x in self.nt_seq]
+
+
+        # Amino acid distance from sequence 0 to sequence 0,..,5
+        self.aa_len = 5.0
+        self.aa_ham = np.array([0.0, 0.0, 2.0, 5.0, 1.0, 0.0])
+
+        # Nucleotide distances sequence 0 to sequence 0,..,5
+        self.nt_len = 15.0
+        # 0 vs 2
+        #   A-C = 0; A-G = 1; A-T = 2;
+        #   C-G = 0; C-T = 1; G-T = 0
+        # 0 vs 3
+        #   A-C = 1; A-G = 2; A-T = 4;
+        #   C-G = 6; C-T = 1; G-T = 1
+        # 0 vs 4
+        #   A-C = 0; A-G = 0; A-T = 0;
+        #   C-G = 1; C-T = 1; G-T = 0
+        self.nt_ham = np.array([0.0, 0.0, 4.0, 15.0, 2.0, 0.0])
+        self.nt_hh_s1f = np.array([0.0,
+                                   0.0,
+                                   0.0 + 1*0.64 + 2*1.16 + 0.0 + 1*0.64 + 0.0,
+                                   1*1.21 + 2*0.64 + 4*1.16 + 6*1.16 + 1*0.64 + 1*1.21,
+                                   0.0 + 0.0 + 0.0 + 0.0 + 1*1.16 + 1*0.64 + 0.0,
+                                   0.0])
+        self.nt_mk_rs1nf = np.array([0.0,
+                                     0.0,
+                                     0.0 + 1*0.32 + 2*1.17 + 0.0 + 1*0.32 + 0.0,
+                                     1*1.51 + 2*0.32 + 4*1.17 + 6*1.17 + 1*0.32 + 1*1.51,
+                                     0.0 + 0.0 + 0.0 + 0.0 + 1*1.17 + 1*0.32 + 0.0,
+                                     0.0])
+
+        # 5-mer models use shorter sequence
+        # For sequence 0 to sequence 0,..,3
+        self.nt_short_len = 9.0
+        # hh_hs5f
+        #   GTGCA-GTATT = [0.97, 0.84]
+        #   TGCAA-TATTT = [0.93, 0.83]
+        #   GCAAG-ATTTG = [1.08, 1.16]
+        #   CAAGG-TTTGG = [0.91, 1.07]
+        self.nt_hh_s5f_avg = np.array([0.0,
+                                       0.0,
+                                       np.mean([0.97, 0.84]) + np.mean([0.93, 0.83]) +
+                                       np.mean([1.08, 1.16]) + np.mean([0.91, 1.07]),
+                                       0.0])
+        self.nt_hh_s5f_min = np.array([0.0,
+                                       0.0,
+                                       np.min([0.97, 0.84]) + np.min([0.93, 0.83]) +
+                                       np.min([1.08, 1.16]) + np.min([0.91, 1.07]),
+                                       0.0])
+        # mk_rs5nf
+        #   GTGCA-GTATT = [0.71, 0.77]
+        #   TGCAA-TATTT = [0.71, 0.93]
+        #   GCAAG-ATTTG = [1.05, 1.03]
+        #   CAAGG-TTTGG = [1.08, 1.13]
+        self.nt_mk_rs5nf_avg = np.array([0.0,
+                                         0.0,
+                                         np.mean([0.71, 0.77]) + np.mean([0.71, 0.93]) +
+                                         np.mean([1.05, 1.03]) + np.mean([1.08, 1.13]),
+                                         0.0])
+        self.nt_mk_rs5nf_min = np.array([0.0,
+                                         0.0,
+                                         np.min([0.71, 0.77]) + np.min([0.71, 0.93]) +
+                                         np.min([1.05, 1.03]) + np.min([1.08, 1.13]),
+                                         0.0])
+
+        self.start = time.time()
+
+    def tearDown(self):
+        t = time.time() - self.start
+        print("<- %s() %.3f" % (self._testMethodName, t))
+        
+    def test_calcDistances(self):
+        # aa
+        print(' AA>', self.aa_seq)
+        result = Distance.calcDistances(self.aa_seq, n=1, dist_mat=Distance.aa_model, norm=None)
+        print(' AA>', result[1, :])
+        np.testing.assert_almost_equal(result[1, :], self.aa_ham)
+
+        result = Distance.calcDistances(self.aa_seq, n=1, dist_mat=Distance.aa_model, norm='len')
+        np.testing.assert_almost_equal(result[1, :], self.aa_ham / self.aa_len)
+
+        # ham
+        print(' NT>', self.nt_seq)
+        result = Distance.calcDistances(self.nt_seq, n=1, dist_mat=Distance.ham_model, norm=None)
+        print('HAM>', result[1, :])
+        np.testing.assert_almost_equal(result[1, :], self.nt_ham)
+
+        result = Distance.calcDistances(self.nt_seq, n=1, dist_mat=Distance.ham_model, norm='len')
+        np.testing.assert_almost_equal(result[1, :], self.nt_ham / self.nt_len)
+
+        # hh_s1f
+        result = Distance.calcDistances(self.nt_seq, n=1, dist_mat=Distance.hh_s1f_model, norm=None)
+        print('HH_S1F>\n', result[1, :])
+        np.testing.assert_almost_equal(result[1, :], self.nt_hh_s1f)
+
+        result = Distance.calcDistances(self.nt_seq, n=1, dist_mat=Distance.hh_s1f_model, norm='len')
+        np.testing.assert_almost_equal(result[1, :], self.nt_hh_s1f / self.nt_len)
+
+        # mk_rs1nf
+        result = Distance.calcDistances(self.nt_seq, n=1, dist_mat=Distance.mk_rs1nf_model, norm=None)
+        print('MK_RS1NF>\n', result[1, :])
+        np.testing.assert_almost_equal(result[1, :], self.nt_mk_rs1nf)
+
+        result = Distance.calcDistances(self.nt_seq, n=1, dist_mat=Distance.mk_rs1nf_model, norm='len')
+        np.testing.assert_almost_equal(result[1, :], self.nt_mk_rs1nf / self.nt_len)
+
+        # hh_s5f
+        result = Distance.calcDistances(self.nt_seq_short, n=5, dist_mat=Distance.hh_s5f_model, sym='avg', norm=None)
+        print('HH_S5F-AVG>\n', result[1, :])
+        np.testing.assert_almost_equal(result[1, :], self.nt_hh_s5f_avg)
+
+        result = Distance.calcDistances(self.nt_seq_short, n=5, dist_mat=Distance.hh_s5f_model, sym='avg', norm='len')
+        np.testing.assert_almost_equal(result[1, :], self.nt_hh_s5f_avg / self.nt_short_len)
+
+        result = Distance.calcDistances(self.nt_seq_short, n=5, dist_mat=Distance.hh_s5f_model, sym='min', norm=None)
+        print('HH_S5F-MIN>\n', result[1, :])
+        np.testing.assert_almost_equal(result[1, :], self.nt_hh_s5f_min)
+
+        result = Distance.calcDistances(self.nt_seq_short, n=5, dist_mat=Distance.hh_s5f_model, sym='min', norm='len')
+        np.testing.assert_almost_equal(result[1, :], self.nt_hh_s5f_min / self.nt_short_len)
+
+        # mk_rs5nf
+        result = Distance.calcDistances(self.nt_seq_short, n=5, dist_mat=Distance.mk_rs5nf_model, sym='avg', norm=None)
+        print('MK_RS5NF-AVG>\n', result[1, :])
+        np.testing.assert_almost_equal(result[1, :], self.nt_mk_rs5nf_avg)
+
+        result = Distance.calcDistances(self.nt_seq_short, n=5, dist_mat=Distance.mk_rs5nf_model, sym='avg', norm='len')
+        np.testing.assert_almost_equal(result[1, :], self.nt_mk_rs5nf_avg / self.nt_short_len)
+
+        result = Distance.calcDistances(self.nt_seq_short, n=5, dist_mat=Distance.mk_rs5nf_model, sym='min', norm=None)
+        print('MK_RS5NF-MIN>\n', result[1, :])
+        np.testing.assert_almost_equal(result[1, :], self.nt_mk_rs5nf_min)
+
+        result = Distance.calcDistances(self.nt_seq_short, n=5, dist_mat=Distance.mk_rs5nf_model, sym='min', norm='len')
+        np.testing.assert_almost_equal(result[1, :], self.nt_mk_rs5nf_min / self.nt_short_len)
+
+        # Raise error on length mismatch
+        with self.assertRaises(IndexError):
+            Distance.calcDistances(['ATGCAA', 'ATGC'], n=1, dist_mat=Distance.ham_model)
+
+
+    @unittest.skip("-> loadModels() skipped\n")
+    def test_loadModels(self):
+        print('MK_RS1NF> ', Distance.mk_rs1nf_model)
+        print('  HH_S1F> ', Distance.hh_s5f_model)
+
+        self.fail('TODO')
+
+
+if __name__ == '__main__':
+    unittest.main()


=====================================
tests/test_IO.py
=====================================
@@ -0,0 +1,264 @@
+"""
+Unit tests for MakeDb
+"""
+# Info
+__author__ = 'Jason Anthony Vander Heiden'
+from changeo import __version__, __date__
+
+# Imports
+import gzip
+import os
+import shutil
+import sys
+import tempfile
+import time
+import unittest
+from Bio import SeqIO
+
+# Presto and changeo imports
+from changeo.Commandline import getCommonArgParser, parseCommonArgs
+from changeo.IO import getDbFields, extractIMGT, readGermlines, ChangeoReader, \
+                       IgBLASTReader, IgBLASTReaderAA, IHMMuneReader, IMGTReader, openFile
+
+# Paths
+test_path = os.path.dirname(os.path.realpath(__file__))
+data_path = os.path.join(test_path, 'data')
+
+# Import script
+sys.path.append(os.path.join(test_path, os.pardir, 'bin'))
+import MakeDb
+
+
+class Test_MakeDb(unittest.TestCase):
+    def setUp(self):
+        print('-> %s()' % self._testMethodName)
+
+        # Germline files
+        self.repo_ig = '/usr/local/share/germlines/imgt/human/vdj'
+        self.repo_ig_aa = '/usr/local/share/germlines/imgt/human/vdj_aa'
+        # Read files
+        self.reads_ig = os.path.join(data_path, 'reads_ig.fasta')
+        self.reads_ig_aa = os.path.join(data_path, 'reads_ig_aa.fasta')
+        # IMGT output
+        self.imgt_ig = os.path.join(data_path, 'imgt_ig.txz')
+        # IgBLAST output
+        self.igblast_ig = os.path.join(data_path, 'igblast1.7_ig.fmt7')
+        self.igblast_ig_aa = os.path.join(data_path, 'igblast1.14_ig_aa.fmt7')
+        # iHMMune-Align output
+        self.ihmmune_ig = os.path.join(data_path, 'ihmmune_ig.csv')
+        # Change-O files
+        self.db_ig = os.path.join(data_path, 'imgt_ig_db-pass.tsv')
+
+        self.start = time.time()
+
+    def tearDown(self):
+        t = time.time() - self.start
+        print("<- %s() %.3f" % (self._testMethodName, t))
+
+    @unittest.skip("-> ChangeoReader() skipped\n")
+    def test_getDbFields(self):
+        # Get fields
+        x = getDbFields(self.db_ig)
+        print(x)
+
+        # Add fields
+        x = getDbFields(self.db_ig, add=['A', 'B', 'C'])
+        print(x)
+
+        # Exclude fields
+        x = getDbFields(self.db_ig, exclude=['V_SCORE', 'V_IDENTITY', 'J_SCORE', 'J_IDENTITY'])
+        print(x)
+
+        # Add and exclude fields
+        x = getDbFields(self.db_ig, add=['A', 'B', 'C'],
+                        exclude=['V_SCORE', 'V_IDENTITY', 'J_SCORE', 'J_IDENTITY'])
+        print(x)
+
+        self.fail('TODO')
+
+    @unittest.skip("-> ChangeoReader() skipped\n")
+    def test_ChangeoReader(self):
+        # Parse
+        with open(self.db_ig, 'r') as f:
+            result = ChangeoReader(f)
+            for x in result: print(x.toDict())
+
+        self.fail('TODO')
+
+    @unittest.skip("-> IMGTReader() skipped\n")
+    def test_IMGTReader(self):
+        # Extract IMGT files
+        temp_dir, files = extractIMGT(self.imgt_ig)
+
+        # Parse
+        with open(files['summary'], 'r') as summary, \
+                open(files['gapped'], 'r') as gapped, \
+                open(files['ntseq'], 'r') as ntseq, \
+                open(files['junction'], 'r') as junction:
+            result = IMGTReader(summary, gapped, ntseq, junction, receptor=False)
+            for x in result: print(x)
+
+        # Remove IMGT temporary directory
+        temp_dir.cleanup()
+
+        self.fail('TODO')
+
+    @unittest.skip("-> IgBLASTReader() skipped\n")
+    def test_IgBLASTReader(self):
+        # Load germlines and sequences
+        seq_dict = MakeDb.getSeqDict(self.reads_ig_aa)
+        repo_dict = readGermlines([self.repo_ig])
+
+        # Parse
+        with open(self.igblast_ig, 'r') as f:
+            result = IgBLASTReader(f, seq_dict, repo_dict, receptor=False)
+            for x in result: print(x)
+
+        self.fail('TODO')
+
+    @unittest.skip("-> IgBLASTAAReader() skipped\n")
+    def test_IgBLASTAAReader(self):
+        # Load germlines and sequences
+        seq_dict = MakeDb.getSeqDict(self.reads_ig_aa)
+        repo_dict = readGermlines([self.repo_ig_aa])
+
+        # Parse
+        with open(self.igblast_ig_aa, 'r') as f:
+            result = IgBLASTReaderAA(f, seq_dict, repo_dict, receptor=False)
+            for x in result: print(x)
+
+        # self.fail('TODO')
+
+    @unittest.skip("-> IHMMReader() skipped\n")
+    def test_IHMMReader(self):
+        # Load germlines and sequences
+        seq_dict = MakeDb.getSeqDict(self.reads_ig)
+        repo_dict = readGermlines([self.repo_ig])
+
+        # Parse
+        with open(self.ihmmune_ig, 'r') as f:
+            result = IHMMuneReader(f, seq_dict, repo_dict, receptor=False)
+            for x in result: print(x)
+
+        self.fail('TODO')
+
+    @unittest.skip("-> igblast-aa partial execution test skipped\n")
+    def test_igblast_aa_partial_execution(self):
+        """Test actual execution of MakeDb.py with and without --partial"""
+        import subprocess, tempfile, shutil
+        temp_dir = tempfile.mkdtemp()
+        try:
+            strict_pass = os.path.join(temp_dir, 'strict_db-pass.tsv')
+            strict_fail = os.path.join(temp_dir, 'strict_db-fail.tsv')
+            partial_pass = os.path.join(temp_dir, 'partial_db-pass.tsv')
+            partial_fail = os.path.join(temp_dir, 'partial_db-fail.tsv')
+            bin_path = os.path.join(os.path.dirname(os.path.dirname(os.path.realpath(__file__))), 'bin')
+            # Run MakeDb.py with strict validation (default)
+            strict_cmd = [
+                os.path.join(bin_path, 'MakeDb.py'),
+                'igblast-aa',
+                '-i', self.igblast_ig_aa,
+                '-s', self.reads_ig_aa,
+                '-r', self.repo_ig_aa,
+                '--outname', os.path.join(temp_dir, 'strict')
+            ]
+            subprocess.run(strict_cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+            # Run MakeDb.py with partial validation
+            partial_cmd = [
+                os.path.join(bin_path, 'MakeDb.py'),
+                'igblast-aa',
+                '-i', self.igblast_ig_aa,
+                '-s', self.reads_ig_aa,
+                '-r', self.repo_ig_aa,
+                '--partial',
+                '--outname', os.path.join(temp_dir, 'partial')
+            ]
+            subprocess.run(partial_cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+            # Output files may not exist if there are no records for that category
+            for f in [strict_pass, strict_fail, partial_pass, partial_fail]:
+                if os.path.exists(f):
+                    self.assertTrue(os.path.isfile(f))
+            def count_records(filepath):
+                if os.path.isfile(filepath) and os.path.getsize(filepath) > 0:
+                    return sum(1 for line in open(filepath) if not line.startswith('#'))
+                return 0
+            strict_pass_count = count_records(strict_pass)
+            strict_fail_count = count_records(strict_fail)
+            partial_pass_count = count_records(partial_pass)
+            partial_fail_count = count_records(partial_fail)
+            self.assertGreaterEqual(partial_pass_count + partial_fail_count,
+                                 strict_pass_count + strict_fail_count,
+                                 "Total records should be the same or greater with partial validation")
+            if strict_fail_count > 0:
+                self.assertTrue(
+                    partial_pass_count > strict_pass_count or partial_fail_count < strict_fail_count,
+                    "Partial validation should pass more records or fail fewer records than strict validation"
+                )
+        finally:
+            shutil.rmtree(temp_dir)
+
+
+class Test_Gzip(unittest.TestCase):
+    def setUp(self):
+        self.temp_dir = tempfile.mkdtemp()
+
+    def tearDown(self):
+        shutil.rmtree(self.temp_dir)
+
+    def test_openFile_writes_real_gzip(self):
+        """openFile('w') on a .gz name produces an actual gzip file, verified
+        independently of openFile itself (via the stdlib gzip module and the
+        gzip magic bytes), not just round-tripped through the same function."""
+        file_name = os.path.join(self.temp_dir, 'records.tsv.gz')
+        content = 'sequence_id\tproductive\nseq1\tT\n'
+
+        with openFile(file_name, 'w') as handle:
+            handle.write(content)
+
+        with open(file_name, 'rb') as handle:
+            self.assertEqual(handle.read(2), b'\x1f\x8b')  # gzip magic bytes
+        with gzip.open(file_name, 'rt') as handle:
+            self.assertEqual(handle.read(), content)
+
+    def test_openFile_reads_real_gzip(self):
+        """openFile('r') on a .gz name transparently decompresses a file
+        written independently by the stdlib gzip module."""
+        file_name = os.path.join(self.temp_dir, 'records.tsv.gz')
+        content = 'sequence_id\tproductive\nseq1\tT\n'
+
+        with gzip.open(file_name, 'wt') as handle:
+            handle.write(content)
+
+        with openFile(file_name, 'r') as handle:
+            self.assertEqual(handle.read(), content)
+
+    def test_gzip_output_cannot_overwrite_input(self):
+        """The final gzip output name may not resolve to the input file."""
+        input_file = os.path.join(self.temp_dir, 'records.tsv.gz')
+        with gzip.open(input_file, 'wt') as handle:
+            handle.write('sequence_id\nseq1\n')
+
+        parser = getCommonArgParser()
+        args = parser.parse_args(['-d', input_file, '-o', input_file[:-3], '--gzip-output'])
+        with self.assertRaises(SystemExit):
+            parseCommonArgs(args)
+
+    def test_gzip_output_cannot_overwrite_input_via_path_alias(self):
+        """A -o that is only path-equivalent to the input (e.g. via a redundant
+        './' segment, as in -d ./input.tsv.gz -o input.tsv) must still be caught,
+        even though the two names are not string-identical."""
+        input_file = os.path.join(self.temp_dir, 'records.tsv.gz')
+        with gzip.open(input_file, 'wt') as handle:
+            handle.write('sequence_id\nseq1\n')
+
+        alias_out = os.path.join(self.temp_dir, '.', 'records.tsv')
+        self.assertNotEqual(alias_out, input_file[:-3])
+
+        parser = getCommonArgParser()
+        args = parser.parse_args(['-d', input_file, '-o', alias_out, '--gzip-output'])
+        with self.assertRaises(SystemExit):
+            parseCommonArgs(args)
+
+
+if __name__ == '__main__':
+    unittest.main()


=====================================
tests/test_ParseDb.py
=====================================
@@ -0,0 +1,171 @@
+"""
+Unit tests for ParseDb
+"""
+
+# Info
+__author__ = 'Jason Anthony Vander Heiden'
+from changeo import __version__, __date__
+
+# Imports
+import csv
+import gzip
+import io
+import os
+import shutil
+import subprocess
+import sys
+import tempfile
+import time
+import unittest
+from contextlib import redirect_stderr
+
+# Paths
+test_path = os.path.dirname(os.path.realpath(__file__))
+data_path = os.path.join(test_path, 'data')
+bin_path = os.path.join(test_path, os.pardir, 'bin')
+
+# Import script
+sys.path.append(bin_path)
+import ParseDb
+
+
+class Test_ParseDb(unittest.TestCase):
+    def setUp(self):
+        print('-> %s()' % self._testMethodName)
+        self.start = time.time()
+        self.tmp_dir = tempfile.mkdtemp()
+
+    def tearDown(self):
+        shutil.rmtree(self.tmp_dir)
+        t = time.time() - self.start
+        print("<- %s() %.3f" % (self._testMethodName, t))
+
+    def _write_tsv(self, name, rows):
+        path = os.path.join(self.tmp_dir, name)
+        with open(path, 'w', newline='') as f:
+            writer = csv.DictWriter(f, fieldnames=list(rows[0].keys()), delimiter='\t')
+            writer.writeheader()
+            writer.writerows(rows)
+        return path
+
+    def _read_tsv(self, path):
+        with open(path, 'r', newline='') as f:
+            return list(csv.DictReader(f, delimiter='\t'))
+
+    # @unittest.skip("-> mergeDbFiles() skipped\n")
+    def test_mergeDbFiles(self):
+        file1 = self._write_tsv('sample1.tsv', [{'SEQUENCE_ID': 'A1', 'V_CALL': 'IGHV1'},
+                                                 {'SEQUENCE_ID': 'A2', 'V_CALL': 'IGHV2'}])
+        file2 = self._write_tsv('sample2.tsv', [{'SEQUENCE_ID': 'B1', 'V_CALL': 'IGHV3'}])
+        out_file = os.path.join(self.tmp_dir, 'merged.tsv')
+
+        # Merge without sample annotation
+        ParseDb.mergeDbFiles([file1, file2], out_file=out_file)
+        records = self._read_tsv(out_file)
+        self.assertEqual(len(records), 3)
+        self.assertNotIn('SAMPLE', records[0])
+
+        # Merge with sample annotation
+        ParseDb.mergeDbFiles([file1, file2], field='SAMPLE', values=['sample1', 'sample2'],
+                              out_file=out_file)
+        records = self._read_tsv(out_file)
+        results = {r['SEQUENCE_ID']: r['SAMPLE'] for r in records}
+        self.assertDictEqual(results, {'A1': 'sample1', 'A2': 'sample1', 'B1': 'sample2'})
+
+    def test_mergeDbFiles_values_without_field(self):
+        file1 = self._write_tsv('sample1.tsv', [{'SEQUENCE_ID': 'A1', 'V_CALL': 'IGHV1'}])
+        file2 = self._write_tsv('sample2.tsv', [{'SEQUENCE_ID': 'B1', 'V_CALL': 'IGHV3'}])
+        out_file = os.path.join(self.tmp_dir, 'merged.tsv')
+
+        # Values without a field are ignored, with a warning
+        stderr = io.StringIO()
+        with redirect_stderr(stderr):
+            ParseDb.mergeDbFiles([file1, file2], values=['sample1', 'sample2'], out_file=out_file)
+        self.assertIn('WARNING', stderr.getvalue())
+
+        records = self._read_tsv(out_file)
+        self.assertEqual(len(records), 2)
+        self.assertNotIn('SAMPLE', records[0])
+
+    def test_mergeDbFiles_field_overwrites_existing_column(self):
+        file1 = self._write_tsv('sample1.tsv', [{'SEQUENCE_ID': 'A1', 'SAMPLE': 'original'}])
+        file2 = self._write_tsv('sample2.tsv', [{'SEQUENCE_ID': 'B1', 'SAMPLE': 'original'}])
+        out_file = os.path.join(self.tmp_dir, 'merged.tsv')
+
+        # Reusing an existing column name overwrites its values, with a warning
+        stderr = io.StringIO()
+        with redirect_stderr(stderr):
+            ParseDb.mergeDbFiles([file1, file2], field='SAMPLE', values=['sample1', 'sample2'],
+                                  out_file=out_file)
+        self.assertIn('WARNING', stderr.getvalue())
+
+        records = self._read_tsv(out_file)
+        results = {r['SEQUENCE_ID']: r['SAMPLE'] for r in records}
+        self.assertDictEqual(results, {'A1': 'sample1', 'B1': 'sample2'})
+
+    def test_mergeDbFiles_field_overwrites_partial_column_with_drop(self):
+        # Only file1 has a SAMPLE column; with drop=True it is excluded from
+        # out_fields before the annotation field is considered, so the warning
+        # must be based on the input headers rather than the post-drop output.
+        file1 = self._write_tsv('sample1.tsv', [{'SEQUENCE_ID': 'A1', 'SAMPLE': 'original'}])
+        file2 = self._write_tsv('sample2.tsv', [{'SEQUENCE_ID': 'B1', 'V_CALL': 'IGHV1'}])
+        out_file = os.path.join(self.tmp_dir, 'merged.tsv')
+
+        stderr = io.StringIO()
+        with redirect_stderr(stderr):
+            ParseDb.mergeDbFiles([file1, file2], drop=True, field='SAMPLE',
+                                  values=['sample1', 'sample2'], out_file=out_file)
+        self.assertIn('WARNING', stderr.getvalue())
+
+        records = self._read_tsv(out_file)
+        results = {r['SEQUENCE_ID']: r['SAMPLE'] for r in records}
+        self.assertDictEqual(results, {'A1': 'sample1', 'B1': 'sample2'})
+
+class Test_ParseDbGzip(unittest.TestCase):
+    """Tests for compressed input and output."""
+    def setUp(self):
+        self.temp_dir = tempfile.mkdtemp()
+        source_file = os.path.join(data_path, 'imgt_ig_db-pass.tsv')
+        self.plain_file = os.path.join(self.temp_dir, 'input.tsv')
+        self.gzip_file = '%s.gz' % self.plain_file
+        shutil.copyfile(source_file, self.plain_file)
+        with open(self.plain_file, 'rb') as source, gzip.open(self.gzip_file, 'wb') as target:
+            shutil.copyfileobj(source, target)
+
+        with open(self.plain_file, 'r') as handle:
+            self.expected_lines = handle.read().splitlines()[:2]
+        self.sequence_id = self.expected_lines[1].split('\t')[0]
+
+    def tearDown(self):
+        shutil.rmtree(self.temp_dir)
+
+    def _select(self, input_file, gzip_output=False):
+        command = [sys.executable, os.path.join(bin_path, 'ParseDb.py'), 'select',
+                   '-d', input_file, '-f', 'SEQUENCE_ID', '-u', self.sequence_id,
+                   '--outdir', self.temp_dir]
+        if gzip_output:
+            command.append('--gzip-output')
+        result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+        self.assertEqual(result.returncode, 0, result.stdout.decode('utf-8'))
+
+        input_names = {os.path.basename(self.plain_file), os.path.basename(self.gzip_file)}
+        output_names = [name for name in os.listdir(self.temp_dir) if name not in input_names]
+        self.assertEqual(len(output_names), 1)
+        return os.path.join(self.temp_dir, output_names[0])
+
+    def _assert_selected_gzip_output(self, output_file):
+        self.assertTrue(output_file.endswith('.gz'))
+        with gzip.open(output_file, 'rt') as handle:
+            self.assertEqual(handle.read().splitlines(), self.expected_lines)
+
+    def test_compressed_input_preserves_compressed_output(self):
+        output_file = self._select(self.gzip_file)
+        self._assert_selected_gzip_output(output_file)
+
+    def test_gzip_output_compresses_plain_input_result(self):
+        output_file = self._select(self.plain_file, gzip_output=True)
+        self._assert_selected_gzip_output(output_file)
+
+
+if __name__ == '__main__':
+    unittest.main()


=====================================
tests/test_Receptor.py
=====================================
@@ -0,0 +1,220 @@
+"""
+Unit tests for MakeDb
+"""
+
+# Imports
+import os
+import time
+import unittest
+
+# Presto and changeo imports
+from changeo.Receptor import Receptor
+from changeo.Alignment import decodeBTOP, decodeCIGAR, encodeCIGAR, padAlignment, alignmentPositions
+
+# Paths
+test_path = os.path.dirname(os.path.realpath(__file__))
+data_path = os.path.join(test_path, 'data')
+
+
+class Test_Receptor(unittest.TestCase):
+    def setUp(self):
+        print('-> %s()' % self._testMethodName)
+
+        row = {'SEQUENCE_ID':    'TEST1',
+               'SEQUENCE_INPUT': 'TGTGCAAGGGGGCCATTGGACTACTTCTACTACGGTGTGGACGNNNNN',
+               'V_CALL':         'IGHV6-1*01,IGHV6-1*02',
+               'D_CALL':         'IGHD6-6*01',
+               'J_CALL':         'IGHJ6*02',}
+        self.ig_rec = Receptor(row)
+
+        row = {'SEQUENCE_ID':    'TEST2',
+               'SEQUENCE_INPUT': 'TGTGCAAGGGGGCCATTGGACTACTTCTACTACGGTGTGGACGNNNNN',
+               'V_CALL':         'TRAV6-1*01,TRAV6-1*02',
+               'D_CALL':         'TRAD6-6*01',
+               'J_CALL':         'TRAJ6*02',}
+        self.tr_rec = Receptor(row)
+
+        row = {'SEQUENCE_ID':    'TEST3',
+               'SEQUENCE_INPUT': '',
+               'V_CALL':         '',
+               'D_CALL':         None,
+               'J_CALL':         'princess,banana,hammock',}
+        self.bad_rec = Receptor(row)
+
+        # CIGAR strings
+        self.cigar_string = ['30M1I69M3D', '4S16N30M1I69M3D']
+        self.cigar_decoded = [[('M', 30), ('I', 1), ('M', 69), ('D', 3)],
+                              [('S', 4), ('N', 16), ('M', 30), ('I', 1), ('M', 69), ('D', 3)]]
+        self.cigar_len = [{'q_start': 0, 'q_length': 30 + 1 + 69 + 0, 'r_start': 0, 'r_length': 30 + 0 + 69 + 3},
+                          {'q_start': 4, 'q_length': 30 + 1 + 69 + 0, 'r_start': 16, 'r_length': 30 + 0 + 69 + 3}]
+
+        self.cigar_pos = [(0, 0), (5, 0), (0, 3), (5, 3)]
+        self.cigar_pad_1 = [[('M', 30), ('I', 1), ('M', 69), ('D', 3)],
+                            [('S', 5), ('M', 30), ('I', 1), ('M', 69), ('D', 3)],
+                            [('N', 3), ('M', 30), ('I', 1), ('M', 69), ('D', 3)],
+                            [('S', 5), ('N', 3), ('M', 30), ('I', 1), ('M', 69), ('D', 3)]]
+        self.cigar_pad_2 = [[('S', 4), ('N', 16), ('M', 30), ('I', 1), ('M', 69), ('D', 3)],
+                            [('S', 9), ('N', 16), ('M', 30), ('I', 1), ('M', 69), ('D', 3)],
+                            [('S', 4), ('N', 19), ('M', 30), ('I', 1), ('M', 69), ('D', 3)],
+                            [('S', 9), ('N', 19), ('M', 30), ('I', 1), ('M', 69), ('D', 3)]]
+
+        # BTOP strings
+        self.btop_string = ['7AGAC39',
+                            '7A-39',
+                            '6-G-A41',
+                            'AG8-GC-CTCT']
+        self.btop_decoded_full = [[('=', 7), ('X', 2), ('=', 39)],
+                                  [('=', 7), ('D', 1), ('=', 39)],
+                                  [('=', 6), ('I', 2), ('=', 41)],
+                                  [('X', 1), ('=', 8), ('I', 1), ('D', 1), ('X', 2)]]
+
+        self.start = time.time()
+
+    def tearDown(self):
+        t = time.time() - self.start
+        print('<- %s() %.3f' % (self._testMethodName, t))
+
+    @unittest.skip("-> Receptor() skipped\n")
+    def test_Receptor(self):
+        print('IG>')
+        print(self.ig_rec.getAlleleCalls(['v','d','j'], action='first'))
+        print(self.ig_rec.getAlleleCalls(['v','j'], action='first'))
+        print(self.ig_rec.getAlleleCalls(['j','v'], action='first'))
+        print(self.ig_rec.getGeneCalls(['v','d','j'], action='first'))
+        print(self.ig_rec.getGeneCalls(['v','j'], action='first'))
+        print(self.ig_rec.getGeneCalls(['j','v'], action='first'))
+        print(self.ig_rec.getFamilyCalls(['v','d','j'], action='first'))
+        print(self.ig_rec.getFamilyCalls(['v','j'], action='first'))
+        print(self.ig_rec.getFamilyCalls(['j','v'], action='first'))
+
+        print(self.ig_rec.getAlleleCalls(['v','d','j'], action='set'))
+        print(self.ig_rec.getAlleleCalls(['v','j'], action='set'))
+        print(self.ig_rec.getAlleleCalls(['j','v'], action='set'))
+        print(self.ig_rec.getGeneCalls(['v','d','j'], action='set'))
+        print(self.ig_rec.getGeneCalls(['v','j'], action='set'))
+        print(self.ig_rec.getGeneCalls(['j','v'], action='set'))
+        print(self.ig_rec.getFamilyCalls(['v','d','j'], action='set'))
+        print(self.ig_rec.getFamilyCalls(['v','j'], action='set'))
+        print(self.ig_rec.getFamilyCalls(['j','v'], action='set'))
+
+        print(self.ig_rec.getAlleleCalls(['v','d','j'], action='list'))
+        print(self.ig_rec.getAlleleCalls(['v','j'], action='list'))
+        print(self.ig_rec.getAlleleCalls(['j','v'], action='list'))
+        print(self.ig_rec.getGeneCalls(['v','d','j'], action='list'))
+        print(self.ig_rec.getGeneCalls(['v','j'], action='list'))
+        print(self.ig_rec.getGeneCalls(['j','v'], action='list'))
+        print(self.ig_rec.getFamilyCalls(['v','d','j'], action='list'))
+        print(self.ig_rec.getFamilyCalls(['v','j'], action='list'))
+        print(self.ig_rec.getFamilyCalls(['j','v'], action='list'))
+
+        print('TR>')
+        print(self.tr_rec.getAlleleCalls(['v','d','j'], action='first'))
+        print(self.tr_rec.getAlleleCalls(['v','j'], action='first'))
+        print(self.tr_rec.getAlleleCalls(['j','v'], action='first'))
+        print(self.tr_rec.getGeneCalls(['v','d','j'], action='first'))
+        print(self.tr_rec.getGeneCalls(['v','j'], action='first'))
+        print(self.tr_rec.getGeneCalls(['j','v'], action='first'))
+        print(self.tr_rec.getFamilyCalls(['v','d','j'], action='first'))
+        print(self.tr_rec.getFamilyCalls(['v','j'], action='first'))
+        print(self.tr_rec.getFamilyCalls(['j','v'], action='first'))
+
+        print(self.tr_rec.getAlleleCalls(['v','d','j'], action='set'))
+        print(self.tr_rec.getAlleleCalls(['v','j'], action='set'))
+        print(self.tr_rec.getAlleleCalls(['j','v'], action='set'))
+        print(self.tr_rec.getGeneCalls(['v','d','j'], action='set'))
+        print(self.tr_rec.getGeneCalls(['v','j'], action='set'))
+        print(self.tr_rec.getGeneCalls(['j','v'], action='set'))
+        print(self.tr_rec.getFamilyCalls(['v','d','j'], action='set'))
+        print(self.tr_rec.getFamilyCalls(['v','j'], action='set'))
+        print(self.tr_rec.getFamilyCalls(['j','v'], action='set'))
+
+        print(self.tr_rec.getAlleleCalls(['v','d','j'], action='list'))
+        print(self.tr_rec.getAlleleCalls(['v','j'], action='list'))
+        print(self.tr_rec.getAlleleCalls(['j','v'], action='list'))
+        print(self.tr_rec.getGeneCalls(['v','d','j'], action='list'))
+        print(self.tr_rec.getGeneCalls(['v','j'], action='list'))
+        print(self.tr_rec.getGeneCalls(['j','v'], action='list'))
+        print(self.tr_rec.getFamilyCalls(['v','d','j'], action='list'))
+        print(self.tr_rec.getFamilyCalls(['v','j'], action='list'))
+        print(self.tr_rec.getFamilyCalls(['j','v'], action='list'))
+
+        print('JUNK>')
+        print(self.bad_rec.getAlleleCalls(['v','d','j'], action='first'))
+        print(self.bad_rec.getAlleleCalls(['v','j'], action='first'))
+        print(self.bad_rec.getAlleleCalls(['j','v'], action='first'))
+        print(self.bad_rec.getGeneCalls(['v','d','j'], action='first'))
+        print(self.bad_rec.getGeneCalls(['v','j'], action='first'))
+        print(self.bad_rec.getGeneCalls(['j','v'], action='first'))
+        print(self.bad_rec.getFamilyCalls(['v','d','j'], action='first'))
+        print(self.bad_rec.getFamilyCalls(['v','j'], action='first'))
+        print(self.bad_rec.getFamilyCalls(['j','v'], action='first'))
+
+        print(self.bad_rec.getAlleleCalls(['v','d','j'], action='set'))
+        print(self.bad_rec.getAlleleCalls(['v','j'], action='set'))
+        print(self.bad_rec.getAlleleCalls(['j','v'], action='set'))
+        print(self.bad_rec.getGeneCalls(['v','d','j'], action='set'))
+        print(self.bad_rec.getGeneCalls(['v','j'], action='set'))
+        print(self.bad_rec.getGeneCalls(['j','v'], action='set'))
+        print(self.bad_rec.getFamilyCalls(['v','d','j'], action='set'))
+        print(self.bad_rec.getFamilyCalls(['v','j'], action='set'))
+        print(self.bad_rec.getFamilyCalls(['j','v'], action='set'))
+
+        print(self.bad_rec.getAlleleCalls(['v','d','j'], action='list'))
+        print(self.bad_rec.getAlleleCalls(['v','j'], action='list'))
+        print(self.bad_rec.getAlleleCalls(['j','v'], action='list'))
+        print(self.bad_rec.getGeneCalls(['v','d','j'], action='list'))
+        print(self.bad_rec.getGeneCalls(['v','j'], action='list'))
+        print(self.bad_rec.getGeneCalls(['j','v'], action='list'))
+        print(self.bad_rec.getFamilyCalls(['v','d','j'], action='list'))
+        print(self.bad_rec.getFamilyCalls(['v','j'], action='list'))
+        print(self.bad_rec.getFamilyCalls(['j','v'], action='list'))
+
+        self.fail('TODO')
+
+    #@unittest.skip("-> decodeCIGAR() skipped\n")
+    def test_decodeCIGAR(self):
+        for cigar, truth in zip(self.cigar_string, self.cigar_decoded):
+            result = decodeCIGAR(cigar)
+            print(result)
+            self.assertListEqual(truth, result)
+
+    #@unittest.skip("-> decodeBTOP() skipped\n")
+    def test_decodeBTOP(self):
+        for btop, truth in zip(self.btop_string, self.btop_decoded_full):
+            result = decodeBTOP(btop)
+            print('FULL> ', result)
+            self.assertListEqual(truth, result)
+
+    #@unittest.skip("-> encodeCIGAR() skipped\n")
+    def test_encodeCIGAR(self):
+        for align, truth in zip(self.cigar_decoded, self.cigar_string):
+            result = encodeCIGAR(align)
+            print(result)
+            self.assertEqual(truth, result)
+
+    #@unittest.skip("-> padAlignment() skipped\n")
+    def test_padAlignment(self):
+        cigar = self.cigar_decoded[0]
+        for (s, r), truth in zip(self.cigar_pos, self.cigar_pad_1):
+            #print('POS>', s, r)
+            result = padAlignment(cigar, s, r)
+            print('PAD>', '(%i, %i) =' % (s, r), result)
+            self.assertEqual(truth, result)
+
+        cigar = self.cigar_decoded[1]
+        for (s, r), truth in zip(self.cigar_pos, self.cigar_pad_2):
+            #print('POS>', s, r)
+            result = padAlignment(cigar, s, r)
+            print('PAD>', '(%i, %i) =' % (s, r), result)
+            self.assertEqual(truth, result)
+
+    #@unittest.skip("-> alignmentPositions() skipped\n")
+    def test_alignmentPositions(self):
+        for align, truth in zip(self.cigar_decoded, self.cigar_len):
+            result = alignmentPositions(align)
+            print('POS>', result)
+            self.assertDictEqual(truth, result)
+
+
+if __name__ == '__main__':
+    unittest.main()
\ No newline at end of file



View it on GitLab: https://salsa.debian.org/med-team/changeo/-/commit/421d53889231d8172b75c692ba7dbe41bd215704

-- 
View it on GitLab: https://salsa.debian.org/med-team/changeo/-/commit/421d53889231d8172b75c692ba7dbe41bd215704
You're receiving this email because of your account on salsa.debian.org. Manage all notifications: https://salsa.debian.org/-/profile/notifications | Help: https://salsa.debian.org/help


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/debian-med-commit/attachments/20261115/33492ffe/attachment-0001.htm>


More information about the debian-med-commit mailing list