[med-svn] [Git][med-team/python-pysam][ariba] Replace faidx-stdout.patch with the upstream fix

Andreas Tille (@tille) gitlab at salsa.debian.org
Wed Nov 4 10:07:59 GMT 2026



Andreas Tille pushed to branch ariba at Debian Med / python-pysam


Commits:
b1e79724 by Andreas Tille at 2026-11-04T10:44:08+01:00
Replace faidx-stdout.patch with the upstream fix

The previous version worked around the empty pysam.faidx() result by
injecting a synthetic -o option, which needed a heuristic to decide
whether a faidx invocation writes sequences at all, and which clobbered
a -o option given by the caller.

Upstream fixed the cause in commit 6261249228: faidx now uses
bgzf_open(samtools_stdout_fn, ...) like calmd, so pysam's stdout capture
works again, the synthetic -o option is prepended rather than appended so
that the caller's -o wins, and the obsolete "view -c" special case is
dropped.  Take that commit as a patch and enable it, since it is now a
proper upstream backport rather than an unreviewed workaround.

- - - - -


3 changed files:

- debian/changelog
- debian/patches/faidx-stdout.patch
- debian/patches/series


Changes:

=====================================
debian/changelog
=====================================
@@ -1,5 +1,10 @@
 python-pysam (0.24.1+ds-3) UNRELEASED; urgency=medium
 
+  * New patch faidx-stdout.patch, taken from upstream: fix pysam.faidx()
+    returning an empty string with the bundled samtools 1.22 and later,
+    which write their output with bgzf_open() instead of through the
+    replaced stdout FILE*.  This broke faidx sequence retrieval in ariba
+    (see https://github.com/sanger-pathogens/ariba/issues/349).
   * skip-32bit-tests.patch: skip the 64-bit object-size checks
     and the PileupColumn access-after-iterator test on 32-bit platforms.
     Closes: #1152973


=====================================
debian/patches/faidx-stdout.patch
=====================================
@@ -1,151 +1,130 @@
-Description: Fix pysam.faidx() returning an empty string with samtools >= 1.22
- Since the bundled samtools was updated to 1.22 in pysam 0.24.0, the faidx
- subcommand writes its output via bgzf_open("-") instead of the patched
- stdout FILE*, so the sequence fetched with pysam.faidx(fa, region) was no
- longer captured and the call returned an empty string while the sequence
- leaked to the process' real stdout.
- Capture the output with -o for faidx, like the other samtools commands
- that support it, and route save_stdout requests the same way.  Index
- building invocations (no regions) write nothing to stdout, so for those
- the -o option is deliberately not added to avoid redirecting the index
- file into the capture file.
- Add regression tests for the returned wrapped sequence, multiple
- regions, and index-only invocations.
-Author: Andreas Tille <tille at debian.org>
-Assisted-By: https://inference.debian.net/ Model DeepSeek V4 Flash 0731
-Forwarded: no
-Last-Update: 2026-09-22
-Comment: Needs review, not applied yet
+Description: Fix pysam.samtools.faidx() output capture with samtools >= 1.22
+ Since samtools 1.22 the bundled faidx writes its output, like calmd, via
+ bgzf_open() to standard output rather than through the stdout FILE* that
+ pysam replaces, so pysam.faidx(fa, region) returned an empty string and
+ the fetched sequence leaked to the process' real stdout.  This broke
+ ariba (sanger-pathogens/ariba#349).
+ Redirect bgzf_open("-") through samtools_stdout_fn the same way calmd
+ does, and map faidx to the pysam_stdout_fn capture mechanism instead of
+ a synthetic -o option.
+ Also prepend rather than append the synthetic -o option, so that a
+ user-supplied -o takes precedence instead of being clobbered, and drop
+ the special case for "view -c", which has printed to stdout rather than
+ through sam_open_format() since samtools 1.4.
+Origin: upstream, https://github.com/pysam-developers/pysam/commit/6261249228
+Bug: https://github.com/sanger-pathogens/ariba/issues/349
+Forwarded: not-needed
+Last-Update: 2026-11-04
 
+diff --git a/devtools/import.py b/devtools/import.py
+index 2f41163..4e83bff 100644
+--- a/devtools/import.py
++++ b/devtools/import.py
+@@ -107,6 +107,9 @@ def _update_pysam_files(cf, destdir):
+                     "bam_md.c": (
+                         'sam_open_format("-", mode_w',
+                         'sam_open_format({}_stdout_fn, mode_w'.format(basename)),
++                    "faidx.c": (
++                        'bgzf_open("-", "wu"',
++                        'bgzf_open({}_stdout_fn, "wu"'.format(basename)),
+                     "phase.c": (
+                         'putc("ACGT"[f->seq[j] == 1? (c&3, {}_stdout) : (c>>16&3)]);'.format(basename),
+                         'putc("ACGT"[f->seq[j] == 1? (c&3) : (c>>16&3)], {}_stdout);'.format(basename)),
+diff --git a/pysam/libcutils.pyx b/pysam/libcutils.pyx
+index f8c13aa..cf6cd3c 100644
 --- a/pysam/libcutils.pyx
 +++ b/pysam/libcutils.pyx
-@@ -300,6 +300,57 @@ cdef int libc_whence_from_io(int whence):
-     return whence  # Otherwise likely invalid, but let HTSlib or OS report it
- 
- 
-+# ``samtools faidx`` options that consume the following argument as
-+# their value.  Used to count positional arguments (input file plus
-+# regions) when deciding whether a faidx call has output to capture.
-+_FAIDX_VALUE_OPTIONS = frozenset((
-+    "-o", "-n", "-r", "-@",
-+    "--output", "--length", "--region-file", "--fai-idx", "--gzi-idx",
-+    "--mark-strand", "--input-fmt", "--input-fmt-option", "--output-fmt",
-+    "--output-fmt-option", "--reference", "--threads", "--verbosity",
-+))
-+
-+
-+def _faidx_fetches_to_stdout(args):
-+    """Return True if a ``samtools faidx`` invocation writes sequences to
-+    stdout, i.e. it fetches regions and does not pass an explicit -o.
-+
-+    faidx builds the index when given a single positional argument (the
-+    input file) and no regions; such calls write nothing to stdout.
-+    Fetching sequences is done by giving regions as further positional
-+    arguments or via -r/--region-file.  Only in these cases is output
-+    produced, and only if the caller did not already provide -o.
-+    """
-+    n_positional = 0
-+    skip_next = False
-+    has_output = False
-+    has_regions = False
-+    for arg in args:
-+        if skip_next:
-+            skip_next = False
-+            continue
-+        if arg == "-o" or arg.startswith("-o"):
-+            has_output = True
-+            skip_next = True
-+            continue
-+        if arg == "--output" or arg.startswith("--output="):
-+            has_output = True
-+            continue
-+        if arg == "-r" or arg.startswith("-r"):
-+            has_regions = True
-+            skip_next = True
-+            continue
-+        if arg == "--region-file" or arg.startswith("--region-file="):
-+            has_regions = True
-+            continue
-+        if arg.startswith("-") and arg != "-":
-+            if arg in _FAIDX_VALUE_OPTIONS:
-+                skip_next = True
-+            continue
-+        n_positional += 1
-+    return not has_output and (has_regions or n_positional > 1)
-+
-+
- def _pysam_dispatch(collection,
-                     method,
-                     args=None,
-@@ -352,6 +403,12 @@ def _pysam_dispatch(collection,
-         stdout_h = c_open(stdout_f_bytes, O_WRONLY|O_CREAT|O_TRUNC, 0o666)
-         if stdout_h == -1:
-             raise OSError_from_errno("Could not redirect standard output", stdout_f)
-+        # samtools faidx writes via bgzf_open("-") to the real stdout
-+        # rather than the patched stdout FILE*, so route it to the
-+        # requested file with -o instead
-+        if collection == "samtools" and method == "faidx" \
-+                and _faidx_fetches_to_stdout(args):
-+            args.extend(["-o", stdout_f])
- 
-     elif catch_stdout:
-         stdout_h, stdout_f = tempfile.mkstemp()
-@@ -361,6 +418,9 @@ def _pysam_dispatch(collection,
+@@ -361,6 +361,7 @@ def _pysam_dispatch(collection,
              "mpileup": "-o {}",
              "depad": "-o {}",
              "calmd": "",  # uses pysam_stdout_fn
-+            # since samtools 1.22, faidx writes via bgzf_open("-") rather
-+            # than the patched stdout FILE*, so capture with -o instead
-+            "faidx": "-o {}",
++            "faidx": "",  # uses pysam_stdout_fn
          },
              "bcftools": {}
          }
-@@ -375,6 +435,11 @@ def _pysam_dispatch(collection,
-             # special case - samtools view -c outputs on stdout
-             if not(method == "view" and "-c" in args):
-                 stdout_option = MAP_STDOUT_OPTIONS[collection][method]
-+            # special case - samtools faidx only produces output when
-+            # fetching regions; when it merely builds the index, or when
-+            # the caller already passes -o, nothing should be captured
-+            if method == "faidx" and not _faidx_fetches_to_stdout(args):
-+                stdout_option = None
+@@ -372,14 +373,12 @@ def _pysam_dispatch(collection,
+             if method not in ("head", "index", "roh", "stats"):
+                 stdout_option = "-o {}"
+         elif method in MAP_STDOUT_OPTIONS[collection]:
+-            # special case - samtools view -c outputs on stdout
+-            if not(method == "view" and "-c" in args):
+-                stdout_option = MAP_STDOUT_OPTIONS[collection][method]
++            stdout_option = MAP_STDOUT_OPTIONS[collection][method]
  
          if stdout_option is not None and not is_usage:
              os.close(stdout_h)
---- a/tests/test_samtools_python.py
-+++ b/tests/test_samtools_python.py
-@@ -48,3 +48,34 @@ def test_bedcov():
-         assert len(fields) in [4, 5], \
-             ("bedcov should give tab delimited output with 4 or 5 fields. "
-              "Split line (%s) gives %d fields." % (fields, len(fields)))
+             stdout_f_bytes = force_bytes(stdout_f)
+-            args.extend(stdout_option.format(stdout_f).split(" "))
++            args[0:0] = stdout_option.format(stdout_f).split(maxsplit=1)
+             stdout_h = c_open(b"/dev/null", O_WRONLY)
+         else:
+             stdout_f_bytes = None
+diff --git a/samtools/faidx.c.pysam.c b/samtools/faidx.c.pysam.c
+index 014d7b6..08c6948 100644
+--- a/samtools/faidx.c.pysam.c
++++ b/samtools/faidx.c.pysam.c
+@@ -613,7 +613,7 @@ int faidx_core(int argc, char *argv[], enum fai_format_options format)
+             }
+         }
+     } else {
+-        if (!(out.bgzf_fp = bgzf_open("-", "wu"))) {
++        if (!(out.bgzf_fp = bgzf_open(samtools_stdout_fn, "wu"))) {
+             fprintf(samtools_stderr,"[faidx] Cannot open output for writing :%s.\n", strerror(errno) );
+             goto exit2;
+         }
+diff --git a/tests/samtools_test.py b/tests/samtools_test.py
+index 050ac27..cb99fd6 100644
+--- a/tests/samtools_test.py
++++ b/tests/samtools_test.py
+@@ -1,4 +1,5 @@
+ import warnings
++import json
+ import os
+ import re
+ import glob
+@@ -329,6 +330,45 @@ class TestStdout:
+         assert len(r) > 0
+ 
+ 
++class TestSamtoolsCapture:
++    """Test that each subcommand's standard output is correctly captured.
++    """
++    @pytest.mark.skipif(pysam.__version__ < "0.24.0", reason="calmd redirection broken")
++    def test_calmd(self):
++        text = pysam.samtools.calmd("-Q", os.path.join(BAM_DATADIR, "ex1.bam"), os.path.join(BAM_DATADIR, "ex1.fa"))
++        assert "EAS56_57:6:190:289:82" in text
 +
++    def test_depad(self):
++        text = pysam.samtools.depad("-s", "-T", os.path.join(BAM_DATADIR, "ex1.fa"), os.path.join(BAM_DATADIR, "ex5.bam"))
++        assert "read_28833_29006_6945" in text
 +
-+def _faidx_filename():
-+    return os.path.join(BAM_DATADIR, "ex1.fa")
++    def test_faidx(self):
++        seq = pysam.samtools.faidx(os.path.join(BAM_DATADIR, "ex1.fa"), "chr2:101-120")
++        assert seq == ">chr2:101-120\nGAACTTGAAGACAAGTCTCT\n"
 +
++    def test_flags(self):
++        text = pysam.samtools.flags("16")
++        assert text == "0x10\t16\tREVERSE\n"
 +
-+def test_faidx_fetch_returns_sequence():
-+    # Regression test: since pysam 0.24.0 (bundled samtools 1.22) faidx
-+    # writes via bgzf_open() instead of the patched stdout FILE*, so the
-+    # fetched sequence must be captured with -o. Before the fix this
-+    # returned an empty string.
-+    seq = pysam.faidx(_faidx_filename(), "chr1")
-+    assert seq.startswith(">chr1\n")
-+    lines = seq.splitlines()
-+    # samtools faidx line-wraps at 60 characters
-+    assert len(lines[1]) == 60
-+    assert all(len(l) == 60 for l in lines[1:-1])
-+    with pysam.FastaFile(_faidx_filename()) as fasta:
-+        assert "".join(lines[1:]) == fasta.fetch("chr1")
++    def test_flagstat(self):
++        stats = json.loads(pysam.samtools.flagstat("-Ojson", os.path.join(BAM_DATADIR, "ex1.bam")))
++        assert stats["QC-passed reads"]["properly paired"] == 3124
 +
++    def test_mpileup(self):
++        text = pysam.samtools.mpileup("--region", "chr2:500-501", os.path.join(BAM_DATADIR, "ex1.bam"))
++        assert "chr2\t500\tN\t44" in text
 +
-+def test_faidx_fetch_multiple_regions():
-+    out = pysam.faidx(_faidx_filename(), "chr2", "chr1")
-+    assert [l for l in out.splitlines() if l.startswith(">")] == [">chr2", ">chr1"]
++    def test_view(self):
++        line = pysam.samtools.view(os.path.join(BAM_DATADIR, "ex1.bam"), "chr2:500-501", split_lines=True)
++        assert len(line) == 47
++        assert "EAS1_93:7:319:280:57" in line[0]
++        assert "EAS192_3:6:45:183:25" in line[-1]
 +
++    def test_view_count(self):
++        line = pysam.samtools.view("--count", os.path.join(BAM_DATADIR, "ex1.bam"), "chr2:500-501")
++        assert line.rstrip() == "47"
 +
-+def test_faidx_index_only_returns_empty():
-+    # Building the index writes no sequences to stdout, so nothing should
-+    # be captured (and no stray index file should be left behind).
-+    assert pysam.faidx(_faidx_filename()) == ""
 +
+ class TestPysam(TestSamtools):
+     """check access to samtools command in the pysam
+     main package.


=====================================
debian/patches/series
=====================================
@@ -1,5 +1,4 @@
 # skip_test_remote.patch
 clean_less
-# faidx-stdout.patch # needs human review first
+faidx-stdout.patch
 skip-32bit-tests.patch
-



View it on GitLab: https://salsa.debian.org/med-team/python-pysam/-/commit/b1e797242081171d6ab29b29401437cbf1fa7d4d

-- 
View it on GitLab: https://salsa.debian.org/med-team/python-pysam/-/commit/b1e797242081171d6ab29b29401437cbf1fa7d4d
You're receiving this email because of your account on salsa.debian.org. Manage all notifications: https://salsa.debian.org/-/profile/notifications | Help: https://salsa.debian.org/help


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/debian-med-commit/attachments/20261104/990dcc51/attachment-0001.htm>


More information about the debian-med-commit mailing list