[med-svn] [Git][med-team/heudiconv][upstream] New upstream version 1.1.0
Alexandre Detiste (@detiste-guest)
gitlab at salsa.debian.org
Wed Mar 13 08:58:38 GMT 2024
Alexandre Detiste pushed to branch upstream at Debian Med / heudiconv
Commits:
9abcb508 by Alexandre Detiste at 2024-03-13T09:47:01+01:00
New upstream version 1.1.0
- - - - -
23 changed files:
- LICENSE
- PKG-INFO
- README.rst
- heudiconv.egg-info/PKG-INFO
- heudiconv.egg-info/SOURCES.txt
- heudiconv/_version.py
- heudiconv/bids.py
- heudiconv/convert.py
- heudiconv/dicoms.py
- heudiconv/external/dlad.py
- heudiconv/external/tests/test_dlad.py
- heudiconv/heuristics/convertall.py
- + heudiconv/heuristics/convertall_custom.py
- heudiconv/heuristics/example.py
- heudiconv/heuristics/reproin.py
- heudiconv/heuristics/test_reproin.py
- heudiconv/info.py
- heudiconv/main.py
- heudiconv/parser.py
- heudiconv/tests/test_convert.py
- heudiconv/tests/test_dicoms.py
- heudiconv/utils.py
- pyproject.toml
Changes:
=====================================
LICENSE
=====================================
@@ -1,4 +1,4 @@
-Copyright [2014-2019] [Heudiconv developers]
+Copyright [2014-2024] [HeuDiConv developers]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -11,3 +11,10 @@ Copyright [2014-2019] [Heudiconv developers]
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+
+
+Some parts of the codebase/documentation are borrowed from other sources:
+
+- HeuDiConv tutorial from https://bitbucket.org/dpat/neuroimaging_core_docs/src
+
+ Copyright 2023 Dianne Patterson
=====================================
PKG-INFO
=====================================
@@ -1,25 +1,40 @@
Metadata-Version: 2.1
Name: heudiconv
-Version: 0.13.1
+Version: 1.1.0
Summary: Heuristic DICOM Converter
Author: HeuDiConv team and contributors
License: Apache 2.0
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
-Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering
Classifier: Typing :: Typed
-Requires-Python: >=3.7
+Requires-Python: >=3.8
+License-File: LICENSE
+Requires-Dist: dcmstack>=0.8
+Requires-Dist: etelemetry
+Requires-Dist: filelock>=3.0.12
+Requires-Dist: nibabel
+Requires-Dist: nipype>=1.2.3
+Requires-Dist: pydicom>=1.0.0
Provides-Extra: tests
+Requires-Dist: pytest; extra == "tests"
+Requires-Dist: tinydb; extra == "tests"
+Requires-Dist: inotify; extra == "tests"
Provides-Extra: extras
+Requires-Dist: duecredit; extra == "extras"
Provides-Extra: datalad
+Requires-Dist: datalad>=0.13.0; extra == "datalad"
Provides-Extra: all
-License-File: LICENSE
+Requires-Dist: pytest; extra == "all"
+Requires-Dist: tinydb; extra == "all"
+Requires-Dist: inotify; extra == "all"
+Requires-Dist: duecredit; extra == "all"
+Requires-Dist: datalad>=0.13.0; extra == "all"
Convert DICOM dirs based on heuristic info - HeuDiConv
uses the dcmstack package and dcm2niix tool to convert DICOM directories or
=====================================
README.rst
=====================================
@@ -51,6 +51,10 @@ into structured directory layouts.
- It integrates with `DataLad <https://www.datalad.org/>`_ to place converted and original data under git/git-annex
version control while automatically annotating files with sensitive information (e.g., non-defaced anatomicals, etc).
+Heudiconv can be inserted into your workflow to provide automatic conversion as part of a data acquisition pipeline, as seen in the figure below:
+
+.. image:: figs/environment.png
+
Installation
------------
@@ -60,13 +64,18 @@ on heudiconv.readthedocs.io .
HOWTO 101
---------
-In a nutshell -- ``heudiconv`` operates using a heuristic which, given metadata from DICOMs, would decide how to name
-resultant (from conversion using `dcm2niix`_) files. Heuristic `convertall <https://github
-.com/nipy/heudiconv/blob/master/heudiconv/heuristics/convertall.py>`_ could actually be used with no real
-heuristic and by simply establish your own conversion mapping through editing produced mapping files.
-In most use-cases of retrospective study data conversion, you would need to create your custom heuristic following
-`existing heuristics as examples <https://github.com/nipy/heudiconv/tree/master/heudiconv/heuristics>`_ and/or
-referring to `"Heuristic" section <https://heudiconv.readthedocs.io/en/latest/heuristics.html>`_ in the documentation.
+In a nutshell -- ``heudiconv`` is given a file tree of DICOMs, and it produces a restructured file tree of NifTI files (conversion handled by `dcm2niix`_) with accompanying metadata files.
+The input and output structure is as flexible as your data, which is accomplished by using a Python file called a ``heuristic`` that knows how to read your input structure and decides how to name the resultant files.
+You can run your conversion automatically (which will produce a ``.heudiconv`` directory storing the used parameters), or generate the default parameters, edit them to customize file naming, and continue conversion via an additional invocation of `heudiconv`:
+
+.. image:: figs/workflow.png
+
+
+``heudiconv`` comes with `existing heuristics <https://github.com/nipy/heudiconv/tree/master/heudiconv/heuristics>`_ which can be used as is, or as examples.
+For instance, the Heuristic `convertall <https://github.com/nipy/heudiconv/blob/master/heudiconv/heuristics/convertall.py>`_ extracts standard metadata from all matching DICOMs.
+``heudiconv`` creates mapping files, ``<something>.edit.text`` which lets researchers simply establish their own conversion mapping.
+
+In most use-cases of retrospective study data conversion, you would need to create your custom heuristic following the examples and the `"Heuristic" section <https://heudiconv.readthedocs.io/en/latest/heuristics.html>`_ in the documentation.
**Note** that `ReproIn heuristic <https://github.com/nipy/heudiconv/blob/master/heudiconv/heuristics/reproin.py>`_ is
generic and powerful enough to be adopted virtually for *any* study: For prospective studies, you would just need
to name your sequences following the `ReproIn convention <https://github.com/nipy/heudiconv/blob/master/heudiconv/heuristics/reproin.py#L26>`_, and for
@@ -100,3 +109,23 @@ For a detailed into, see our `contributing guide <CONTRIBUTING.rst>`_.
Our releases are packaged using Intuit auto, with the corresponding workflow including
Docker image preparation being found in ``.github/workflows/release.yml``.
+
+
+3-rd party heuristics
+---------------------
+
+- https://github.com/courtois-neuromod/ds_prep/blob/main/mri/convert/heuristics_unf.py
+
+
+Support
+-------
+
+All bugs, concerns and enhancement requests for this software can be submitted here:
+https://github.com/nipy/heudiconv/issues.
+
+If you have a problem or would like to ask a question about how to use ``heudiconv``,
+please submit a question to `NeuroStars.org <http://neurostars.org/tags/heudiconv>`_ with a ``heudiconv`` tag.
+NeuroStars.org is a platform similar to StackOverflow but dedicated to neuroinformatics.
+
+All previous ``heudiconv`` questions are available here:
+http://neurostars.org/tags/heudiconv/
=====================================
heudiconv.egg-info/PKG-INFO
=====================================
@@ -1,25 +1,40 @@
Metadata-Version: 2.1
Name: heudiconv
-Version: 0.13.1
+Version: 1.1.0
Summary: Heuristic DICOM Converter
Author: HeuDiConv team and contributors
License: Apache 2.0
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
-Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering
Classifier: Typing :: Typed
-Requires-Python: >=3.7
+Requires-Python: >=3.8
+License-File: LICENSE
+Requires-Dist: dcmstack>=0.8
+Requires-Dist: etelemetry
+Requires-Dist: filelock>=3.0.12
+Requires-Dist: nibabel
+Requires-Dist: nipype>=1.2.3
+Requires-Dist: pydicom>=1.0.0
Provides-Extra: tests
+Requires-Dist: pytest; extra == "tests"
+Requires-Dist: tinydb; extra == "tests"
+Requires-Dist: inotify; extra == "tests"
Provides-Extra: extras
+Requires-Dist: duecredit; extra == "extras"
Provides-Extra: datalad
+Requires-Dist: datalad>=0.13.0; extra == "datalad"
Provides-Extra: all
-License-File: LICENSE
+Requires-Dist: pytest; extra == "all"
+Requires-Dist: tinydb; extra == "all"
+Requires-Dist: inotify; extra == "all"
+Requires-Dist: duecredit; extra == "all"
+Requires-Dist: datalad>=0.13.0; extra == "all"
Convert DICOM dirs based on heuristic info - HeuDiConv
uses the dcmstack package and dcm2niix tool to convert DICOM directories or
=====================================
heudiconv.egg-info/SOURCES.txt
=====================================
@@ -34,6 +34,7 @@ heudiconv/heuristics/bids_PhoenixReport.py
heudiconv/heuristics/bids_with_ses.py
heudiconv/heuristics/cmrr_heuristic.py
heudiconv/heuristics/convertall.py
+heudiconv/heuristics/convertall_custom.py
heudiconv/heuristics/example.py
heudiconv/heuristics/multires_7Tbold.py
heudiconv/heuristics/reproin.py
=====================================
heudiconv/_version.py
=====================================
@@ -1 +1 @@
-__version__ = "0.13.1"
+__version__ = "1.1.0"
=====================================
heudiconv/bids.py
=====================================
@@ -68,7 +68,7 @@ class BIDSError(Exception):
pass
-BIDS_VERSION = "1.4.1"
+BIDS_VERSION = "1.8.0"
# List defining allowed parameter matching for fmap assignment:
SHIM_KEY = "ShimSetting"
=====================================
heudiconv/convert.py
=====================================
@@ -15,6 +15,7 @@ from typing import TYPE_CHECKING, Any, List, Optional, cast
import filelock
from nipype import Node
+from nipype.interfaces.base import TraitListObject
from .bids import (
BIDS_VERSION,
@@ -220,6 +221,9 @@ def prep_conversion(
dcmfilter=getattr(heuristic, "filter_dicom", None),
flatten=True,
custom_grouping=getattr(heuristic, "grouping", None),
+ # callable which will be provided dcminfo and returned
+ # structure extend seqinfo
+ custom_seqinfo=getattr(heuristic, "custom_seqinfo", None),
)
elif seqinfo is None:
raise ValueError("Neither 'dicoms' nor 'seqinfo' is given")
@@ -880,19 +884,19 @@ def save_converted_files(
return []
if isdefined(res.outputs.bvecs) and isdefined(res.outputs.bvals):
+ bvals, bvecs = res.outputs.bvals, res.outputs.bvecs
+ bvals = list(bvals) if isinstance(bvals, TraitListObject) else bvals
+ bvecs = list(bvecs) if isinstance(bvecs, TraitListObject) else bvecs
if prefix_dirname.endswith("dwi"):
outname_bvecs, outname_bvals = prefix + ".bvec", prefix + ".bval"
- safe_movefile(res.outputs.bvecs, outname_bvecs, overwrite)
- safe_movefile(res.outputs.bvals, outname_bvals, overwrite)
+ safe_movefile(bvecs, outname_bvecs, overwrite)
+ safe_movefile(bvals, outname_bvals, overwrite)
else:
- if bvals_are_zero(res.outputs.bvals):
- os.remove(res.outputs.bvecs)
- os.remove(res.outputs.bvals)
- lgr.debug(
- "%s and %s were removed since not dwi",
- res.outputs.bvecs,
- res.outputs.bvals,
- )
+ if bvals_are_zero(bvals):
+ to_remove = bvals + bvecs if isinstance(bvals, list) else [bvals, bvecs]
+ for ftr in to_remove:
+ os.remove(ftr)
+ lgr.debug("%s and %s were removed since not dwi", bvecs, bvals)
else:
lgr.warning(
DW_IMAGE_IN_FMAP_FOLDER_WARNING.format(folder=prefix_dirname)
@@ -901,8 +905,8 @@ def save_converted_files(
".bvec and .bval files will be generated. This is NOT BIDS compliant"
)
outname_bvecs, outname_bvals = prefix + ".bvec", prefix + ".bval"
- safe_movefile(res.outputs.bvecs, outname_bvecs, overwrite)
- safe_movefile(res.outputs.bvals, outname_bvals, overwrite)
+ safe_movefile(bvecs, outname_bvecs, overwrite)
+ safe_movefile(bvals, outname_bvals, overwrite)
if isinstance(res_files, list):
res_files = sorted(res_files)
@@ -1064,7 +1068,7 @@ def add_taskname_to_infofile(infofiles: str | list[str]) -> None:
save_json(infofile, meta_info)
-def bvals_are_zero(bval_file: str) -> bool:
+def bvals_are_zero(bval_file: str | list) -> bool:
"""Checks if all entries in a bvals file are zero (or 5, for Siemens files).
Parameters
@@ -1077,6 +1081,10 @@ def bvals_are_zero(bval_file: str) -> bool:
True if all are all 0 or 5; False otherwise.
"""
+ # GE hyperband multi-echo containing diffusion info
+ if isinstance(bval_file, list):
+ return all(map(bvals_are_zero, bval_file))
+
with open(bval_file) as f:
bvals = f.read().split()
=====================================
heudiconv/dicoms.py
=====================================
@@ -9,7 +9,18 @@ import os.path as op
from pathlib import Path
import sys
import tarfile
-from typing import TYPE_CHECKING, Any, Dict, List, NamedTuple, Optional, Union, overload
+from typing import (
+ TYPE_CHECKING,
+ Any,
+ Dict,
+ Hashable,
+ List,
+ NamedTuple,
+ Optional,
+ Protocol,
+ Union,
+ overload,
+)
from unittest.mock import patch
import warnings
@@ -42,7 +53,17 @@ total_files = 0
compresslevel = 9
-def create_seqinfo(mw: dw.Wrapper, series_files: list[str], series_id: str) -> SeqInfo:
+class CustomSeqinfoT(Protocol):
+ def __call__(self, wrapper: dw.Wrapper, series_files: list[str]) -> Hashable:
+ ...
+
+
+def create_seqinfo(
+ mw: dw.Wrapper,
+ series_files: list[str],
+ series_id: str,
+ custom_seqinfo: CustomSeqinfoT | None = None,
+) -> SeqInfo:
"""Generate sequence info
Parameters
@@ -80,6 +101,20 @@ def create_seqinfo(mw: dw.Wrapper, series_files: list[str], series_id: str) -> S
global total_files
total_files += len(series_files)
+ custom_seqinfo_data = (
+ custom_seqinfo(wrapper=mw, series_files=series_files)
+ if custom_seqinfo
+ else None
+ )
+ try:
+ hash(custom_seqinfo_data)
+ except TypeError:
+ raise RuntimeError(
+ "Data returned by the heuristics custom_seqinfo is not hashable. "
+ "See https://heudiconv.readthedocs.io/en/latest/heuristics.html#custom_seqinfo for more "
+ "details."
+ )
+
return SeqInfo(
total_files_till_now=total_files,
example_dcm_file=op.basename(series_files[0]),
@@ -109,6 +144,7 @@ def create_seqinfo(mw: dw.Wrapper, series_files: list[str], series_id: str) -> S
date=dcminfo.get("AcquisitionDate"),
series_uid=dcminfo.get("SeriesInstanceUID"),
time=dcminfo.get("AcquisitionTime"),
+ custom=custom_seqinfo_data,
)
@@ -181,6 +217,7 @@ def group_dicoms_into_seqinfos(
dict[SeqInfo, list[str]],
]
| None = None,
+ custom_seqinfo: CustomSeqinfoT | None = None,
) -> dict[Optional[str], dict[SeqInfo, list[str]]]:
...
@@ -199,6 +236,7 @@ def group_dicoms_into_seqinfos(
dict[SeqInfo, list[str]],
]
| None = None,
+ custom_seqinfo: CustomSeqinfoT | None = None,
) -> dict[SeqInfo, list[str]]:
...
@@ -215,6 +253,7 @@ def group_dicoms_into_seqinfos(
dict[SeqInfo, list[str]],
]
| None = None,
+ custom_seqinfo: CustomSeqinfoT | None = None,
) -> dict[Optional[str], dict[SeqInfo, list[str]]] | dict[SeqInfo, list[str]]:
"""Process list of dicoms and return seqinfo and file group
`seqinfo` contains per-sequence extract of fields from DICOMs which
@@ -236,9 +275,11 @@ def group_dicoms_into_seqinfos(
Creates a flattened `seqinfo` with corresponding DICOM files. True when
invoked with `dicom_dir_template`.
custom_grouping: str or callable, optional
- grouping key defined within heuristic. Can be a string of a
- DICOM attribute, or a method that handles more complex groupings.
-
+ grouping key defined within heuristic. Can be a string of a
+ DICOM attribute, or a method that handles more complex groupings.
+ custom_seqinfo: callable, optional
+ A callable which will be provided MosaicWrapper giving possibility to
+ extract any custom DICOM metadata of interest.
Returns
-------
@@ -358,7 +399,7 @@ def group_dicoms_into_seqinfos(
else:
# nothing to see here, just move on
continue
- seqinfo = create_seqinfo(mw, series_files, series_id_str)
+ seqinfo = create_seqinfo(mw, series_files, series_id_str, custom_seqinfo)
key: Optional[str]
if per_studyUID:
=====================================
heudiconv/external/dlad.py
=====================================
@@ -146,21 +146,25 @@ def add_to_datalad(
message="Added gitattributes to place all .heudiconv content"
" under annex",
)
- ds.save(
+ save_res = ds.save(
".",
recursive=True
# not in effect! ?
# annex_add_opts=['--include-dotfiles']
)
+ annexed_files = [sr["path"] for sr in save_res if sr.get("key", None)]
- # TODO: filter for only changed files?
# Provide metadata for sensitive information
- mark_sensitive(ds, "sourcedata")
- mark_sensitive(ds, "*_scans.tsv") # top level
- mark_sensitive(ds, "*/*_scans.tsv") # within subj
- mark_sensitive(ds, "*/*/*_scans.tsv") # within sess/subj
- mark_sensitive(ds, "*/anat") # within subj
- mark_sensitive(ds, "*/*/anat") # within ses/subj
+ sensitive_patterns = [
+ "sourcedata",
+ "*_scans.tsv", # top level
+ "*/*_scans.tsv", # within subj
+ "*/*/*_scans.tsv", # within sess/subj
+ "*/anat", # within subj
+ "*/*/anat", # within ses/subj
+ ]
+ for sp in sensitive_patterns:
+ mark_sensitive(ds, sp, annexed_files)
if dsh_path:
mark_sensitive(ds, ".heudiconv") # entire .heudiconv!
superds.save(path=ds.path, message=msg, recursive=True)
@@ -178,7 +182,7 @@ def add_to_datalad(
"""
-def mark_sensitive(ds: Dataset, path_glob: str) -> None:
+def mark_sensitive(ds: Dataset, path_glob: str, files: list[str] | None = None) -> None:
"""
Parameters
@@ -186,18 +190,22 @@ def mark_sensitive(ds: Dataset, path_glob: str) -> None:
ds : Dataset to operate on
path_glob : str
glob of the paths within dataset to work on
+ files : list[str]
+ subset of files to mark
Returns
-------
None
"""
paths = glob(op.join(ds.path, path_glob))
+ if files:
+ paths = [p for p in paths if p in files]
if not paths:
return
lgr.debug("Marking %d files with distribution-restrictions field", len(paths))
# set_metadata can be a bloody generator
res = ds.repo.set_metadata(
- paths, init=dict([("distribution-restrictions", "sensitive")]), recursive=True
+ paths, add=dict([("distribution-restrictions", "sensitive")]), recursive=True
)
if inspect.isgenerator(res):
res = list(res)
=====================================
heudiconv/external/tests/test_dlad.py
=====================================
@@ -28,3 +28,24 @@ def test_mark_sensitive(tmp_path: Path) -> None:
# g2 since the same content
assert not all_meta.pop("g1", None) # nothing or empty record
assert all_meta == {"f1": target_rec, "f2": target_rec, "g2": target_rec}
+
+
+def test_mark_sensitive_subset(tmp_path: Path) -> None:
+ ds = dl.Dataset(tmp_path).create(force=True)
+ create_tree(
+ str(tmp_path),
+ {
+ "f1": "d1",
+ "f2": "d2",
+ "g1": "d3",
+ "g2": "d1",
+ },
+ )
+ ds.save(".")
+ mark_sensitive(ds, "f*", [str(tmp_path / "f1")])
+ all_meta = dict(ds.repo.get_metadata("."))
+ target_rec = {"distribution-restrictions": ["sensitive"]}
+ # g2 since the same content
+ assert not all_meta.pop("g1", None) # nothing or empty record
+ assert not all_meta.pop("f2", None) # nothing or empty record
+ assert all_meta == {"f1": target_rec, "g2": target_rec}
=====================================
heudiconv/heuristics/convertall.py
=====================================
@@ -1,9 +1,12 @@
from __future__ import annotations
+import logging
from typing import Optional
from heudiconv.utils import SeqInfo
+lgr = logging.getLogger("heudiconv")
+
def create_key(
template: Optional[str],
=====================================
heudiconv/heuristics/convertall_custom.py
=====================================
@@ -0,0 +1,32 @@
+"""A demo convertall heuristic with custom_seqinfo extracting affine and sample DICOM path
+
+This heuristic also demonstrates on how to create a "derived" heuristic which would augment
+behavior of an already existing heuristic without complete rewrite. Such approach could be
+useful for heuristic like reproin to overload mapping etc.
+"""
+from __future__ import annotations
+
+from typing import Any
+
+import nibabel.nicom.dicomwrappers as dw
+
+from .convertall import * # noqa: F403
+
+
+def custom_seqinfo(
+ series_files: list[str], wrapper: dw.Wrapper, **kw: Any # noqa: U100
+) -> tuple[str | None, str]:
+ """Demo for extracting custom header fields into custom_seqinfo field
+
+ Operates on already loaded DICOM data.
+ Origin: https://github.com/nipy/heudiconv/pull/333
+ """
+
+ from nibabel.nicom.dicomwrappers import WrapperError
+
+ try:
+ affine = str(wrapper.affine)
+ except WrapperError:
+ lgr.exception("Errored out while obtaining/converting affine") # noqa: F405
+ affine = None
+ return affine, series_files[0]
=====================================
heudiconv/heuristics/example.py
=====================================
@@ -72,11 +72,12 @@ def infotodict(
}
last_run = len(seqinfo)
for s in seqinfo:
- series_num_str = s.series_id.split('-', 1)[0]
+ series_num_str = s.series_id.split("-", 1)[0]
if not series_num_str.isdecimal():
raise ValueError(
f"This heuristic can operate only on data when series_id has form <series-number>-<something else>, "
- f"and <series-number> is a numeric number. Got series_id={s.series_id}")
+ f"and <series-number> is a numeric number. Got series_id={s.series_id}"
+ )
series_num: int = int(series_num_str)
sl, nt = (s.dim3, s.dim4)
if (sl == 176) and (nt == 1) and ("MPRAGE" in s.protocol_name):
=====================================
heudiconv/heuristics/reproin.py
=====================================
@@ -637,12 +637,14 @@ def infotodict(
# For scouts -- we want only dicoms
# https://github.com/nipy/heudiconv/issues/145
outtype: tuple[str, ...]
- if "_Scout" in s.series_description or (
- datatype == "anat"
- and datatype_suffix
- and datatype_suffix.startswith("scout")
- ) or (
- s.series_description.lower() == s.protocol_name.lower() + "_setter"
+ if (
+ "_Scout" in s.series_description
+ or (
+ datatype == "anat"
+ and datatype_suffix
+ and datatype_suffix.startswith("scout")
+ )
+ or (s.series_description.lower() == s.protocol_name.lower() + "_setter")
):
outtype = ("dicom",)
else:
@@ -725,7 +727,11 @@ def get_unique(seqinfos: list[SeqInfo], attr: str) -> Any:
"""
values = set(getattr(si, attr) for si in seqinfos)
- assert len(values) == 1
+ if len(values) != 1:
+ raise AssertionError(
+ f"Was expecting a single value for attribute {attr!r} "
+ f"but got: {', '.join(sorted(values))}"
+ )
return values.pop()
@@ -742,9 +748,9 @@ def infotoids(seqinfos: Iterable[SeqInfo], outdir: str) -> dict[str, Optional[st
# TODO: fix up subject id if missing some 0s
if study_description:
# Generally it is a ^ but if entered manually, ppl place space in it
- split = re.split("[ ^]", study_description, 1)
- # split first one even more, since couldbe PI_Student or PI-Student
- split = re.split("-|_", split[0], 1) + split[1:]
+ split = re.split("[ ^]", study_description, maxsplit=1)
+ # split first one even more, since could be PI_Student or PI-Student
+ split = re.split("[-_]", split[0], maxsplit=1) + split[1:]
# locator = study_description.replace('^', '/')
locator = "/".join(split)
=====================================
heudiconv/heuristics/test_reproin.py
=====================================
@@ -7,6 +7,8 @@ import re
from typing import NamedTuple
from unittest.mock import patch
+import pytest
+
from . import reproin
from .reproin import (
filter_files,
@@ -14,12 +16,20 @@ from .reproin import (
fix_dbic_protocol,
fixup_subjectid,
get_dups_marked,
+ get_unique,
md5sum,
parse_series_spec,
sanitize_str,
)
+class FakeSeqInfo(NamedTuple):
+ accession_number: str
+ study_description: str
+ field1: str
+ field2: str
+
+
def test_get_dups_marked() -> None:
no_dups: dict[tuple[str, tuple[str, ...], None], list[int]] = {
("some", ("foo",), None): [1]
@@ -97,12 +107,6 @@ def test_fix_canceled_runs() -> None:
def test_fix_dbic_protocol() -> None:
- class FakeSeqInfo(NamedTuple):
- accession_number: str
- study_description: str
- field1: str
- field2: str
-
accession_number = "A003"
seq1 = FakeSeqInfo(
accession_number,
@@ -235,3 +239,19 @@ def test_parse_series_spec() -> None:
"session": "01",
"dir": "AP",
}
+
+
+def test_get_unique() -> None:
+ accession_number = "A003"
+ acqs = [
+ FakeSeqInfo(accession_number, "mystudy", "nochangeplease", "nochangeeither"),
+ FakeSeqInfo(accession_number, "mystudy2", "nochangeplease", "nochangeeither"),
+ ]
+
+ assert get_unique(acqs, "accession_number") == accession_number # type: ignore[arg-type]
+ with pytest.raises(AssertionError) as ce:
+ get_unique(acqs, "study_description") # type: ignore[arg-type]
+ assert (
+ str(ce.value)
+ == "Was expecting a single value for attribute 'study_description' but got: mystudy, mystudy2"
+ )
=====================================
heudiconv/info.py
=====================================
@@ -11,7 +11,6 @@ CLASSIFIERS = [
"Environment :: Console",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: Apache Software License",
- "Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
@@ -20,7 +19,7 @@ CLASSIFIERS = [
"Typing :: Typed",
]
-PYTHON_REQUIRES = ">=3.7"
+PYTHON_REQUIRES = ">=3.8"
REQUIRES = [
# not usable in some use cases since might be just a downloader, not binary
=====================================
heudiconv/main.py
=====================================
@@ -56,7 +56,7 @@ def setup_exceptionhook() -> None:
def process_extra_commands(
outdir: str,
command: str,
- files: list[str],
+ files: Optional[list[str]],
heuristic: Optional[str],
session: Optional[str],
subjs: Optional[list[str]],
@@ -72,8 +72,8 @@ def process_extra_commands(
Output directory
command : {'treat-json', 'ls', 'populate-templates', 'populate-intended-for'}
Heudiconv command to run
- files : list of str
- List of files
+ files : list of str or None
+ List of files if command needs/expects it
heuristic : str or None
Path to heuristic file or name of builtin heuristic.
session : str or None
@@ -83,12 +83,21 @@ def process_extra_commands(
grouping : {'studyUID', 'accession_number', 'all', 'custom'}
How to group dicoms.
"""
+
+ def ensure_has_files() -> None:
+ if files is None:
+ raise ValueError(f"command {command} expects --files being provided")
+
if command == "treat-jsons":
+ ensure_has_files()
+ assert files is not None # for mypy now
for fname in files:
treat_infofile(fname)
elif command == "ls":
ensure_heuristic_arg(heuristic)
assert heuristic is not None
+ ensure_has_files()
+ assert files is not None # for mypy now
heuristic_mod = load_heuristic(heuristic)
heuristic_ls = getattr(heuristic_mod, "ls", None)
for fname in files:
@@ -111,10 +120,14 @@ def process_extra_commands(
elif command == "populate-templates":
ensure_heuristic_arg(heuristic)
assert heuristic is not None
+ ensure_has_files()
+ assert files is not None # for mypy now
heuristic_mod = load_heuristic(heuristic)
for fname in files:
populate_bids_templates(fname, getattr(heuristic_mod, "DEFAULT_FIELDS", {}))
elif command == "sanitize-jsons":
+ ensure_has_files()
+ assert files is not None # for mypy now
tuneup_bids_json_files(files)
elif command == "heuristics":
from .utils import get_known_heuristics_with_descriptions
@@ -357,9 +370,12 @@ def workflow(
)
if command:
- if files is None:
- raise ValueError("'command' given but 'files' is None")
- assert dicom_dir_template is None
+ if dicom_dir_template:
+ lgr.warning(
+ f"DICOM directory template {dicom_dir_template!r} was provided but will be ignored since "
+ f"commands do not care about it ATM"
+ )
+
process_extra_commands(
outdir,
command,
=====================================
heudiconv/parser.py
=====================================
@@ -224,6 +224,7 @@ def get_study_sessions(
file_filter=getattr(heuristic, "filter_files", None),
dcmfilter=getattr(heuristic, "filter_dicom", None),
custom_grouping=getattr(heuristic, "grouping", None),
+ custom_seqinfo=getattr(heuristic, "custom_seqinfo", None),
)
if sids:
@@ -278,15 +279,10 @@ def get_study_sessions(
)
lgr.info("Study session for %r", study_session_info)
- if study_session_info in study_sessions:
- if grouping != "all":
- # MG - should this blow up to mimic -d invocation?
- lgr.warning(
- "Existing study session with the same values (%r)."
- " Skipping DICOMS %s",
- study_session_info,
- seqinfo.values(),
- )
- continue
+ if grouping != "all":
+ assert study_session_info not in study_sessions, (
+ f"Existing study session {study_session_info} "
+ f"already in analyzed sessions {study_sessions.keys()}"
+ )
study_sessions[study_session_info] = seqinfo
return study_sessions
=====================================
heudiconv/tests/test_convert.py
=====================================
@@ -14,6 +14,7 @@ from heudiconv.cli.run import main as runner
import heudiconv.convert
from heudiconv.convert import (
DW_IMAGE_IN_FMAP_FOLDER_WARNING,
+ bvals_are_zero,
update_complex_name,
update_multiecho_name,
update_uncombined_name,
@@ -287,3 +288,16 @@ def test_populate_intended_for(
else:
# If there was no heuristic, make sure populate_intended_for was not called
assert not output.out
+
+
+def test_bvals_are_zero() -> None:
+ """Unit testing for heudiconv.convert.bvals_are_zero(),
+ which checks if non-dwi bvals are all zeros and can be removed
+ """
+ zero_bvals = op.join(TESTS_DATA_PATH, "zeros.bval")
+ non_zero_bvals = op.join(TESTS_DATA_PATH, "non_zeros.bval")
+
+ assert bvals_are_zero(zero_bvals)
+ assert not bvals_are_zero(non_zero_bvals)
+ assert bvals_are_zero([zero_bvals, zero_bvals])
+ assert not bvals_are_zero([non_zero_bvals, zero_bvals])
=====================================
heudiconv/tests/test_dicoms.py
=====================================
@@ -99,6 +99,25 @@ def test_group_dicoms_into_seqinfos() -> None:
]
+def test_custom_seqinfo() -> None:
+ """Tests for custom seqinfo extraction"""
+
+ from heudiconv.heuristics.convertall_custom import custom_seqinfo
+
+ dcmfiles = glob(op.join(TESTS_DATA_PATH, "phantom.dcm"))
+
+ seqinfos = group_dicoms_into_seqinfos(
+ dcmfiles, "studyUID", flatten=True, custom_seqinfo=custom_seqinfo
+ ) # type: ignore
+
+ seqinfo = list(seqinfos.keys())[0]
+
+ assert hasattr(seqinfo, "custom")
+ assert isinstance(seqinfo.custom, tuple)
+ assert len(seqinfo.custom) == 2
+ assert seqinfo.custom[1] == dcmfiles[0]
+
+
def test_get_datetime_from_dcm_from_acq_date_time() -> None:
typical_dcm = dcm.dcmread(
op.join(TESTS_DATA_PATH, "phantom.dcm"), stop_before_pixels=True
=====================================
heudiconv/utils.py
=====================================
@@ -24,6 +24,7 @@ from types import ModuleType
from typing import (
Any,
AnyStr,
+ Hashable,
Mapping,
NamedTuple,
Optional,
@@ -69,6 +70,7 @@ class SeqInfo(NamedTuple):
date: Optional[str] # 24
series_uid: Optional[str] # 25
time: Optional[str] # 26
+ custom: Optional[Hashable] # 27
class StudySessionInfo(NamedTuple):
=====================================
pyproject.toml
=====================================
@@ -1,7 +1,7 @@
[build-system]
requires = [
"setuptools >= 46.4.0",
- "versioningit ~= 1.0",
+ "versioningit ~= 2.3",
"wheel ~= 0.32"
]
build-backend = "setuptools.build_meta"
View it on GitLab: https://salsa.debian.org/med-team/heudiconv/-/commit/9abcb50854a3868111ed767bc5f3b7d8ed8c700f
--
View it on GitLab: https://salsa.debian.org/med-team/heudiconv/-/commit/9abcb50854a3868111ed767bc5f3b7d8ed8c700f
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/debian-med-commit/attachments/20240313/7927245e/attachment-0001.htm>
More information about the debian-med-commit
mailing list