[Git][debian-gis-team/netcdf4-python][master] 5 commits: New upstream version 1.7.5
Bas Couwenberg (@sebastic)
gitlab at salsa.debian.org
Fri Oct 16 05:51:24 BST 2026
Bas Couwenberg pushed to branch master at Debian GIS Project / netcdf4-python
Commits:
c7b4bce6 by Bas Couwenberg at 2026-10-16T06:41:39+02:00
New upstream version 1.7.5
- - - - -
3115ce0f by Bas Couwenberg at 2026-10-16T06:41:41+02:00
Update upstream source from tag 'upstream/1.7.5'
Update to upstream version '1.7.5'
with Debian dir d9401874e81f95d93b981f93d0e194e9d5ce38ee
- - - - -
6b0a7a4a by Bas Couwenberg at 2026-10-16T06:42:36+02:00
New upstream release.
- - - - -
9e1d081d by Bas Couwenberg at 2026-10-16T06:43:24+02:00
Refresh patches.
- - - - -
486ba7f4 by Bas Couwenberg at 2026-10-16T06:43:32+02:00
Set distribution to unstable.
- - - - -
28 changed files:
- .github/workflows/build_latest.yml
- .github/workflows/build_master.yml
- .github/workflows/build_old.yml
- .github/workflows/cibuildwheel.yml
- .github/workflows/miniconda.yml
- Changelog
- MANIFEST.in
- README.md
- + _build/backend.py
- + _build/utils.py
- debian/changelog
- debian/patches/rpath.patch
- + docs/_config.yml
- docs/index.html
- examples/bench_diskless.py
- examples/tutorial.py
- include/netCDF4.pxi
- pyproject.toml
- setup.py
- src/netCDF4/__init__.pyi
- src/netCDF4/_netCDF4.pyx
- src/netCDF4/utils.py
- test/test_atts.py
- + test/test_cli.py
- test/test_dap.py
- + test/test_empty_indexing.py
- test/test_fancyslicing.py
- test/test_utils.py
Changes:
=====================================
.github/workflows/build_latest.yml
=====================================
@@ -17,12 +17,12 @@ jobs:
python-version: ["3.14"]
steps:
- - uses: actions/checkout at v6
+ - uses: actions/checkout at v7
with:
submodules: true
- name: Set up Python ${{ matrix.python-version }}
- uses: actions/setup-python at v6
+ uses: actions/setup-python at v7
with:
python-version: ${{ matrix.python-version }}
@@ -63,7 +63,7 @@ jobs:
- name: Install python dependencies via pip
run: |
python -m pip install --upgrade pip
- python -m pip install numpy cython cftime pytest twine wheel check-manifest mpi4py typing-extensions
+ python -m pip install --group dev
- name: Install netcdf4-python
run: |
@@ -107,6 +107,7 @@ jobs:
# - name: Tarball
# run: |
+# python -m pip install twine wheel check-manifest
# export PATH=${NETCDF_DIR}/bin:${PATH}
# python setup.py --version
# check-manifest --version
=====================================
.github/workflows/build_master.yml
=====================================
@@ -14,12 +14,12 @@ jobs:
python-version: ["3.14"]
steps:
- - uses: actions/checkout at v6
+ - uses: actions/checkout at v7
with:
submodules: true
- name: Set up Python ${{ matrix.python-version }}
- uses: actions/setup-python at v6
+ uses: actions/setup-python at v7
with:
python-version: ${{ matrix.python-version }}
@@ -51,7 +51,7 @@ jobs:
- name: Install python dependencies via pip
run: |
python -m pip install --upgrade pip
- python -m pip install numpy cython cftime pytest twine wheel check-manifest mpi4py mypy types-setuptools typing-extensions
+ python -m pip install --group dev
- name: Install netcdf4-python
run: |
@@ -88,6 +88,6 @@ jobs:
- name: Stubtest
run: |
- stubtest netCDF4 --allowlist .github/stubtest-allowlist --mypy-config-file=pyproject.toml
+ stubtest netCDF4 --allowlist .github/stubtest-allowlist --ignore-unused-allowlist --mypy-config-file=pyproject.toml
mypy test
mypy examples
=====================================
.github/workflows/build_old.yml
=====================================
@@ -17,12 +17,12 @@ jobs:
python-version: ["3.14"]
steps:
- - uses: actions/checkout at v6
+ - uses: actions/checkout at v7
with:
submodules: true
- name: Set up Python ${{ matrix.python-version }}
- uses: actions/setup-python at v6
+ uses: actions/setup-python at v7
with:
python-version: ${{ matrix.python-version }}
@@ -63,7 +63,7 @@ jobs:
- name: Install python dependencies via pip
run: |
python -m pip install --upgrade pip
- python -m pip install numpy cython cftime pytest twine wheel check-manifest mpi4py typing-extensions
+ python -m pip install --group dev
- name: Install netcdf4-python
run: |
=====================================
.github/workflows/cibuildwheel.yml
=====================================
@@ -18,11 +18,11 @@ jobs:
name: Build source distribution
runs-on: ubuntu-22.04
steps:
- - uses: actions/checkout at v6
+ - uses: actions/checkout at v7
with:
fetch-depth: 0
- - uses: actions/setup-python at v6
+ - uses: actions/setup-python at v7
name: Install Python
with:
python-version: 3.x
@@ -38,7 +38,7 @@ jobs:
pip install build
&& python -m build --sdist . --outdir dist
- - uses: actions/upload-artifact at v6
+ - uses: actions/upload-artifact at v7
with:
name: pypi-artifacts
path: ${{ github.workspace }}/dist/*.tar.gz
@@ -59,13 +59,13 @@ jobs:
arch: aarch64
steps:
- - uses: actions/checkout at v6
+ - uses: actions/checkout at v7
with:
fetch-depth: 0
# For aarch64 support
# https://cibuildwheel.pypa.io/en/stable/faq/#emulation
- - uses: docker/setup-qemu-action at v3
+ - uses: docker/setup-qemu-action at v4
with:
platforms: all
if: runner.os == 'Linux' && matrix.arch == 'aarch64'
@@ -74,22 +74,22 @@ jobs:
shell: bash
# On PRs we run only oldest and newest Python versions to reduce CI load.
# Skips pypy and musllinux everywhere.
- # We are building 310, 311 and 314 for now.
+ # We are building 311 and 315 for now.
# (3.11 is the oldest version for which we support abi3 wheels)
# These needs to rotate every new Python release.
run: |
set -x
- echo "CIBW_BUILD=cp310-* cp311-* cp314-* cp314t-*" >> $GITHUB_ENV
+ echo "CIBW_BUILD=cp311-* cp315-* cp315t-*" >> $GITHUB_ENV
set +x
if: ${{ github.event_name }} == "pull_request"
- name: "Building ${{ matrix.os }} (${{ matrix.arch }}) wheels"
- uses: pypa/cibuildwheel at v3.3.0
+ uses: pypa/cibuildwheel at v4.3.1
env:
CIBW_ARCHS: ${{ matrix.arch }}
- - uses: actions/upload-artifact at v6
+ - uses: actions/upload-artifact at v7
with:
name: pypi-artifacts-${{ matrix.os }}-${{ matrix.arch }}
path: ${{ github.workspace }}/wheelhouse/*.whl
@@ -104,35 +104,35 @@ jobs:
include:
- os: windows-latest
arch: AMD64
- - os: macos-14
+ - os: macos-15
arch: arm64
- os: macos-15-intel
arch: x86_64
steps:
- - uses: actions/checkout at v6
+ - uses: actions/checkout at v7
with:
fetch-depth: 0
- - uses: actions/setup-python at v6
+ - uses: actions/setup-python at v7
name: Install Python
with:
python-version: 3.x
- name: Setup Micromamba Python ${{ matrix.python-version }}
- uses: mamba-org/setup-micromamba at v2
+ uses: mamba-org/setup-micromamba at v3
with:
environment-name: build
init-shell: bash
create-args: >-
- python=${{ matrix.python-version }} libnetcdf=4.9.3 --channel conda-forge
+ python=${{ matrix.python-version }} libnetcdf=4.10.1 --channel conda-forge
- name: Build wheels for Windows/Mac
- uses: pypa/cibuildwheel at v3.3.0
+ uses: pypa/cibuildwheel at v4.3.1
env:
CIBW_ARCHS: ${{ matrix.arch }}
- - uses: actions/upload-artifact at v6
+ - uses: actions/upload-artifact at v7
with:
name: pypi-artifacts-${{ matrix.os }}-${{ matrix.arch }}
path: ${{ github.workspace }}/wheelhouse/*.whl
@@ -142,11 +142,11 @@ jobs:
name: Build wheels for ARM64 on Windows
runs-on: windows-11-arm
steps:
- - uses: actions/checkout at v6
+ - uses: actions/checkout at v7
with:
fetch-depth: 0
- - uses: actions/setup-python at v6
+ - uses: actions/setup-python at v7
name: Install Python
with:
python-version: 3.x
@@ -167,12 +167,12 @@ jobs:
echo "NETCDF4_DIR=C:\vcpkg\installed\arm64-windows" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
- name: Build wheels for Windows ARM64
- uses: pypa/cibuildwheel at v3.3.0
+ uses: pypa/cibuildwheel at v4.3.1
env:
CIBW_ARCHS: ARM64
CIBW_SKIP: "cp310-*"
- - uses: actions/upload-artifact at v6
+ - uses: actions/upload-artifact at v7
with:
name: pypi-artifacts-windows-11-arm-ARM64
path: ${{ github.workspace }}/wheelhouse/*.whl
@@ -183,7 +183,7 @@ jobs:
name: "Show artifacts"
runs-on: ubuntu-22.04
steps:
- - uses: actions/download-artifact at v7
+ - uses: actions/download-artifact at v8
with:
pattern: pypi-artifacts*
path: ${{ github.workspace }}/dist
@@ -198,10 +198,10 @@ jobs:
needs: [build_bdist, build_sdist, build_wheels_winmac, build_wheels_windows_arm]
name: "Publish to PyPI"
runs-on: ubuntu-22.04
- # upload to PyPI for every tag starting with 'v'
- if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/v')
+ # upload to PyPI when github release made.
+ if: github.event_name == 'release' && github.event.action == 'published'
steps:
- - uses: actions/download-artifact at v7
+ - uses: actions/download-artifact at v8
with:
pattern: pypi-artifacts*
path: ${{ github.workspace }}/dist
=====================================
.github/workflows/miniconda.yml
=====================================
@@ -12,7 +12,14 @@ jobs:
# NO_NET: 1
strategy:
matrix:
- python-version: [ "3.10", "3.11", "3.12", "3.13", "3.14" ]
+ python-version: [
+ ["python", "3.11"],
+ ["python", "3.12"],
+ ["python", "3.13"],
+ ["python", "3.14"],
+ ["python", "3.15"],
+ ["python-freethreading", "3.14" ],
+ ]
os: [windows-latest, ubuntu-latest, macos-latest]
platform: [x64, x32]
exclude:
@@ -24,17 +31,17 @@ jobs:
shell: bash -l {0}
steps:
- - uses: actions/checkout at v6
+ - uses: actions/checkout at v7
with:
submodules: true
- name: Setup Micromamba
- uses: mamba-org/setup-micromamba at v2
+ uses: mamba-org/setup-micromamba at v3
with:
environment-name: TEST
init-shell: bash
create-args: >-
- python=${{ matrix.python-version }}
+ ${{ matrix.python-version[0] }}=${{ matrix.python-version[1] }}
numpy cython pip setuptools pytest hdf5 libnetcdf cftime zlib certifi typing-extensions
--channel conda-forge
@@ -48,7 +55,7 @@ jobs:
if [ "$RUNNER_OS" == "Windows" ]; then
export HDF5_PLUGIN_PATH="${CONDA_PREFIX}\\Library\\hdf5\\lib\\plugin"
else
- export HDF5_PLUGIN_PATH="${CONDA_PREFIX}/hdf5/lib/plugin/"
+ export HDF5_PLUGIN_PATH="${CONDA_PREFIX}/lib/hdf5/plugin/"
fi
pytest -s -rxs -v test
@@ -63,12 +70,12 @@ jobs:
run:
shell: bash -l {0}
steps:
- - uses: actions/checkout at v6
+ - uses: actions/checkout at v7
with:
submodules: true
- name: Setup Micromamba
- uses: mamba-org/setup-micromamba at v2
+ uses: mamba-org/setup-micromamba at v3
with:
environment-name: TEST
init-shell: bash
=====================================
Changelog
=====================================
@@ -1,3 +1,17 @@
+ version 1.7.5 (tag v1.7.5rel)
+ =============================
+ * remove python 3.10 support (it reaches EOL October 31, 2026).
+ * Reject fractional scalar indices instead of silently truncating reads or writes (issue #1350).
+ * Preserve surviving dimensions and decoded types for empty orthogonal selections,
+ including empty integer indices and all-false masks (PR #1499).
+ * Convert non-native-endian numeric attributes to native byte order before writing (issue #1502).
+ * Fix command-line usage error exit codes and --help handling, honour nc4tonc3 --chunk,
+ and fix ncinfo group dimension selection (issue #1500).
+ * Change default encoding for stringtochar/chartostring functions from 'utf-8' to 'utf-8'/'ascii' for dtype.kind='U'/'S'
+ (issue #1464).
+ * Fix DeprecationWarning for assigning to numpy.ndarray.shape for numpy >= 2.5.0 (issue #1468).
+ * Disable parallel support even if libs support it via DISABLE_PARALLEL_SUPPORT env var (issue #1389).
+
version 1.7.4 (tag v1.7.4rel)
================================
* Make sure automatic conversion of character arrays <--> string arrays works for Unicode strings (issue #1440).
=====================================
MANIFEST.in
=====================================
@@ -27,3 +27,4 @@ include *.md
include *.py
include *.release
include *.sh
+include _build/*.py
=====================================
README.md
=====================================
@@ -10,6 +10,8 @@
## News
For details on the latest updates, see the [Changelog](https://github.com/Unidata/netcdf4-python/blob/master/Changelog).
+10/17/2026: Version [1.7.5](https://pypi.python.org/pypi/netCDF4/1.7.5) released. See Changelog for details.
+
1/5/2026: Version [1.7.4](https://pypi.python.org/pypi/netCDF4/1.7.4) released. Compression plugins now included in wheels, windows/arm64 and
free-threaded python wheels provided. Automatic conversion of character arrays <--> string arrays works for Unicode (not just ascii) strings.
WARNING: netcdf-c is not thread-safe and netcdf4-python does have internal locking so expect segfaults if you
@@ -269,15 +271,16 @@ conda install -c conda-forge netCDF4
```
## Development installation
-* Clone GitHub repository (`git clone https://github.com/Unidata/netcdf4-python.git`)
-
-* Make sure [numpy](http://www.numpy.org/) and [Cython](http://cython.org/) are
- installed and you have [Python](https://www.python.org) 3.8 or newer.
+* Clone GitHub repository: `git clone https://github.com/Unidata/netcdf4-python.git`
* Make sure [HDF5](http://www.h5py.org/) and netcdf-4 are installed,
and the `nc-config` utility is in your Unix PATH.
-* Run `python setup.py build`, then `pip install -e .`.
+* Make sure [Python](https://www.python.org) 3.10 or later is installed, with [pip](https://pip.pypa.io/) 25.1 or later.
+
+* Install developemnt dependencies with: `pip install --group dev`
+
+* Install this package as an "editable" build with: `pip install -e . --no-build-isolation`
* To run all the tests, execute `cd test && python run_all.py`.
=====================================
_build/backend.py
=====================================
@@ -0,0 +1,21 @@
+# -*- coding: utf-8 -*-
+"""
+In-tree build backend that programmatically adds mpi4py to the list of build dependencies if the
+underlying netCDF4 C library has parallel support enabled.
+"""
+
+from setuptools.build_meta import *
+
+import utils
+
+
+def get_requires_for_build_editable(config_settings=None):
+ return ["mpi4py>=3.1"] if utils.netcdf4_has_parallel_support() else []
+
+
+def get_requires_for_build_sdist(config_settings=None):
+ return ["mpi4py>=3.1"] if utils.netcdf4_has_parallel_support() else []
+
+
+def get_requires_for_build_wheel(config_settings=None):
+ return ["mpi4py>=3.1"] if utils.netcdf4_has_parallel_support() else []
=====================================
_build/utils.py
=====================================
@@ -0,0 +1,131 @@
+# -*- coding: utf-8 -*-
+"""
+This module contains a streamlined version of some utilities defined in `setup.py`, to be at
+disposal of in-tree build backends.
+"""
+
+import configparser
+import os
+import subprocess
+
+
+PROJECT_ROOT_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
+OPEN_KWARGS = {"encoding": "utf-8"}
+
+
+def get_config_flags(command: list[str], flag: str) -> list[str]:
+ """Pull out specific flags from a config command (pkg-config or nc-config)"""
+ flags = subprocess.run(command, capture_output=True, text=True)
+ return [arg[2:] for arg in flags.stdout.split() if arg.startswith(flag)]
+
+
+def is_netcdf4_include_dir(inc_dir: str) -> bool:
+ try:
+ f = open(os.path.join(inc_dir, "netcdf.h"), **OPEN_KWARGS)
+ except OSError:
+ return False
+
+ for line in f:
+ if line.startswith("nc_inq_compound"):
+ return True
+ return False
+
+
+def get_netcdf4_include_dir():
+ netcdf4_dir = os.environ.get("NETCDF4_DIR")
+ netcdf4_incdir = os.environ.get("NETCDF4_INCDIR")
+
+ if bool(int(os.environ.get("USE_SETUPCFG", 1))) and os.path.exists(
+ setup_cfg := os.path.join(PROJECT_ROOT_DIR, "setup.cfg")
+ ):
+ config = configparser.ConfigParser()
+ config.read(setup_cfg)
+
+ netcdf4_dir = config.get("directories", "NETCDF4_DIR", fallback=netcdf4_dir)
+ netcdf4_incdir = config.get(
+ "directories", "NETCDF4_INCDIR", fallback=netcdf4_incdir
+ )
+
+ # make sure USE_NCCONFIG from environment takes precedence over use_ncconfig from setup.cfg
+ # (issue #341)
+ if "USE_NCCONFIG" in os.environ:
+ use_ncconfig = bool(int(os.environ.get("USE_NCCONFIG", 0)))
+ else:
+ use_ncconfig = config.getboolean("options", "use_ncconfig", fallback=None)
+
+ ncconfig = config.get("options", "ncconfig", fallback=None)
+ else:
+ use_ncconfig = None
+ ncconfig = None
+
+ try:
+ if ncconfig is None:
+ if netcdf4_dir is not None:
+ ncconfig = os.path.join(netcdf4_dir, "bin", "nc-config")
+ else: # otherwise, just hope it's in the users PATH
+ ncconfig = "nc-config"
+ has_ncconfig = subprocess.call([ncconfig, "--libs"]) == 0
+ except OSError:
+ has_ncconfig = False
+
+ # if nc-config exists, and use_ncconfig not set, try to use it
+ if use_ncconfig is None and has_ncconfig:
+ use_ncconfig = has_ncconfig
+
+ dirs_to_search = []
+ if os.environ.get("CONDA_PREFIX"):
+ dirs_to_search.append(os.environ["CONDA_PREFIX"]) # linux,macosx
+ dirs_to_search.append(
+ os.path.join(os.environ["CONDA_PREFIX"], "Library")
+ ) # windows
+ dirs_to_search += [
+ os.path.expanduser("~"),
+ "/usr/local",
+ "/sw",
+ "/opt",
+ "/opt/local",
+ "/opt/homebrew",
+ "/usr",
+ ]
+
+ if netcdf4_incdir is None and netcdf4_dir is None:
+ if use_ncconfig and has_ncconfig:
+ inc_dirs = get_config_flags([ncconfig, "--cflags"], "-I")
+ else:
+ inc_dirs = [os.path.join(dir_, "include") for dir_ in dirs_to_search]
+
+ for inc_dir in inc_dirs:
+ if is_netcdf4_include_dir(inc_dir):
+ netcdf4_incdir = inc_dir
+ break
+
+ if netcdf4_incdir is None:
+ raise ValueError("Did not find netCDF version 4 headers.")
+ else:
+ if netcdf4_incdir is None:
+ netcdf4_incdir = os.path.join(netcdf4_dir, "include")
+ if not is_netcdf4_include_dir(netcdf4_incdir):
+ raise ValueError(
+ f"Did not find netCDF version 4 headers under `{netcdf4_incdir}`."
+ )
+
+ return netcdf4_incdir
+
+
+def netcdf4_has_parallel_support() -> bool:
+ # disable parallel support even if libs support it (issue #1389)
+ DISABLE_PARALLEL_SUPPORT = bool(int(os.environ.get('DISABLE_PARALLEL_SUPPORT', 0)))
+ if DISABLE_PARALLEL_SUPPORT:
+ return False
+ netcdf4_incdir = get_netcdf4_include_dir()
+ if os.path.exists(ncmetapath := os.path.join(netcdf4_incdir, "netcdf_meta.h")):
+ with open(ncmetapath) as f:
+ for line in f:
+ if line.startswith("#define NC_HAS_PARALLEL"):
+ try:
+ return bool(int(line.split()[2]))
+ except ValueError:
+ pass
+ return False
+ else:
+ return False
=====================================
debian/changelog
=====================================
@@ -1,8 +1,10 @@
-netcdf4-python (1.7.4-2) UNRELEASED; urgency=medium
+netcdf4-python (1.7.5-1) unstable; urgency=medium
+ * New upstream release.
* Bump Standards-Version to 4.7.4, no changes.
+ * Refresh patches.
- -- Bas Couwenberg <sebastic at debian.org> Sat, 04 Apr 2026 10:15:25 +0200
+ -- Bas Couwenberg <sebastic at debian.org> Fri, 16 Oct 2026 06:43:25 +0200
netcdf4-python (1.7.4-1) unstable; urgency=medium
=====================================
debian/patches/rpath.patch
=====================================
@@ -4,7 +4,7 @@ Forwarded: not-needed
--- a/setup.py
+++ b/setup.py
-@@ -371,11 +371,6 @@ else:
+@@ -373,11 +373,6 @@ else:
lib_dirs.append(curl_libdir)
inc_dirs.append(curl_incdir)
@@ -16,7 +16,7 @@ Forwarded: not-needed
# Do not require numpy for just querying the package
# Taken from the h5py setup file.
if any('--' + opt in sys.argv for opt in Distribution.display_option_names +
-@@ -461,7 +456,6 @@ if 'sdist' not in sys.argv[1:] and 'clea
+@@ -468,7 +463,6 @@ if 'sdist' not in sys.argv[1:] and 'clea
libraries=libs,
library_dirs=lib_dirs,
include_dirs=include_dirs,
=====================================
docs/_config.yml
=====================================
@@ -0,0 +1 @@
+theme: jekyll-theme-minimal
=====================================
docs/index.html
=====================================
@@ -1014,7 +1014,7 @@ to a variable in an existing file, you must use <code><a title="netCDF4.Variable
to write to it.</li>
<li>You cannot use variable-length (VLEN) data types.</li>
</ul>
-<p><strong><em>Import warning regarding threads:</em></strong>
+<p><strong><em>Important warning regarding threads:</em></strong>
The underlying netcdf-c library is not thread-safe, so netcdf4-python cannot perform parallel
IO in a multi-threaded environment.
Users should expect segfaults if a netcdf file is opened on multiple threads - care should
@@ -1226,7 +1226,7 @@ further details.</p>
<h2 class="section-title" id="header-functions">Functions</h2>
<dl>
<dt id="netCDF4.chartostring"><code class="name flex">
-<span>def <span class="ident">chartostring</span></span>(<span>b, encoding='utf-8')</span>
+<span>def <span class="ident">chartostring</span></span>(<span>b, encoding=None)</span>
</code></dt>
<dd>
<div class="desc"><p><strong><code>chartostring(b,encoding='utf-8')</code></strong></p>
@@ -1236,8 +1236,8 @@ Input character array (numpy datatype <code>'S1'</code> or <code>'U1'</code>).
Will be converted to a array of strings, where each string has a fixed
length of <code>b.shape[-1]</code> characters.</p>
<p>optional kwarg <code>encoding</code> can be used to specify character encoding (default
-<code>utf-8</code>). If <code>encoding</code> is 'none' or 'bytes', a <code>numpy.string_</code> byte array is
-returned.</p>
+<code>utf-8</code> for dtype=<code>'UN'</code> or <code>ascii</code> for dtype=<code>'SN'</code>). If <code>encoding</code> is 'none' or 'bytes',
+a <code>numpy.string_</code> byte array is returned.</p>
<p>returns a numpy string array with datatype <code>'UN'</code> (or <code>'SN'</code>) and shape
<code>b.shape[:-1]</code> where where <code>N=b.shape[-1]</code>.</p></div>
</dd>
@@ -1254,7 +1254,7 @@ The datetime objects should not include a time-zone offset.</p>
<p><strong>calendar</strong>: describes the calendar to be used in the time calculations.
All the values currently defined in the
<code>CF metadata convention <http://cfconventions.org/cf-conventions/cf-conventions#calendar></code>__ are supported.
-Valid calendars <strong>'standard', 'gregorian', 'proleptic_gregorian'
+Valid calendars <strong>'standard', 'gregorian', 'proleptic_gregorian', 'tai',
'noleap', '365_day', '360_day', 'julian', 'all_leap', '366_day'</strong>.
Default is <code>None</code> which means the calendar associated with the first
input datetime instance will be used.</p>
@@ -1305,7 +1305,7 @@ and <strong>common_years since</strong> is allowed <em>only</em> for the <strong
<p><strong>calendar</strong>: describes the calendar to be used in the time calculations.
All the values currently defined in the
<code>CF metadata convention <http://cfconventions.org/cf-conventions/cf-conventions#calendar></code>__ are supported.
-Valid calendars <strong>'standard', 'gregorian', 'proleptic_gregorian'
+Valid calendars <strong>'standard', 'gregorian', 'proleptic_gregorian', 'tai',
'noleap', '365_day', '360_day', 'julian', 'all_leap', '366_day'</strong>.
Default is <code>None</code> which means the calendar associated with the first
input datetime instance will be used.</p>
@@ -1381,7 +1381,7 @@ and <strong>common_years since</strong> is allowed <em>only</em> for the <strong
<p><strong>calendar</strong>: describes the calendar used in the time calculations.
All the values currently defined in the
<code>CF metadata convention <http://cfconventions.org/cf-conventions/cf-conventions#calendar></code>__ are supported.
-Valid calendars <strong>'standard', 'gregorian', 'proleptic_gregorian'
+Valid calendars <strong>'standard', 'gregorian', 'proleptic_gregorian', 'tai',
'noleap', '365_day', '360_day', 'julian', 'all_leap', '366_day'</strong>.
Default is <strong>'standard'</strong>, which is a mixed Julian/Gregorian calendar.</p>
<p><strong>only_use_cftime_datetimes</strong>: if False, python datetime.datetime
@@ -1476,7 +1476,7 @@ unicode array (dtype = <code>'U1'</code>) will be returned.</p>
(default) or <code>'U1'</code> (if dtype=<code>'U'</code>)</p></div>
</dd>
<dt id="netCDF4.stringtochar"><code class="name flex">
-<span>def <span class="ident">stringtochar</span></span>(<span>a, encoding='utf-8', n_strlen=None)</span>
+<span>def <span class="ident">stringtochar</span></span>(<span>a, encoding=None, n_strlen=None)</span>
</code></dt>
<dd>
<div class="desc"><p><strong><code>stringtochar(a,encoding='utf-8',n_strlen=None)</code></strong></p>
@@ -1487,8 +1487,8 @@ is the number of characters in each string.
Will be converted to
an array of characters (datatype <code>'S1'</code> or <code>'U1'</code>) of shape <code>a.shape + (N,)</code>.</p>
<p>optional kwarg <code>encoding</code> can be used to specify character encoding (default
-<code>utf-8</code>). If <code>encoding</code> is 'none' or 'bytes', a <code>numpy.string_</code> the input array
-is treated a raw byte strings (<code>numpy.string_</code>).</p>
+<code>utf-8</code> for dtype=<code>'UN'</code> or <code>ascii</code> for dtype=<code>'SN'</code>). If <code>encoding</code> is 'none' or 'bytes',
+a <code>numpy.string_</code> the input array is treated a raw byte strings (<code>numpy.string_</code>).</p>
<p>optional kwarg <code>n_strlen</code> is the number of characters in each string.
Default
is None, which means <code>n_strlen</code> will be set to a.itemsize (the number of bytes
=====================================
examples/bench_diskless.py
=====================================
@@ -53,11 +53,11 @@ for format in ['NETCDF4','NETCDF3_CLASSIC','NETCDF3_64BIT']:
repr(sum(t.repeat(ntrials,1))/ntrials))
# test diskless=True in nc_open
-format: Literal["NETCDF3_CLASSIC"] = 'NETCDF3_CLASSIC' # mypy should know this but it needs help...
+fmt: Literal["NETCDF3_CLASSIC"] = 'NETCDF3_CLASSIC' # mypy should know this but it needs help...
trials=50
-sys.stdout.write('test caching of file in memory on open for %s\n' % format)
-sys.stdout.write('testing file format %s ...\n' % format)
-write_netcdf('test1.nc',format=format,closeit=True)
+sys.stdout.write('test caching of file in memory on open for %s\n' % fmt)
+sys.stdout.write('testing file format %s ...\n' % fmt)
+write_netcdf('test1.nc',format=fmt,closeit=True)
ncfile = netCDF4.Dataset('test1.nc',diskless=False)
t = Timer("read_netcdf(ncfile)","from __main__ import read_netcdf,ncfile")
sys.stdout.write('reading (from disk) took %s seconds\n' %
=====================================
examples/tutorial.py
=====================================
@@ -163,6 +163,7 @@ datac2.real = datain['real']
datac2.imag = datain['imag']
print(datac.dtype,datac)
print(datac2.dtype,datac2)
+nc.close()
# more complex compound type example.
nc = Dataset('compound_example.nc','w') # create a new dataset.
=====================================
include/netCDF4.pxi
=====================================
@@ -373,7 +373,7 @@ cdef extern from "mpi-compat.h":
# taken from numpy.pxi in numpy 1.0rc2.
cdef extern from "numpy/arrayobject.h":
ctypedef int npy_intp
- ctypedef extern class numpy.ndarray [object PyArrayObject]:
+ ctypedef extern class numpy.ndarray [object PyArrayObject, check_size ignore]:
pass
npy_intp PyArray_SIZE(ndarray arr) nogil
npy_intp PyArray_ISCONTIGUOUS(ndarray arr) nogil
=====================================
pyproject.toml
=====================================
@@ -1,11 +1,13 @@
[build-system]
requires = [
+ # keep in sync with [dependency-groups.build]
"Cython>=0.29",
"numpy>=2.0.0",
"setuptools>=77.0.1",
"setuptools_scm[toml]>=3.4",
]
-build-backend = "setuptools.build_meta"
+build-backend = "backend"
+backend-path = ["_build"]
[project]
name = "netCDF4"
@@ -13,7 +15,7 @@ description = "Provides an object-oriented python interface to the netCDF versio
authors = [
{name = "Jeff Whitaker", email = "whitaker.jeffrey at gmail.com"},
]
-requires-python = ">=3.10"
+requires-python = ">=3.11"
keywords = [
"numpy", "netcdf", "data", "science", "network", "oceanography",
"meteorology", "climate",
@@ -23,11 +25,11 @@ license-files = ["LICENSE"]
classifiers = [
"Development Status :: 3 - Alpha",
"Programming Language :: Python :: 3",
- "Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
+ "Programming Language :: Python :: 3.15",
"Intended Audience :: Science/Research",
"Topic :: Software Development :: Libraries :: Python Modules",
"Topic :: System :: Archiving :: Compression",
@@ -36,20 +38,37 @@ classifiers = [
dependencies = [
"cftime",
"certifi",
- "numpy>=2.3.0; platform_system == 'Windows' and platform_machine == 'ARM64'",
- "numpy>=1.21.2; platform_system != 'Windows' or platform_machine != 'ARM64'",
+ "packaging",
+ "numpy>=1.23.2",
]
dynamic = ["version"]
[project.optional-dependencies]
-tests = [
- "Cython",
- "packaging",
- "pytest",
- "typing-extensions>=4.15.0",
-]
parallel = [
- "mpi4py",
+ "mpi4py",
+]
+
+[dependency-groups]
+build = [
+ # keep in sync with [build-system.requires]
+ "Cython>=0.29",
+ "numpy>=2.0.0",
+ "setuptools>=77.0.1",
+ "setuptools_scm[toml]>=3.4",
+]
+test = [
+ "pytest",
+]
+typing = [
+ "mypy",
+ "typing-extensions>=4.15.0",
+ "types-setuptools",
+]
+dev = [
+ "mpi4py",
+ {include-group = "build"},
+ {include-group = "test"},
+ {include-group = "typing"},
]
[project.readme]
@@ -112,7 +131,7 @@ build-frontend = "build"
skip = [
"*-musllinux*",
]
-test-extras = "tests"
+test-groups = "test"
test-sources = [
"test",
"pyproject.toml"
@@ -121,8 +140,8 @@ test-command = [
'''python -c "import netCDF4; print(f'netCDF4 v{netCDF4.__version__}')"''',
"pytest -s -rxs -v test",
]
-manylinux-x86_64-image = "ghcr.io/ocefpaf/manylinux_2_28_x86_64-netcdf"
-manylinux-aarch64-image = "ghcr.io/ocefpaf/manylinux_2_28_aarch64-netcdf"
+manylinux-x86_64-image = "ghcr.io/unidata/manylinux_2_28_x86_64-netcdf"
+manylinux-aarch64-image = "ghcr.io/unidata/manylinux_2_28_aarch64-netcdf"
environment = {NETCDF4_LIMITED_API="1"}
[tool.cibuildwheel.macos]
@@ -134,6 +153,7 @@ delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel} \
[tool.cibuildwheel.windows]
before-build = "python -m pip install delvewheel"
+environment = "PATH=$PATH:$CONDA_PREFIX/bin"
repair-wheel-command = [
"delvewheel show --include blosc.dll;zstd.dll;lz4.dll {wheel}",
"delvewheel repair --include blosc.dll;zstd.dll;lz4.dll -w {dest_dir} {wheel}",
@@ -141,17 +161,18 @@ repair-wheel-command = [
[[tool.cibuildwheel.overrides]]
select = "*linux*"
+inherit.environment = "append"
environment = {NETCDF_PLUGIN_DIR="/usr/local/hdf5/lib/plugin/"}
[[tool.cibuildwheel.overrides]]
select = "*-macosx_x86_64"
inherit.environment = "append"
-environment = {MACOSX_DEPLOYMENT_TARGET="13.0",HDF5_DIR="/Users/runner/micromamba/envs/build",netCDF4_DIR="/Users/runner/micromamba/envs/build",PATH="${PATH}:/Users/runner/micromamba/envs/build/bin",NETCDF_PLUGIN_DIR="/Users/runner/micromamba/envs/build/hdf5/lib/plugin"}
+environment = {MACOSX_DEPLOYMENT_TARGET="15.0",HDF5_DIR="/Users/runner/micromamba/envs/build",netCDF4_DIR="/Users/runner/micromamba/envs/build",PATH="${PATH}:/Users/runner/micromamba/envs/build/bin",NETCDF_PLUGIN_DIR="/Users/runner/micromamba/envs/build/hdf5/lib/plugin"}
[[tool.cibuildwheel.overrides]]
select = "*-macosx_arm64"
inherit.environment = "append"
-environment = {MACOSX_DEPLOYMENT_TARGET="14.0",HDF5_DIR="/Users/runner/micromambe/envs/build",netCDF4_DIR="/Users/runner/micromambe/envs/build",PATH="${PATH}:/Users/runner/micromamba/envs/build/bin",NETCDF_PLUGIN_DIR="/Users/runner/micromamba/envs/build/hdf5/lib/plugin"}
+environment = {MACOSX_DEPLOYMENT_TARGET="15.0",HDF5_DIR="/Users/runner/micromambe/envs/build",netCDF4_DIR="/Users/runner/micromambe/envs/build",PATH="${PATH}:/Users/runner/micromamba/envs/build/bin",NETCDF_PLUGIN_DIR="/Users/runner/micromamba/envs/build/hdf5/lib/plugin"}
[[tool.cibuildwheel.overrides]]
select = "*-win_*"
=====================================
setup.py
=====================================
@@ -157,6 +157,8 @@ mpi_incdir = os.environ.get('MPI_INCDIR')
USE_NCCONFIG = bool(int(os.environ.get('USE_NCCONFIG', 0)))
# override use of setup.cfg with env var.
USE_SETUPCFG = bool(int(os.environ.get('USE_SETUPCFG', 1)))
+# disable parallel support even if libs support it (issue #1389)
+DISABLE_PARALLEL_SUPPORT = bool(int(os.environ.get('DISABLE_PARALLEL_SUPPORT', 0)))
setup_cfg = 'setup.cfg'
# contents of setup.cfg will override env vars, unless
@@ -322,7 +324,7 @@ else:
netCDF4_libdir = os.path.join(netCDF4_dir, 'lib')
if sys.platform == 'win32':
- libs.extend(['netcdf', 'zlib'])
+ libs.extend(['netcdf'])
else:
libs.extend(['netcdf', 'z'])
@@ -412,22 +414,27 @@ if 'sdist' not in sys.argv[1:] and 'clean' not in sys.argv[1:] and '--version' n
(netcdf_lib_version > "4.4" and netcdf_lib_version < "4.5"):
has_cdf5_format = True
- has_parallel_support = check_has_parallel_support(inc_dirs)
- has_has_not = "has" if has_parallel_support else "does not have"
- print(f"netcdf lib {has_has_not} parallel functions")
+ if DISABLE_PARALLEL_SUPPORT:
+ has_parallel_support = False
+ print("parallel support disabled via DISABLE_PARALLEL_SUPPORT env var")
+ else:
+ has_parallel_support = check_has_parallel_support(inc_dirs)
+ has_has_not = "has" if has_parallel_support else "does not have"
+ print(f"netcdf lib {has_has_not} parallel functions")
if has_parallel_support:
+ # note(stubbiali): mpi4py is not available when using the in-tree build backend
try:
import mpi4py
except ImportError:
- msg = "Parallel support requires mpi4py but it is not installed."
- raise ImportError(msg)
-
- inc_dirs.append(mpi4py.get_include())
- # mpi_incdir should not be needed if using nc-config
- # (should be included in nc-config --cflags)
- if mpi_incdir is not None:
- inc_dirs.append(mpi_incdir)
+ mpi4py = None
+
+ if mpi4py is not None:
+ inc_dirs.append(mpi4py.get_include())
+ # mpi_incdir should not be needed if using nc-config
+ # (should be included in nc-config --cflags)
+ if mpi_incdir is not None:
+ inc_dirs.append(mpi_incdir)
# Name of file containing imports required for parallel support
parallel_support_imports = "parallel_support_imports.pxi.in"
=====================================
src/netCDF4/__init__.pyi
=====================================
@@ -385,13 +385,13 @@ class Dataset:
def has_zstd_filter(self) -> bool: ...
def has_bzip2_filter(self) -> bool: ...
def has_szip_filter(self) -> bool: ...
- def __getitem__(self, elem: str) -> Any: ... # should be Group | Variable, but this causes too many problems
+ def __getitem__(self, elem: str, /) -> Any: ... # should be Group | Variable, but this causes too many problems
# __iter__ and __contains__ always error because iteration and membership ops are not allowed
def __iter__(self) -> NoReturn: ...
- def __contains__(self, key) -> NoReturn: ...
- def __setattr__(self, name: str, value: Any) -> None: ...
- def __getattr__(self, name: str) -> Any: ...
- def __delattr__(self, name: str): ...
+ def __contains__(self, key, /) -> NoReturn: ...
+ def __setattr__(self, name: str, value: Any, /) -> None: ...
+ def __getattr__(self, name: str, /) -> Any: ...
+ def __delattr__(self, name: str, /): ...
def __reduce__(self) -> NoReturn: ...
def __enter__(self) -> Self: ...
def __exit__(self, atype, value, traceback) -> None: ...
@@ -585,11 +585,11 @@ class Variable(Generic[VarT]):
def set_ncstring_attrs(self, ncstring_attrs: bool) -> None: ...
def set_collective(self, value: bool) -> None: ...
def get_dims(self) -> tuple[Dimension, ...]: ...
- def __delattr__(self, name: str) -> None: ...
- def __setattr__(self, name: str, value: Any) -> None: ...
- def __getattr__(self, name: str) -> Any: ...
- def __getitem__(self, elem: Any) -> Any: ...
- def __setitem__(self, elem: Any, data: npt.ArrayLike) -> None: ...
+ def __delattr__(self, name: str, /) -> None: ...
+ def __setattr__(self, name: str, value: Any, /) -> None: ...
+ def __getattr__(self, name: str, /) -> Any: ...
+ def __getitem__(self, elem: Any, /) -> Any: ...
+ def __setitem__(self, elem: Any, data: npt.ArrayLike, /) -> None: ...
def __array__(self) -> np.ndarray: ...
def __len__(self) -> int: ...
def __iter__(self) -> Iterator[Any]: ... # faux method so mypy believes Variable is iterable
@@ -704,7 +704,8 @@ def stringtochar(
@overload
def stringtochar(
a: npt.NDArray[np.character],
- encoding: str = ...,
+ encoding: str | None = None,
+ n_strlen: int | None = None,
) -> npt.NDArray[np.str_] | npt.NDArray[np.bytes_]: ...
@overload
def chartostring(
@@ -714,7 +715,7 @@ def chartostring(
@overload
def chartostring(
b: npt.NDArray[np.character],
- encoding: str = ...,
+ encoding: str | None = None,
) -> npt.NDArray[np.str_] | npt.NDArray[np.bytes_]: ...
def getlibversion() -> str: ...
def rc_get(key: str) -> str | None: ...
=====================================
src/netCDF4/_netCDF4.pyx
=====================================
@@ -1,4 +1,4 @@
-"""Version 1.7.4
+"""Version 1.7.5
-------------
# Introduction
@@ -1050,7 +1050,7 @@ are collective. There are a couple of important limitations of parallel IO:
to write to it.
- You cannot use variable-length (VLEN) data types.
-***Import warning regarding threads:*** The underlying netcdf-c library is not thread-safe, so netcdf4-python cannot perform parallel
+***Important warning regarding threads:*** The underlying netcdf-c library is not thread-safe, so netcdf4-python cannot perform parallel
IO in a multi-threaded environment. Users should expect segfaults if a netcdf file is opened on multiple threads - care should
be taken to restrict netcdf4-python usage to a single thread, even when using free-threaded python.
@@ -1283,11 +1283,12 @@ import sys
import functools
from typing import Union
-__version__ = "1.7.4"
+__version__ = "1.7.5"
# Initialize numpy
import posixpath
from cftime import date2num, num2date, date2index
+from packaging.version import Version
import numpy
cimport numpy
import weakref
@@ -1888,6 +1889,9 @@ be raised in the next release."""
raise TypeError, 'illegal data type for attribute %r, must be one of %s, got %s' % (attname, _supportedtypes, value_arr.dtype.str[1:])
elif xtype == -99: # if xtype is not passed in as kwarg.
xtype = _nptonctype[value_arr.dtype.str[1:]]
+ # netcdf-c expects native-endian buffers for primitive numeric types.
+ if value_arr.dtype.kind in 'iuf' and not value_arr.dtype.isnative:
+ value_arr = value_arr.astype(value_arr.dtype.newbyteorder('='))
lenarr = PyArray_SIZE(value_arr)
with nogil:
ierr = nc_put_att(grpid, varid, attname, xtype, lenarr,
@@ -2410,7 +2414,7 @@ strings.
# flag to indicate that Variables in this Dataset support orthogonal indexing.
self.__orthogonal_indexing__ = True
- if diskless and __netcdf4libversion__ < '4.2.1':
+ if diskless and Version(__netcdf4libversion__) < Version('4.2.1'):
#diskless = False # don't raise error, instead silently ignore
raise ValueError('diskless mode requires netcdf lib >= 4.2.1, you have %s' % __netcdf4libversion__)
# convert filename into string (from os.path object for example),
@@ -4439,7 +4443,6 @@ behavior is similar to Fortran or Matlab, but different than numpy.
elif fill_value == 'default':
if self._isprimitive:
fillval = numpy.array(default_fillvals[self.dtype.str[1:]])
- if not fillval.dtype.isnative: fillval.byteswap(True)
_set_att(self._grp, self._varid, '_FillValue',\
fillval, xtype=xtype)
else:
@@ -4455,7 +4458,6 @@ does not do anything."""
_tostr(fill_value), xtype=xtype, force_ncstring=True)
else:
fillval = numpy.array(fill_value, self.dtype)
- if not fillval.dtype.isnative: fillval.byteswap(True)
_set_att(self._grp, self._varid, '_FillValue',\
fillval, xtype=xtype)
else:
@@ -5019,13 +5021,10 @@ details."""
# raise AttributeError(msg)
elif name in ['valid_min','valid_max','valid_range','missing_value'] and self._isprimitive:
# make sure these attributes written in same data type as variable.
- # also make sure it is written in native byte order
- # (the same as the data)
valuea = numpy.array(value, self.dtype)
# check to see if array cast is safe
if _safecast(numpy.array(value),valuea):
value = valuea
- if not value.dtype.isnative: value.byteswap(True)
else: # otherwise don't do it, but issue a warning
msg="WARNING: %s cannot be safely cast to variable dtype" \
% name
@@ -5104,6 +5103,8 @@ rename a `Variable` attribute named `oldname` to `newname`."""
# Fill output array with data chunks.
for (a,b,c,i) in zip(start, count, stride, put_ind):
+ if 0 in b:
+ continue # retain the empty shape without issuing native I/O
datout = self._get(a,b,c)
if not hasattr(datout,'shape') or data.shape == datout.shape:
data = datout
@@ -5186,8 +5187,8 @@ rename a `Variable` attribute named `oldname` to `newname`."""
if len(data.shape) > 0 and data.shape[-1] == self.shape[-1]:
# also make sure slice is along last dimension
matchdim = True
- for cnt in count:
- if cnt[-1] != self.shape[-1]:
+ for cnt, ind in zip(count, put_ind):
+ if cnt[-1] != self.shape[-1] or (0 in cnt and ind[-1] != slice(None)):
matchdim = False
break
if matchdim:
@@ -5626,11 +5627,11 @@ cannot be safely cast to variable data type""" % attname
data = numpy.tile(data,datashape)
# reshape data array if needed to conform with start,count,stride.
if data.ndim != len(datashape) or\
- (data.shape != datashape and data.ndim > 1): # issue #1083
+ (data.shape != datashape and (data.ndim > 1 or 0 in datashape)): # issue #1083
# create a view so shape in caller is not modified (issue 90)
try: # if extra singleton dims, just reshape
data = data.view()
- data.shape = tuple(datashape)
+ data = data.reshape(tuple(datashape))
except ValueError: # otherwise broadcast
data = numpy.broadcast_to(data, datashape)
@@ -6788,7 +6789,7 @@ returns a rank 1 numpy character array of length NUMCHARS with datatype `'S1'`
arr[0:len(string)] = tuple(string)
return arr
-def stringtochar(a,encoding='utf-8',n_strlen=None):
+def stringtochar(a,encoding=None,n_strlen=None):
"""
**`stringtochar(a,encoding='utf-8',n_strlen=None)`**
@@ -6799,8 +6800,8 @@ is the number of characters in each string. Will be converted to
an array of characters (datatype `'S1'` or `'U1'`) of shape `a.shape + (N,)`.
optional kwarg `encoding` can be used to specify character encoding (default
-`utf-8`). If `encoding` is 'none' or 'bytes', a `numpy.string_` the input array
-is treated a raw byte strings (`numpy.string_`).
+`utf-8` for dtype=`'UN'` or `ascii` for dtype=`'SN'`). If `encoding` is 'none' or 'bytes',
+a `numpy.string_` the input array is treated a raw byte strings (`numpy.string_`).
optional kwarg `n_strlen` is the number of characters in each string. Default
is None, which means `n_strlen` will be set to a.itemsize (the number of bytes
@@ -6809,15 +6810,19 @@ used to represent each string in the input array).
returns a numpy character array with datatype `'S1'` or `'U1'`
and shape `a.shape + (N,)`, where N is the length of each string in a."""
dtype = a.dtype.kind
- if n_strlen is None:
- n_strlen = a.dtype.itemsize
if dtype not in ["S","U"]:
raise ValueError("type must string or unicode ('S' or 'U')")
+ if encoding is None:
+ if dtype == 'S':
+ encoding = 'ascii'
+ else:
+ encoding = 'utf-8'
+ if n_strlen is None:
+ n_strlen = a.dtype.itemsize
if encoding in ['none','None','bytes']:
b = numpy.array(tuple(a.tobytes()),'S1')
elif encoding == 'ascii':
- b = numpy.array(tuple(a.tobytes().decode(encoding)),dtype+'1')
- b.shape = a.shape + (n_strlen,)
+ b = (numpy.array(tuple(a.tobytes().decode(encoding)),dtype+'1')).reshape(a.shape + (n_strlen,))
else:
if not a.ndim:
a = numpy.array([a])
@@ -6827,7 +6832,7 @@ and shape `a.shape + (N,)`, where N is the length of each string in a."""
b = numpy.array([[bb[i:i+1] for i in range(n_strlen)] for bb in bbytes])
return b
-def chartostring(b,encoding='utf-8'):
+def chartostring(b,encoding=None):
"""
**`chartostring(b,encoding='utf-8')`**
@@ -6838,22 +6843,26 @@ Will be converted to a array of strings, where each string has a fixed
length of `b.shape[-1]` characters.
optional kwarg `encoding` can be used to specify character encoding (default
-`utf-8`). If `encoding` is 'none' or 'bytes', a `numpy.string_` byte array is
-returned.
+`utf-8` for dtype=`'UN'` or `ascii` for dtype=`'SN'`). If `encoding` is 'none' or 'bytes',
+a `numpy.string_` byte array is returned.
returns a numpy string array with datatype `'UN'` (or `'SN'`) and shape
`b.shape[:-1]` where where `N=b.shape[-1]`."""
dtype = b.dtype.kind
if dtype not in ["S","U"]:
raise ValueError("type must be string or unicode ('S' or 'U')")
+ if encoding is None:
+ if dtype == 'S':
+ encoding = 'ascii'
+ else:
+ encoding = 'utf-8'
bs = b.tobytes()
slen = int(b.shape[-1])
if encoding in ['none','None','bytes']:
a = numpy.array([bs[n1:n1+slen] for n1 in range(0,len(bs),slen)],'S'+repr(slen))
else:
a = numpy.array([bs[n1:n1+slen].decode(encoding) for n1 in range(0,len(bs),slen)],'U'+repr(slen))
- a.shape = b.shape[:-1]
- return a
+ return a.reshape(b.shape[:-1])
class MFDataset(Dataset):
"""
@@ -7236,8 +7245,8 @@ class _Variable:
# is a perfect match for the "start", "count" and "stride"
# arguments to the nc_get_var() function, and is much more easy
# to use.
- start, count, stride, put_ind =\
- _StartCountStride(elem, self.shape)
+ start, count, stride, put_ind, selection =\
+ _StartCountStride(elem, self.shape, return_selection=True)
datashape = _out_array_shape(count)
data = ma.empty(datashape, dtype=self.dtype)
@@ -7264,6 +7273,51 @@ class _Variable:
count = [abs(cnt) for cnt in count]
if (numpy.array(stride) < 0).any():
raise IndexError('negative strides not allowed when slicing MFVariable Variable instance')
+ if 0 in count:
+ # An empty underlying read retains automatic unpacking and
+ # unsigned conversion without reading or concatenating records.
+ empty = (slice(0, 0),) + (slice(None),) * (self.ndim - 1)
+ if self.dtype.kind == 'S' and (count[-1] != self.shape[-1] or ind[-1] != slice(None)):
+ # A partial, scalar or fancy character selection is not
+ # eligible for automatic conversion of the full width.
+ empty = empty[:-1] + (slice(0, 0),)
+ records = selection[0]
+ selected = []
+ offset = 0
+ if isinstance(records, slice):
+ beg, end, inc = records.indices(self.shape[0])
+ for n, length in enumerate(self._recLen):
+ # Intersect an arithmetic progression with each file's
+ # record interval without materializing all records.
+ lower = max(beg, offset)
+ first = beg + ((lower - beg + inc - 1) // inc) * inc
+ if first < min(end, offset + length):
+ selected.append(n)
+ offset += length
+ else:
+ # Non-slice fancy indexing processes one record chunk at a
+ # time; preserve the final chunk's dtype, as for nonempty
+ # reads. Scalars select their own file by the same rule.
+ record = records[-1] if numpy.iterable(records) else records
+ if record < 0:
+ record += self.shape[0]
+ for n, length in enumerate(self._recLen):
+ if offset <= record < offset + length:
+ selected.append(n)
+ break
+ offset += length
+ # With no selected records there is no contributing file;
+ # retain the first file's empty-decoding convention.
+ if not selected:
+ selected = [0]
+ decoded = [Variable.__getitem__(self._recVar[n], empty) for n in selected]
+ if any(part.ndim != decoded[0].ndim for part in decoded):
+ raise ValueError('all the input arrays must have same number of dimensions')
+ data = data.astype(numpy.result_type(*(part.dtype for part in decoded)))
+ if decoded[0].ndim < self.ndim:
+ data = data.reshape(datashape[:-1])
+ squeeze = squeeze[:-1]
+ continue
# Start, stop and step along 1st dimension, eg the unlimited
# dimension.
sta = start[0]
=====================================
src/netCDF4/utils.py
=====================================
@@ -79,7 +79,7 @@ least_significant_digit=1, bits will be 4.
return datout
def _StartCountStride(elem, shape, dimensions=None, grp=None, datashape=None,\
- put=False, use_get_vars = True):
+ put=False, use_get_vars = True, return_selection=False):
"""Return start, count, stride and indices needed to store/extract data
into/from a netCDF variable.
@@ -134,6 +134,9 @@ def _StartCountStride(elem, shape, dimensions=None, grp=None, datashape=None,\
datashape : sequence
The shape of the data that is being stored. Only needed by __setitem__
put : True|False (default False). If called from __setitem__, put is True.
+ return_selection : bool (default False)
+ Also return the normalized selectors, for callers that need their exact
+ indices after an empty selection has been represented by compact chunks.
Returns
-------
@@ -148,6 +151,8 @@ def _StartCountStride(elem, shape, dimensions=None, grp=None, datashape=None,\
indices : ndarray (..., n)
An array storing the indices describing the location of the
data chunk in the target/source array (__getitem__/__setitem__).
+ selection : list, optional
+ Normalized selectors, included only when return_selection is True.
Notes:
@@ -261,6 +266,13 @@ Boolean array must have the same shape as the data along this dimension."""
# integer scalar
elif ea.dtype.kind == 'i':
newElem.append(e)
+ # Scalar floats retain compatibility when they represent an exact
+ # integer, but non-integral values are invalid indices. Letting
+ # int(e) handle both cases silently truncates values such as 0.8.
+ elif ea.dtype.kind == 'f' and not np.iterable(ea):
+ if not np.isfinite(e) or e != np.floor(e):
+ raise IndexError(IndexErrorMsg)
+ newElem.append(int(e))
# slice or ellipsis object
elif type(e) == slice or type(e) == type(Ellipsis):
if not use_get_vars and type(e) == slice and e.step not in [None,-1,1] and\
@@ -322,6 +334,10 @@ Boolean array must have the same shape as the data along this dimension."""
newElem.append(e)
elif np.iterable(e) and len(e) == 1:
newElem.append(slice(e[0], e[0] + 1, 1))
+ elif np.iterable(e) and len(e) == 0:
+ # Retain counts and scalar squeeze markers for the other axes.
+ # A zero-length chunk axis would discard all of that metadata.
+ newElem.append(slice(0, 0, 1))
else:
newElem.append(e)
elem = newElem
@@ -358,8 +374,23 @@ Boolean array must have the same shape as the data along this dimension."""
# pad datashape with zeros for dimensions not being sliced (issue #906)
# only used when data covers slice over subset of dimensions
+ padding_dims = sum(1 for e in elem if type(e) == slice)
+ if datashape and len(datashape) != len(elem):
+ for i, e in enumerate(elem):
+ if type(e) != slice:
+ continue
+ length = shape[i]
+ if hasunlim and unlimd[dimensions[i]]:
+ if e.stop is None:
+ continue # an open-ended write may extend this dimension
+ length = max(length, e.stop)
+ if not range(*e.indices(length)):
+ # Empty writes retain fancy axes too. Keep the historical
+ # padding rules for nonempty writes outside this correction.
+ padding_dims = sum(1 for index in elem if type(index) == slice or np.iterable(index))
+ break
if datashape and len(datashape) != len(elem) and\
- len(datashape) == sum(1 for e in elem if type(e) == slice):
+ len(datashape) == padding_dims:
datashapenew = (); i=0
for e in elem:
if type(e) != slice and not np.iterable(e): # scalar integer slice
@@ -369,13 +400,14 @@ Boolean array must have the same shape as the data along this dimension."""
i+=1
datashape = datashapenew
- # Create the start, count, stride and indices arrays.
-
+ # First describe one chunk with the complete selected shape. Only expand
+ # integer index arrays into their Cartesian chunk grid if it is nonempty.
sdim.append(max(nDims, 1))
- start = np.empty(sdim, dtype=np.intp)
- count = np.empty(sdim, dtype=np.intp)
- stride = np.empty(sdim, dtype=np.intp)
- indices = np.empty(sdim, dtype=object)
+ compact = [1] * nDims + [nDims]
+ start = np.empty(compact, dtype=np.intp)
+ count = np.empty(compact, dtype=np.intp)
+ stride = np.empty(compact, dtype=np.intp)
+ indices = np.empty(compact, dtype=object)
for i, e in enumerate(elem):
@@ -426,11 +458,10 @@ Boolean array must have the same shape as the data along this dimension."""
# ITERABLE #
elif np.iterable(e) and np.array(e).dtype.kind in 'i': # Sequence of integers
- if start[...,i].size:
- start[...,i] = np.apply_along_axis(lambda x: e*x, i, np.ones(sdim[:-1]))
- indices[...,i] = np.apply_along_axis(lambda x: np.arange(sdim[i])*x, i, np.ones(sdim[:-1], int))
- count[...,i] = 1
- stride[...,i] = 1
+ start[...,i] = e[0]
+ count[...,i] = len(e)
+ stride[...,i] = 1
+ indices[...,i] = 0 # preserve the distinction from a full slice
# all that's left is SCALAR INTEGER #
else:
@@ -446,6 +477,25 @@ Boolean array must have the same shape as the data along this dimension."""
indices[...,i] = -1 # Use -1 instead of 0 to indicate that
# this dimension shall be squeezed.
+ if 0 in count and not put:
+ # Empty reads bypass native bounds checks, including those for vlens.
+ for i, e in enumerate(elem):
+ if type(e) != slice and not np.iterable(e) and e >= shape[i]:
+ raise IndexError("integer index exceeds dimension size")
+
+ if 0 not in count and sdim != compact:
+ start = np.broadcast_to(start, sdim).copy()
+ count = np.broadcast_to(count, sdim).copy()
+ stride = np.broadcast_to(stride, sdim).copy()
+ indices = np.broadcast_to(indices, sdim).copy()
+ for i, e in enumerate(elem):
+ if np.iterable(e):
+ start[...,i] = np.apply_along_axis(lambda x: e*x, i, np.ones(sdim[:-1]))
+ indices[...,i] = np.apply_along_axis(lambda x: np.arange(sdim[i])*x, i, np.ones(sdim[:-1], int))
+ count[...,i] = 1
+
+ if return_selection:
+ return start, count, stride, indices, elem
return start, count, stride, indices#, out_shape
def _out_array_shape(count):
@@ -514,14 +564,14 @@ def ncinfo():
try:
opts, pargs = getopt.getopt(sys.argv[1:],'hv:g:d:',
- ['group=',
+ ['help',
+ 'group=',
'variable=',
'dimension='])
- except:
- (type, value, traceback) = sys.exc_info()
- sys.stdout.write("Error parsing the options. The error was: %s\n" % value)
+ except getopt.GetoptError as error:
+ sys.stdout.write("Error parsing the options. The error was: %s\n" % error)
sys.stderr.write(usage)
- sys.exit(0)
+ sys.exit(2)
# Get the options
group = None; var = None; dim=None
@@ -538,7 +588,7 @@ def ncinfo():
else:
sys.stdout.write("%s: Unrecognized option\n" % option[0])
sys.stderr.write(usage)
- sys.exit(0)
+ sys.exit(2)
# filename passed as last argument
try:
@@ -546,7 +596,7 @@ def ncinfo():
except IndexError:
sys.stdout.write("You need to pass netcdf filename!\n.")
sys.stderr.write(usage)
- sys.exit(0)
+ sys.exit(2)
f = Dataset(filename)
if group is None:
@@ -565,7 +615,7 @@ def ncinfo():
if var is not None:
print(g.variables[var])
if dim is not None:
- print(g.dimensions[var])
+ print(g.dimensions[dim])
f.close()
def _nc4tonc3(filename4,filename3,clobber=False,nchunk=10,quiet=False,format='NETCDF3_64BIT'):
@@ -653,16 +703,15 @@ def nc4tonc3():
try:
opts, pargs = getopt.getopt(sys.argv[1:], 'ho',
- ['format=','chunk=','quiet='])
- except:
- (type, value, traceback) = sys.exc_info()
- sys.stdout.write("Error parsing the options. The error was: %s\n" % value)
+ ['help','format=','chunk=','quiet='])
+ except getopt.GetoptError as error:
+ sys.stdout.write("Error parsing the options. The error was: %s\n" % error)
sys.stderr.write(usage)
- sys.exit(0)
+ sys.exit(2)
# default options
quiet = 0
- chunk = 1000
+ chunk = 10
format = 'NETCDF3_64BIT'
overwritefile = 0
@@ -680,22 +729,22 @@ def nc4tonc3():
elif option[0] == '--chunk':
chunk = int(option[1])
else:
- sys.stdout.write("%s : Unrecognized option\n" % options[0])
+ sys.stdout.write("%s : Unrecognized option\n" % option[0])
sys.stderr.write(usage)
- sys.exit(0)
+ sys.exit(2)
# if we pass a number of files different from 2, abort
if len(pargs) < 2 or len(pargs) > 2:
sys.stdout.write("You need to pass both source and destination!\n.")
sys.stderr.write(usage)
- sys.exit(0)
+ sys.exit(2)
# Catch the files passed as the last arguments
filename4 = pargs[0]
filename3 = pargs[1]
# copy the data from filename4 to filename3.
- _nc4tonc3(filename4,filename3,clobber=overwritefile,quiet=quiet,format=format)
+ _nc4tonc3(filename4,filename3,clobber=overwritefile,nchunk=chunk,quiet=quiet,format=format)
def _nc3tonc4(filename3,filename4,unpackshort=True,
@@ -856,7 +905,8 @@ def nc3tonc4():
try:
opts, pargs = getopt.getopt(sys.argv[1:], 'ho',
- ['classic=',
+ ['help',
+ 'classic=',
'vars=',
'zlib=',
'quiet=',
@@ -868,11 +918,10 @@ def nc3tonc4():
'chunk=',
'istart=',
'istop='])
- except:
- (type, value, traceback) = sys.exc_info()
- sys.stdout.write("Error parsing the options. The error was: %s\n" % value)
+ except getopt.GetoptError as error:
+ sys.stdout.write("Error parsing the options. The error was: %s\n" % error)
sys.stderr.write(usage)
- sys.exit(0)
+ sys.exit(2)
# default options
overwritefile = 0
@@ -923,13 +972,13 @@ def nc3tonc4():
else:
sys.stdout.write("%s: Unrecognized option\n" % option[0])
sys.stderr.write(usage)
- sys.exit(0)
+ sys.exit(2)
# if we pass a number of files different from 2, abort
if len(pargs) < 2 or len(pargs) > 2:
sys.stdout.write("You need to pass both source and destination!.\n")
sys.stderr.write(usage)
- sys.exit(0)
+ sys.exit(2)
# Catch the files passed as the last arguments
filename3 = pargs[0]
=====================================
test/test_atts.py
=====================================
@@ -6,6 +6,8 @@ import os
import tempfile
import warnings
import pathlib
+import struct
+from typing import Literal
import numpy as np
from collections import OrderedDict
@@ -223,5 +225,122 @@ class VariablesTestCase(unittest.TestCase):
with netCDF4.Dataset(pathlib.Path(__file__).parent / "test_gold.nc") as f:
assert f['RADIANCE'].VAR_NOTES == ""
+class NumericEndianTestCase(unittest.TestCase):
+
+ formats: 'tuple[netCDF4.Format, ...]' = (
+ 'NETCDF4', 'NETCDF4_CLASSIC', 'NETCDF3_CLASSIC', 'NETCDF3_64BIT_OFFSET')
+ if netCDF4.__has_cdf5_format__ and struct.calcsize('P') >= 8:
+ formats += ('NETCDF3_64BIT_DATA',)
+
+ def setUp(self):
+ directory = tempfile.TemporaryDirectory()
+ self.addCleanup(directory.cleanup)
+ self.path = os.path.join(directory.name, 'attributes.nc')
+
+ def test_numeric_attribute_byte_order(self):
+ # Issue #1502: nc_put_att expects native-endian numeric buffers.
+ for format in self.formats:
+ extended = format in ('NETCDF4', 'NETCDF3_64BIT_DATA')
+ dtypes = ['i1', 'i2', 'i4', 'i8', 'f4', 'f8']
+ if extended:
+ dtypes += ['u1', 'u2', 'u4', 'u8']
+ expected = {}
+ with netCDF4.Dataset(self.path, 'w', format=format) as f:
+ targets: dict[str, netCDF4.Dataset | netCDF4.Group | netCDF4.Variable] = {
+ '/': f, '/v': f.createVariable('v', 'f8')}
+ if format == 'NETCDF4':
+ group = f.createGroup('g')
+ targets['/g'] = group
+ targets['/g/v'] = group.createVariable('v', 'f8')
+ for dtype in dtypes:
+ values = [1, 2, 17] if dtype[0] == 'u' else [1, -2, 17]
+ if dtype[0] == 'f':
+ values = [1.25, -2.5, 17.75]
+ for order in ('=', '<', '>'):
+ array = np.array(values, dtype=np.dtype(dtype).newbyteorder(order))
+ readonly = array.copy()
+ readonly.flags.writeable = False
+ inputs = (array, np.array(values[0], dtype=array.dtype),
+ np.repeat(array, 2)[::2], array[::-1],
+ readonly, array[:0])
+ for layout, value in enumerate(inputs):
+ original = value.tobytes()
+ for target_name, target in targets.items():
+ for setter in ('single', 'bulk', 'syntax'):
+ name = '%s_%s_%s_%s' % (dtype, order, layout, setter)
+ with self.subTest(format=format, target=target_name,
+ attribute=name):
+ if setter == 'single':
+ target.setncattr(name, value)
+ elif setter == 'bulk':
+ target.setncatts({name: value})
+ else:
+ setattr(target, name, value)
+ self.assertEqual(value.tobytes(), original)
+ self.assertEqual(value.dtype, array.dtype)
+ self.assertEqual(value.flags.writeable,
+ layout != 4)
+ stored_dtype = 'i4' if dtype == 'i8' and not extended else dtype
+ expected[target_name, name] = value.astype(stored_dtype)
+ with netCDF4.Dataset(self.path) as f:
+ for (target_name, name), value in expected.items():
+ target = f if target_name == '/' else f[target_name]
+ with self.subTest(format=format, target=target_name, attribute=name):
+ actual = np.asarray(target.getncattr(name))
+ np.testing.assert_array_equal(actual, value)
+ self.assertEqual(actual.dtype, value.dtype)
+
+ def test_valid_range_byte_order(self):
+ for format in self.formats:
+ with self.subTest(format=format):
+ value = np.array([1., 2.], dtype=np.dtype('f8').newbyteorder('S'))
+ with netCDF4.Dataset(self.path, 'w', format=format) as f:
+ f.createDimension('x', 4)
+ v = f.createVariable('v', 'f8', ('x',))
+ v[:] = [0., 1., 2., 3.]
+ v.setncattr('valid_range', value)
+ f.setncattr('levels', value)
+ with netCDF4.Dataset(self.path) as f:
+ v = f['v']
+ np.testing.assert_array_equal(f.levels, [1., 2.])
+ np.testing.assert_array_equal(v.valid_range, [1., 2.])
+ np.testing.assert_array_equal(np.ma.getmaskarray(v[:]),
+ [True, False, False, True])
+ np.testing.assert_array_equal(v[:].compressed(), [1., 2.])
+
+ def test_variable_metadata_byte_order(self):
+ # Existing fill-value and special-attribute callers must not swap twice.
+ endians: tuple[Literal['little', 'big'], ...] = ('little', 'big')
+ for endian in endians:
+ with self.subTest(endian=endian):
+ dtype = np.dtype('f8').newbyteorder(endian)
+ with netCDF4.Dataset(self.path, 'w') as f:
+ f.createDimension('x', 3)
+ v = f.createVariable('v', dtype, ('x',), endian=endian,
+ fill_value=-999.)
+ v[:2] = [1., 2.]
+ v.valid_min = np.array(0., dtype=dtype)
+ v.valid_max = np.array(3., dtype=dtype)
+ v.valid_range = np.array([0., 3.], dtype=dtype)
+ v.missing_value = np.array(-999., dtype=dtype)
+ with netCDF4.Dataset(self.path) as f:
+ v = f['v']
+ self.assertEqual(v._FillValue, -999.)
+ self.assertEqual(v.missing_value, -999.)
+ self.assertEqual(v.valid_min, 0.)
+ self.assertEqual(v.valid_max, 3.)
+ np.testing.assert_array_equal(v.valid_range, [0., 3.])
+ np.testing.assert_array_equal(np.ma.getmaskarray(v[:]),
+ [False, False, True])
+ np.testing.assert_array_equal(v[:].compressed(), [1., 2.])
+
+ def test_unsupported_numeric_attributes(self):
+ with netCDF4.Dataset(self.path, 'w') as f:
+ for dtype in ('?', '<f2', '>f2', '<c8', '>c8', '<c16', '>c16'):
+ with self.subTest(dtype=dtype):
+ with self.assertRaises(TypeError):
+ f.setncattr('unsupported', np.array([1, 0], dtype=dtype))
+
+
if __name__ == '__main__':
unittest.main()
=====================================
test/test_cli.py
=====================================
@@ -0,0 +1,81 @@
+import sys
+from unittest.mock import patch
+
+import numpy as np
+import pytest
+
+from netCDF4 import Dataset, utils
+
+
+ at pytest.mark.parametrize("command", [utils.ncinfo, utils.nc3tonc4, utils.nc4tonc3])
+ at pytest.mark.parametrize("args", [[], ["--unknown"]])
+def test_usage_error(command, args, monkeypatch, capsys):
+ monkeypatch.setattr(sys, "argv", [command.__name__, *args])
+ with pytest.raises(SystemExit) as exc:
+ command()
+ assert exc.value.code == 2
+ assert "usage:" in capsys.readouterr().err
+
+
+ at pytest.mark.parametrize("command, option", [
+ (utils.ncinfo, "--group"),
+ (utils.nc3tonc4, "--chunk"),
+ (utils.nc4tonc3, "--chunk"),
+])
+def test_missing_option_value(command, option, monkeypatch, capsys):
+ monkeypatch.setattr(sys, "argv", [command.__name__, option])
+ with pytest.raises(SystemExit) as exc:
+ command()
+ assert exc.value.code == 2
+ assert "usage:" in capsys.readouterr().err
+
+
+ at pytest.mark.parametrize("command", [utils.nc3tonc4, utils.nc4tonc3])
+ at pytest.mark.parametrize("args", [["source.nc"], ["source.nc", "dest.nc", "extra.nc"]])
+def test_converter_file_count(command, args, monkeypatch, capsys):
+ monkeypatch.setattr(sys, "argv", [command.__name__, *args])
+ with pytest.raises(SystemExit) as exc:
+ command()
+ assert exc.value.code == 2
+ assert "usage:" in capsys.readouterr().err
+
+
+ at pytest.mark.parametrize("command", [utils.ncinfo, utils.nc3tonc4, utils.nc4tonc3])
+ at pytest.mark.parametrize("option", ["-h", "--help"])
+def test_help(command, option, monkeypatch, capsys):
+ monkeypatch.setattr(sys, "argv", [command.__name__, option])
+ with pytest.raises(SystemExit) as exc:
+ command()
+ assert exc.value.code == 0
+ output = capsys.readouterr()
+ assert "usage:" in output.err
+ assert "Error" not in output.out
+
+
+def test_ncinfo_group_dimension(tmp_path, monkeypatch, capsys):
+ filename = str(tmp_path / "groups.nc")
+ with Dataset(filename, "w") as dataset:
+ dataset.createGroup("g1").createDimension("x", 7)
+ monkeypatch.setattr(sys, "argv", ["ncinfo", "-g", "g1", "-d", "x", filename])
+ utils.ncinfo()
+ assert "name = 'x', size = 7" in capsys.readouterr().out
+
+
+ at pytest.mark.parametrize("chunk", [None, 0, 5])
+def test_nc4tonc3_chunk(tmp_path, chunk, monkeypatch):
+ source = str(tmp_path / "source.nc")
+ destination = str(tmp_path / "destination.nc")
+ values = np.arange(25, dtype="i4")
+ with Dataset(source, "w", format="NETCDF4_CLASSIC") as dataset:
+ dataset.createDimension("record", None)
+ dataset.createVariable("values", "i4", ("record",))[:] = values
+ args = ["nc4tonc3", "--quiet=1"]
+ if chunk is not None:
+ args.append("--chunk=%s" % chunk)
+ monkeypatch.setattr(sys, "argv", [*args, source, destination])
+ with patch.object(utils, "_nc4tonc3", wraps=utils._nc4tonc3) as convert:
+ utils.nc4tonc3()
+ assert convert.call_args.kwargs["nchunk"] == (10 if chunk is None else chunk)
+ with Dataset(destination) as dataset:
+ assert dataset.file_format == "NETCDF3_64BIT_OFFSET"
+ np.testing.assert_array_equal(dataset.variables["values"][:], values)
=====================================
test/test_dap.py
=====================================
@@ -8,9 +8,8 @@ import os
# test accessing data over http with opendap.
yesterday = datetime.now() - timedelta(days=1)
-URL = f'http://nomads.ncep.noaa.gov/dods/gfs_1p00/gfs{yesterday:%Y%m%d}/gfs_1p00_00z'
-URL_https = 'https://www.neracoos.org/erddap/griddap/WW3_EastCoast_latest'
-varname = 'hgtsfc'
+URL = f'https://tds.scigw.unidata.ucar.edu/thredds/dodsC/grib/NCEP/GFS/Global_onedegree_noaaport/GFS_Global_onedeg_noaaport_{yesterday:%Y%m%d}_1800.grib2'
+varname = 'Geopotential_height_surface'
data_min = -40; data_max = 5900
varshape = (181, 360)
@@ -24,7 +23,7 @@ class DapTestCase(unittest.TestCase):
pass
def runTest(self):
- """testing access of data over http using opendap"""
+ """testing access of data over https using opendap"""
ncfile = netCDF4.Dataset(URL)
assert varname in ncfile.variables.keys()
var = ncfile.variables[varname]
@@ -33,10 +32,6 @@ class DapTestCase(unittest.TestCase):
assert np.abs(data.min()-data_min) < 10
assert np.abs(data.max()-data_max) < 100
ncfile.close()
- # test https support (linked curl lib must built with openssl support)
- ncfile = netCDF4.Dataset(URL_https)
- assert ncfile['hs'].long_name=='Significant Wave Height'
- ncfile.close()
if __name__ == '__main__':
unittest.main()
=====================================
test/test_empty_indexing.py
=====================================
@@ -0,0 +1,452 @@
+"""Empty orthogonal selections retain all unselected dimensions."""
+import os
+import tempfile
+import unittest
+from typing import Any
+
+import numpy as np
+from numpy.testing import assert_array_equal
+from netCDF4 import Dataset, MFDataset
+
+
+EMPTY = np.array([], dtype=np.intp)
+SHAPE = (3, 5, 7)
+CASES = [
+ (EMPTY, slice(None), slice(None)),
+ (slice(None), EMPTY, slice(None)),
+ (slice(None), slice(None), EMPTY),
+ (np.zeros(3, dtype=bool), slice(None), slice(None)),
+ (slice(None), np.zeros(5, dtype=bool), slice(None)),
+ (slice(None), slice(None), np.zeros(7, dtype=bool)),
+ (EMPTY, slice(None, None, -1), slice(None, None, 2)),
+ (slice(None, None, 2), EMPTY, slice(None, None, -2)),
+ (1, EMPTY, slice(None, None, -1)),
+ (EMPTY, 2, slice(None)),
+ (slice(None), EMPTY, 3),
+ (EMPTY, EMPTY, slice(None)),
+ (EMPTY, slice(None), EMPTY),
+ (EMPTY, EMPTY, EMPTY),
+ (EMPTY, [4, 1, 4], [6, 2]),
+ ([2, 0, 2], EMPTY, [6, 1, 6]),
+ (slice(0), slice(None), slice(None)),
+ (slice(None), slice(0), slice(None)),
+ (slice(None), slice(None), slice(0)),
+ ([2, 0, 2], [4, 1, 4], [6, 2]),
+ (slice(None), slice(None), slice(None)),
+ (1, slice(None, None, -1), slice(None, None, 2)),
+]
+
+
+def orthogonal_selection(data, key):
+ # Apply one axis at a time, so NumPy cannot broadcast multiple index arrays.
+ result = data
+ for axis in range(len(key) - 1, -1, -1):
+ index = key[axis]
+ if isinstance(index, slice):
+ single = [slice(None)] * result.ndim
+ single[axis] = index
+ result = result[tuple(single)]
+ else:
+ index = np.asarray(index)
+ if index.dtype.kind == 'b':
+ index = np.flatnonzero(index)
+ result = np.take(result, index, axis=axis)
+ return result
+
+
+class EmptyIndexingTestCase(unittest.TestCase):
+ def setUp(self):
+ self.directory = tempfile.TemporaryDirectory()
+ self.path = os.path.join(self.directory.name, 'empty.nc')
+ self.sources: dict[str, np.ndarray] = {}
+ with Dataset(self.path, 'w') as ds:
+ for name, length in zip(('x', 'y', 'z'), SHAPE):
+ ds.createDimension(name, length)
+ for dtype in ('i4', 'f8'):
+ source = np.arange(np.prod(SHAPE), dtype=dtype).reshape(SHAPE)
+ source[1, 2, 3] = -9999
+ self.sources[dtype] = np.ma.masked_equal(source, -9999)
+ variable = ds.createVariable(dtype, dtype, ('x', 'y', 'z'), fill_value=-9999)
+ variable[:] = source
+ strings = np.array(['s%d' % n for n in range(np.prod(SHAPE))], dtype=object).reshape(SHAPE)
+ ds.createVariable('strings', str, ('x', 'y', 'z'))[:] = strings
+ self.sources['strings'] = strings
+ vlen_type = ds.createVLType(np.int32, 'integers')
+ vlens = np.empty(SHAPE, dtype=object)
+ for index in np.ndindex(SHAPE):
+ vlens[index] = np.arange(sum(index) + 1, dtype=np.int32)
+ ds.createVariable('vlens', vlen_type, ('x', 'y', 'z'))[:] = vlens
+ self.sources['vlens'] = vlens
+
+ def tearDown(self):
+ self.directory.cleanup()
+
+ def assert_selection(self, variable, source, key):
+ expected = orthogonal_selection(source, key)
+ actual = variable[key]
+ self.assertEqual(actual.shape, expected.shape)
+ self.assertEqual(actual.dtype, expected.dtype)
+ if actual.dtype == object:
+ for index in np.ndindex(expected.shape):
+ assert_array_equal(actual[index], expected[index])
+ else:
+ assert_array_equal(actual, expected)
+ assert_array_equal(np.ma.getmaskarray(actual), np.ma.getmaskarray(expected))
+
+ def test_reads(self):
+ # Reopen to exercise reads from the actual C extension and disk.
+ with Dataset(self.path) as ds:
+ for name, source in self.sources.items():
+ for use_get_vars in (True, False):
+ variable = ds[name]
+ variable.use_nc_get_vars(use_get_vars)
+ for key in CASES:
+ with self.subTest(variable=name, use_get_vars=use_get_vars, key=repr(key)):
+ self.assert_selection(variable, source, key)
+ self.assertEqual(tuple(len(ds.dimensions[n]) for n in ('x', 'y', 'z')), SHAPE)
+
+ def test_empty_writes(self):
+ # Both a correctly shaped empty RHS and a scalar are no-ops.
+ with Dataset(self.path, 'r+') as ds:
+ for name in ('i4', 'f8'):
+ variable = ds[name]
+ for key in CASES:
+ expected = orthogonal_selection(self.sources[name], key)
+ if expected.size:
+ continue
+ with self.subTest(variable=name, key=repr(key)):
+ variable[key] = expected
+ variable[key] = 123
+ assert_array_equal(variable[:], self.sources[name])
+ self.assertEqual(variable.shape, SHAPE)
+ with Dataset(self.path) as ds:
+ for name in ('i4', 'f8'):
+ assert_array_equal(ds[name][:], self.sources[name])
+ self.assertEqual(ds[name].shape, SHAPE)
+
+ def test_empty_unlimited_dimension(self):
+ path = os.path.join(self.directory.name, 'unlimited.nc')
+ with Dataset(path, 'w') as ds:
+ ds.createDimension('time', None)
+ ds.createDimension('range', 7)
+ variable = ds.createVariable('data', 'i4', ('time', 'range'))
+ for key in ((EMPTY, slice(None)), (np.zeros(0, dtype=bool), slice(None))):
+ with self.subTest(key=repr(key)):
+ self.assertEqual(variable[key].shape, (0, 7))
+ variable[key] = np.empty((0, 7), dtype='i4')
+ variable[key] = 123
+ self.assertEqual(variable.shape, (0, 7))
+ variable[:] = np.arange(21, dtype='i4').reshape(3, 7)
+ variable[EMPTY, :] = 123
+ self.assertEqual(variable.shape, (3, 7))
+ assert_array_equal(variable[:], np.arange(21, dtype='i4').reshape(3, 7))
+
+ def test_multifile_reads(self):
+ source = np.arange(6 * 5 * 7, dtype='i4').reshape(6, 5, 7)
+ paths = []
+ for number in range(2):
+ path = os.path.join(self.directory.name, 'part%d.nc' % number)
+ paths.append(path)
+ with Dataset(path, 'w', format='NETCDF4_CLASSIC') as ds:
+ ds.createDimension('time', None)
+ ds.createDimension('y', 5)
+ ds.createDimension('z', 7)
+ variable = ds.createVariable('data', 'i4', ('time', 'y', 'z'))
+ variable[:] = source[number * 3:(number + 1) * 3]
+ with MFDataset(paths) as ds:
+ for key in (
+ (EMPTY, slice(None), slice(None)),
+ (np.zeros(6, dtype=bool), slice(None), slice(None)),
+ (slice(None), EMPTY, slice(None)),
+ (slice(None), slice(None), EMPTY),
+ (EMPTY, EMPTY, slice(None)),
+ (EMPTY, 2, slice(None, None, 2)),
+ (2, EMPTY, slice(None)),
+ (slice(0), slice(None), slice(None)),
+ (slice(None), slice(None), slice(None)),
+ ([5, 1, 5], slice(None), [6, 2]),
+ ):
+ with self.subTest(key=repr(key)):
+ self.assert_selection(ds['data'], source, key)
+
+ def test_empty_writes_read_only(self):
+ # Existing empty-array writes also work on read-only variables.
+ with Dataset(self.path) as ds:
+ for name in ('i4', 'f8'):
+ for key in CASES[:16]:
+ with self.subTest(variable=name, key=repr(key)):
+ ds[name][key] = 123
+ assert_array_equal(ds[name][:], self.sources[name])
+
+ def test_empty_fancy_indices_use_bounded_metadata(self):
+ from netCDF4.utils import _StartCountStride
+
+ with Dataset(self.path) as ds:
+ for length in (10, 20):
+ repeated = np.zeros(length, dtype=np.intp)
+ key = (EMPTY, repeated, repeated)
+ with self.subTest(length=length):
+ metadata = _StartCountStride(key, SHAPE)
+ # Empty output must not allocate the Cartesian product of
+ # repeated indices. This bound permits linear metadata.
+ self.assertLessEqual(sum(a.nbytes for a in metadata), 128 * length)
+ self.assertEqual(ds['i4'][key].shape, (0, length, length))
+ with self.assertRaises(IndexError):
+ ds['i4'][EMPTY, [99], [0]]
+
+ def test_empty_broadcast_writes(self):
+ with Dataset(self.path, 'r+') as ds:
+ for name in ('i4', 'f8'):
+ variable = ds[name]
+ for empty in (EMPTY, slice(0), np.zeros(3, dtype=bool)):
+ for rhs in (np.array([1, 2]), np.ones((3, 2))):
+ with self.subTest(variable=name, empty=repr(empty), shape=rhs.shape):
+ variable[empty, [4, 1, 4], [6, 2]] = rhs
+ assert_array_equal(variable[:], self.sources[name])
+ self.assertEqual(variable.shape, SHAPE)
+
+ def test_multifile_empty_decoding(self):
+ paths = []
+ raw = np.arange(12, dtype='i2').reshape(4, 3)
+ attributes: dict[str, dict[str, Any]] = {
+ 'scaled': {'scale_factor': np.float64(0.5)},
+ 'offset': {'add_offset': np.float64(2.0)},
+ 'identity': {'scale_factor': np.float32(1), 'add_offset': np.float32(0)},
+ 'unsigned': {'_Unsigned': 'true'},
+ }
+ for number in range(2):
+ path = os.path.join(self.directory.name, 'packed%d.nc' % number)
+ paths.append(path)
+ with Dataset(path, 'w', format='NETCDF4_CLASSIC') as ds:
+ ds.createDimension('time', None)
+ ds.createDimension('y', 3)
+ for name, attrs in attributes.items():
+ variable = ds.createVariable(name, 'i2', ('time', 'y'))
+ variable.setncatts(attrs)
+ variable.set_auto_maskandscale(False)
+ variable[:] = raw[number * 2:(number + 1) * 2]
+ with MFDataset(paths) as ds:
+ for name, attrs in attributes.items():
+ for scale in (True, False):
+ for mask in (True, False):
+ variable = ds[name]
+ variable.set_auto_scale(scale)
+ variable.set_auto_mask(mask)
+ expected = raw
+ if scale:
+ if name == 'unsigned':
+ expected = raw.astype('u2')
+ elif name == 'identity':
+ expected = raw.astype('f4')
+ else:
+ expected = raw * attrs.get('scale_factor', 1) + attrs.get('add_offset', 0)
+ for key in ((EMPTY, slice(None)), (slice(0), slice(None)),
+ (slice(None), EMPTY), (slice(None), slice(0)),
+ (1, EMPTY), (slice(None), slice(None))):
+ with self.subTest(variable=name, scale=scale, mask=mask, key=repr(key)):
+ self.assert_selection(variable, expected, key)
+
+ def test_empty_scalar_bounds(self):
+ with Dataset(self.path) as ds:
+ for name in self.sources:
+ for empty in (EMPTY, slice(0), np.zeros(3, dtype=bool)):
+ for index in (99, [99], -99):
+ with self.subTest(variable=name, empty=repr(empty), index=repr(index)):
+ with self.assertRaises(IndexError):
+ ds[name][empty, index, :]
+
+ def test_one_dimensional_empty_broadcast(self):
+ path = os.path.join(self.directory.name, 'vector.nc')
+ with Dataset(path, 'w') as ds:
+ ds.createDimension('x', 3)
+ variable = ds.createVariable('data', 'i4', ('x',))
+ variable[:] = [10, 20, 30]
+ for empty in (EMPTY, slice(0), np.zeros(3, dtype=bool)):
+ with self.subTest(empty=repr(empty)):
+ variable[empty] = np.array([123])
+ assert_array_equal(variable[:], [10, 20, 30])
+ with self.assertRaises((ValueError, IndexError)):
+ variable[empty] = np.array([123, 456])
+
+ def test_empty_encoded_characters(self):
+ raw = np.array(['alpha', 'bravo', 'delta', 'gamma'], dtype='S5').view('S1').reshape(4, 5)
+ paths = []
+ for number, values in enumerate((raw, raw[:2], raw[2:])):
+ path = os.path.join(self.directory.name, 'chars%d.nc' % number)
+ paths.append(path)
+ with Dataset(path, 'w', format='NETCDF4_CLASSIC') as ds:
+ ds.createDimension('time', None)
+ ds.createDimension('width', 5)
+ variable = ds.createVariable('data', 'S1', ('time', 'width'))
+ variable._Encoding = 'utf-8'
+ variable.set_auto_chartostring(False)
+ variable[:] = values
+ keys = [
+ ((EMPTY, slice(None)), True),
+ ((slice(0), slice(None)), True),
+ ((EMPTY, slice(1, 4)), False),
+ ((EMPTY, slice(0)), False),
+ ((slice(None), EMPTY), False),
+ ((slice(None), slice(0)), False),
+ ((1, EMPTY), False),
+ ((EMPTY, 2), False),
+ ((EMPTY, [4, 1, 4, 0, 2]), False),
+ ((slice(0), [0, 2, 1, 4, 3]), False),
+ ((slice(None), slice(1, 4)), False),
+ ((slice(None), [4, 1, 4, 0, 2]), False),
+ ]
+ with Dataset(paths[0]) as regular, MFDataset(paths[1:]) as multi:
+ for route, ds in (('regular', regular), ('multifile', multi)):
+ variable = ds['data']
+ for convert in (True, False):
+ variable.set_auto_chartostring(convert)
+ for key, full_width in keys:
+ with self.subTest(route=route, convert=convert, key=repr(key)):
+ expected = orthogonal_selection(raw, key)
+ if convert and full_width:
+ # All full-width cases here have empty records.
+ expected = np.empty(expected.shape[:-1], dtype='U5')
+ actual = variable[key]
+ self.assertEqual(actual.shape, expected.shape)
+ self.assertEqual(actual.dtype, expected.dtype)
+ assert_array_equal(actual, expected)
+
+ def test_empty_scalar_character_width_one(self):
+ paths = []
+ for number in range(2):
+ path = os.path.join(self.directory.name, 'onechar%d.nc' % number)
+ paths.append(path)
+ with Dataset(path, 'w', format='NETCDF4_CLASSIC') as ds:
+ ds.createDimension('time', None)
+ ds.createDimension('width', 1)
+ variable = ds.createVariable('data', 'S1', ('time', 'width'))
+ variable._Encoding = 'utf-8'
+ variable.set_auto_chartostring(False)
+ variable[:] = np.array([['a'], ['b']], dtype='S1')
+ with Dataset(paths[0]) as regular, MFDataset(paths) as multi:
+ for route, ds in (('regular', regular), ('multifile', multi)):
+ for empty in (EMPTY, slice(0)):
+ with self.subTest(route=route, empty=repr(empty)):
+ actual = ds['data'][empty, 0]
+ self.assertEqual(actual.shape, (0,))
+ self.assertEqual(actual.dtype, np.dtype('S1'))
+
+ def test_multifile_selected_decoding(self):
+ paths = []
+ raw = np.arange(18, dtype='i2').reshape(6, 3)
+ attributes: dict[str, tuple[dict[str, Any], ...]] = {
+ 'first_float': ({'scale_factor': np.float64(0.5)}, {}, {}),
+ 'last_float': ({}, {}, {'scale_factor': np.float32(0.5)}),
+ 'unsigned': ({'_Unsigned': 'true'}, {}, {'_Unsigned': 'true'}),
+ }
+ for number in range(3):
+ path = os.path.join(self.directory.name, 'different%d.nc' % number)
+ paths.append(path)
+ with Dataset(path, 'w', format='NETCDF4_CLASSIC') as ds:
+ ds.createDimension('time', None)
+ ds.createDimension('y', 3)
+ for name, attrs in attributes.items():
+ variable = ds.createVariable(name, 'i2', ('time', 'y'))
+ variable.setncatts(attrs[number])
+ variable.set_auto_maskandscale(False)
+ variable[:] = raw[number * 2:(number + 1) * 2]
+
+ # Slices concatenate contributing files and promote their dtypes.
+ # Existing non-slice fancy reads cast the result for each record chunk,
+ # so their final dtype is the dtype of the last selected record.
+ # With no selected records, retain the first-file decoding convention.
+ selections = (
+ (2, (1,)), (4, (2,)), (-1, (2,)),
+ (slice(None), (0, 1, 2)),
+ (slice(2, 6), (1, 2)), (slice(1, 5), (0, 1, 2)),
+ (slice(None, None, 3), (0, 1)),
+ ([0, 4, 2], (1,)), ([4, 0, 4], (2,)),
+ ([0, 0, 0], (0,)), ([4, 4, 4], (2,)),
+ (np.array([True, False, True, False, True, False]), (0, 1, 2)),
+ (EMPTY, (0,)), (slice(0), (0,)),
+ )
+ # check=True validates structure but permits per-file attributes.
+ with MFDataset(paths, check=True) as ds:
+ for name, attrs in attributes.items():
+ variable = ds[name]
+ for scale in (True, False):
+ variable.set_auto_scale(scale)
+ parts = []
+ for number in range(3):
+ part = raw[number * 2:(number + 1) * 2]
+ if scale:
+ if '_Unsigned' in attrs[number]:
+ part = part.astype('u2')
+ elif 'scale_factor' in attrs[number]:
+ part = part * attrs[number]['scale_factor']
+ parts.append(part)
+ for mask in (True, False):
+ variable.set_auto_mask(mask)
+ for records, contributors in selections:
+ expected_dtype = np.result_type(*(parts[n].dtype for n in contributors))
+ for empty in (EMPTY, slice(0)):
+ with self.subTest(variable=name, scale=scale, mask=mask,
+ records=repr(records), empty=repr(empty)):
+ expected_shape = orthogonal_selection(raw, (records, empty)).shape
+ for key in ((records, empty), (Ellipsis, records, empty)):
+ actual = variable[key]
+ self.assertEqual(actual.shape, expected_shape)
+ self.assertEqual(actual.dtype, expected_dtype)
+ self.assertEqual(actual.size, 0)
+ if isinstance(records, slice) and records == slice(None):
+ actual = variable[..., empty]
+ self.assertEqual(actual.shape, expected_shape)
+ self.assertEqual(actual.dtype, expected_dtype)
+ if isinstance(records, (int, np.integer)):
+ with self.subTest(control=name, scale=scale, mask=mask, records=records):
+ record = int(records) % 6
+ actual = variable[records, :]
+ expected = parts[record // 2][record % 2]
+ self.assertEqual(actual.dtype, expected.dtype)
+ assert_array_equal(actual, expected)
+
+ def test_multifile_selected_character_decoding(self):
+ paths = []
+ raw = np.full((6, 2, 3), b'a', dtype='S1')
+ for number, encoding in enumerate(('utf-8', None, 'bytes')):
+ path = os.path.join(self.directory.name, 'encodings%d.nc' % number)
+ paths.append(path)
+ with Dataset(path, 'w', format='NETCDF4_CLASSIC') as ds:
+ ds.createDimension('time', None)
+ ds.createDimension('y', 2)
+ ds.createDimension('width', 3)
+ variable = ds.createVariable('data', 'S1', ('time', 'y', 'width'))
+ if encoding is not None:
+ variable._Encoding = encoding
+ variable.set_auto_chartostring(False)
+ variable[:] = raw[number * 2:(number + 1) * 2]
+ cases = (
+ (0, 'U3'), (2, None), (4, 'S3'),
+ (slice(2, 4), None), (slice(None, None, 4), 'U3'),
+ ([2, 0, 2], None), ([4, 0, 4], 'S3'), (EMPTY, 'U3'),
+ )
+ with MFDataset(paths, check=True) as ds:
+ variable = ds['data']
+ for convert in (True, False):
+ variable.set_auto_chartostring(convert)
+ for records, decoded_dtype in cases:
+ for empty in (EMPTY, slice(0)):
+ with self.subTest(records=repr(records), convert=convert, empty=repr(empty)):
+ expected = orthogonal_selection(raw, (records, empty, slice(None)))
+ if convert and decoded_dtype is not None:
+ expected = np.empty(expected.shape[:-1], dtype=decoded_dtype)
+ actual = variable[records, empty, :]
+ self.assertEqual(actual.shape, expected.shape)
+ self.assertEqual(actual.dtype, expected.dtype)
+ self.assertEqual(actual.size, 0)
+ if convert:
+ # Some contributors decode the character axis and others
+ # do not. Existing concatenation rejects mixed ranks.
+ with self.assertRaises(ValueError):
+ variable[:, :0, :]
+ for record in (0, 2, 4):
+ assert_array_equal(variable[record, :, 0], raw[record, :, 0])
+
+
+if __name__ == '__main__':
+ unittest.main()
=====================================
test/test_fancyslicing.py
=====================================
@@ -76,6 +76,26 @@ class VariablesTestCase(unittest.TestCase):
# Remove the temporary files
os.remove(self.file)
+ def test_float_scalar_get(self):
+ with Dataset(self.file) as f:
+ variable = f.variables['data']
+ for index in (0.8, -0.8, np.float32(0.8), np.float64(-0.8), np.array(0.8)):
+ with self.subTest(index=repr(index)):
+ with self.assertRaises(IndexError):
+ variable[index]
+ for index in (1.0, np.float32(1.0), np.float64(1.0), np.array(1.0)):
+ assert_array_equal(variable[index], variable[1])
+
+ def test_float_scalar_set(self):
+ with Dataset(self.file, 'a') as f:
+ variable = f.variables['data']
+ original = variable[:].copy()
+ for index in (0.8, -0.8, np.float32(0.8), np.float64(-0.8), np.array(0.8)):
+ with self.subTest(index=repr(index)):
+ with self.assertRaises(IndexError):
+ variable[index] = -999
+ assert_array_equal(variable[:], original)
+
def test_get(self):
"""testing 'fancy indexing'"""
f = Dataset(self.file, 'r')
=====================================
test/test_utils.py
=====================================
@@ -184,6 +184,21 @@ class TestgetStartCountStride(unittest.TestCase):
assert_equal(count, 0)
assert_equal(_out_array_shape(count), (0,))
+ def test_float_scalar_index(self):
+ # Integral float scalars retain the historical scalar-index behavior.
+ for elem in (1.0, np.float32(1.0), np.float64(1.0)):
+ start, count, stride, put_ind = _StartCountStride(elem, (10,))
+ assert_equal(start, 1)
+ assert_equal(count, 1)
+ assert_equal(stride, 1)
+ assert_equal(put_ind, -1)
+
+ # Fractional float scalars must not be silently truncated to an
+ # integer index.
+ for elem in (0.8, -0.8, np.float32(0.8), np.float64(-0.8)):
+ with self.assertRaises(IndexError):
+ _StartCountStride(elem, (10,))
+
def test_ellipsis(self):
elem=(Ellipsis, slice(1, 4))
start, count, stride, put_ind = _StartCountStride(elem, (22,25,4))
View it on GitLab: https://salsa.debian.org/debian-gis-team/netcdf4-python/-/compare/5937099b1d1003419daa6fed67b13b625efb2510...486ba7f4352421a7ff06b955594693aadb8ede36
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/netcdf4-python/-/compare/5937099b1d1003419daa6fed67b13b625efb2510...486ba7f4352421a7ff06b955594693aadb8ede36
You're receiving this email because of your account on salsa.debian.org. Manage all notifications: https://salsa.debian.org/-/profile/notifications | Help: https://salsa.debian.org/help
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20261016/11c8a61c/attachment-0001.htm>
More information about the Pkg-grass-devel
mailing list