[Git][debian-gis-team/pyninjotiff][master] 9 commits: New upstream version 0.2.0
Antonio Valentino
gitlab at salsa.debian.org
Sat Sep 21 10:42:50 BST 2019
Antonio Valentino pushed to branch master at Debian GIS Project / pyninjotiff
Commits:
27575121 by Antonio Valentino at 2019-09-21T07:52:09Z
New upstream version 0.2.0
- - - - -
70d5b60a by Antonio Valentino at 2019-09-21T07:52:09Z
Update upstream source from tag 'upstream/0.2.0'
Update to upstream version '0.2.0'
with Debian dir bb04c6a19d6537b0fa1300b1f5a09ff1078fdcac
- - - - -
c153f87c by Antonio Valentino at 2019-09-21T07:57:34Z
New upstream release
- - - - -
08340838 by Antonio Valentino at 2019-09-21T08:00:08Z
Bump debhelper from old 11 to 12.
Fixes lintian: package-uses-old-debhelper-compat-version
See https://lintian.debian.org/tags/package-uses-old-debhelper-compat-version.html for more details.
- - - - -
3ee2a063 by Antonio Valentino at 2019-09-21T08:00:18Z
Remove obsolete fields Name from debian/upstream/metadata.
- - - - -
20e9d94f by Antonio Valentino at 2019-09-21T08:11:29Z
Update copyright file
- - - - -
14a90248 by Antonio Valentino at 2019-09-21T08:22:20Z
Drop all patches
- - - - -
ebb4c7e1 by Antonio Valentino at 2019-09-21T09:30:03Z
Enable testing
- - - - -
64a19fe9 by Antonio Valentino at 2019-09-21T09:38:42Z
Set distribution to unstable
- - - - -
22 changed files:
- .bumpversion.cfg
- + .stickler.yml
- + .travis.yml
- changelog.rst
- debian/changelog
- − debian/compat
- debian/control
- debian/copyright
- − debian/patches/0001-Python-3-compatibility.patch
- − debian/patches/0002-Disable-pointless-warning.patch
- − debian/patches/series
- debian/rules
- debian/upstream/metadata
- pyninjotiff/ninjotiff.py
- + pyninjotiff/ninjotiff_config-file_satpy_example.py
- pyninjotiff/ninjotiff_satpy_example
- + pyninjotiff/rgb_ninjotiff_satpy_example
- + pyninjotiff/tests/test_ninjotiff.py
- pyninjotiff/tifffile.py
- pyninjotiff/version.py
- + setup.cfg
- setup.py
Changes:
=====================================
.bumpversion.cfg
=====================================
@@ -1,5 +1,5 @@
[bumpversion]
-current_version = 0.1.0
+current_version = 0.2.0
commit = True
tag = True
=====================================
.stickler.yml
=====================================
@@ -0,0 +1,7 @@
+linters:
+ flake8:
+ python: 3
+ fixer: true
+ max-line-length: 120
+fixers:
+ enable: true
=====================================
.travis.yml
=====================================
@@ -0,0 +1,15 @@
+language: python
+python:
+ - "2.7"
+ - "3.6"
+ - "3.7"
+
+install:
+ - pip install codecov pytest pytest-cov trollimage xarray dask[array]
+ - pip install -e .
+
+script:
+ - pytest --cov=./
+
+after_success:
+ - codecov
=====================================
changelog.rst
=====================================
@@ -2,6 +2,85 @@ Changelog
=========
+v0.2.0 (2019-09-19)
+-------------------
+- update changelog. [Martin Raspaud]
+- Bump version: 0.1.0 → 0.2.0. [Martin Raspaud]
+- Merge pull request #18 from mraspaud/fix-user-home-path. [Martin
+ Raspaud]
+
+ Fix user home path
+- Fix travis to improve coverage. [Martin Raspaud]
+- Expand the config filename in case ~ is used. [Martin Raspaud]
+- Merge pull request #17 from mraspaud/fix-python3-configparser. [Martin
+ Raspaud]
+
+ Fix python2-only configparser import
+- Fix python2-only configparser import. [Martin Raspaud]
+- Merge pull request #16 from mraspaud/fix-tests. [Martin Raspaud]
+
+ Fix test dependencies
+- Fix area definitions in the tests. [Martin Raspaud]
+- Add pyresample to setup dependencies. [Martin Raspaud]
+- Add pyproj to setup dependencies. [Martin Raspaud]
+- Fix dask array dependencies. [Martin Raspaud]
+- Fix test dependencies. [Martin Raspaud]
+- Fix .travis.yml file. [Martin Raspaud]
+- Merge pull request #14 from pytroll/feature-python3. [Martin Raspaud]
+
+ Support for python3 and unittests
+- Fix P test. [Martin Raspaud]
+- Add draft test for P mode. [Martin Raspaud]
+- Use _FillValue to mask integer arrays. [Martin Raspaud]
+- Add trollimage to test dependencies. [Martin Raspaud]
+- Add codecov to travis. [Martin Raspaud]
+- Fix channel in vis tests. [Martin Raspaud]
+- Fix stickler line length. [Martin Raspaud]
+- Fixing style errors. [stickler-ci]
+- Add tests. [Martin Raspaud]
+- Fix scaling bw images. [Martin Raspaud]
+- Fix style. [Martin Raspaud]
+- Fixing style errors. [stickler-ci]
+- Start supporting python3. [Martin Raspaud]
+- Merge pull request #13 from pytroll/add-stickler-config. [Martin
+ Raspaud]
+
+ Adding .stickler.yml configuration file
+- Adding .stickler.yml. [stickler-ci]
+- Merge pull request #9 from pytroll/develop. [David Hoese]
+
+ Merge the develop branch in to master
+- Merge pull request #3 from goodsonr/compatability-python3. [Martin
+ Raspaud]
+
+ change all occurences of xrange to range for compatability with Python3
+- change all occurences of xrange to range for compatability with
+ Python3. [ron goodson]
+- Add zero seconds option to zero the seconds of the DateID. [Martin
+ Raspaud]
+- Fix package description. [Martin Raspaud]
+- Merge pull request #5 from loreclem/master. [David Hoese]
+
+ WIP|PCW: first attempt to make pyninjotiff xarray compatible.
+- Merge pull request #2 from vgiuffrida/master. [lorenzo clementi]
+
+ fix not assigned fill_value and a config file loader issue
+- fix not assigned fill_value and config file loader. [root]
+- fix fill_value and config file loader. [root]
+- Merge pull request #1 from vgiuffrida/master. [lorenzo clementi]
+
+ Add new parameter to configure the ninjotiff config file to use
+- Add ninjotiff configuration file loading. [root]
+- Typos corrected and removed is_masked. [cll]
+- Bugfix (is_masked computed twice) [cll]
+- WIP Improvements here and there. [cll]
+- Using finalize instead of fill_or_alpha. [cll]
+- It now can handle also RGB images. [cll]
+- WIP: first attempt to make pyninjotiff xarray compatible. For the
+ moment, only the 'L' case (1 band) has been upgraded. Still to be
+ verified. [cll]
+
+
v0.1.0 (2017-10-16)
-------------------
- update changelog. [Martin Raspaud]
=====================================
debian/changelog
=====================================
@@ -1,10 +1,19 @@
-pyninjotiff (0.1.0-2) UNRELEASED; urgency=medium
+pyninjotiff (0.2.0-1) unstable; urgency=medium
- * Team upload.
+ [ Bas Couwenberg ]
* Update gbp.conf to use --source-only-changes by default.
* Bump Standards-Version to 4.4.0, no changes.
- -- Bas Couwenberg <sebastic at debian.org> Sun, 07 Jul 2019 09:40:37 +0200
+ [ Antonio Valentino ]
+ * New upstream release.
+ * Bump debhelper from old 11 to 12.
+ * Remove obsolete fields Name from debian/upstream/metadata.
+ * Update copyright file.
+ * debian/patches:
+ - remove all patches (no longer needed)
+ * Enable testing (add dependencies on pytest, xarray and trollimage).
+
+ -- Antonio Valentino <antonio.valentino at tiscali.it> Sat, 21 Sep 2019 09:38:30 +0000
pyninjotiff (0.1.0-1) unstable; urgency=medium
=====================================
debian/compat deleted
=====================================
@@ -1 +0,0 @@
-11
=====================================
debian/control
=====================================
@@ -4,15 +4,19 @@ Uploaders: Antonio Valentino <antonio.valentino at tiscali.it>
Section: python
Priority: optional
Testsuite: autopkgtest-pkg-python
-Build-Depends: debhelper (>= 11),
+Build-Depends: debhelper-compat (= 12),
dh-python,
python3-all,
+ python3-dask,
python3-matplotlib,
python3-numpy,
python3-pyproj,
python3-pyresample,
+ python3-pytest,
python3-setuptools,
- python3-six
+ python3-six,
+ python3-trollimage,
+ python3-xarray
Standards-Version: 4.4.0
Vcs-Browser: https://salsa.debian.org/debian-gis-team/pyninjotiff
Vcs-Git: https://salsa.debian.org/debian-gis-team/pyninjotiff.git
=====================================
debian/copyright
=====================================
@@ -4,7 +4,7 @@ Upstream-Contact: Martin Raspaud <martin.raspaud at smhi.se>
Source: https://github.com/pytroll/pyninjotiff
Files: *
-Copyright: 2017 Martin Raspaud <martin.raspaud at smhi.se>
+Copyright: 2017-2019 Martin Raspaud <martin.raspaud at smhi.se>
2013 Space Science and Engineering Center (SSEC),
University of Wisconsin-Madison. Lars Ørum Rasmussen, DMI.
License: GPL-3+
@@ -14,7 +14,7 @@ Copyright: 2008-2014 Christoph Gohlke
License: BSD-3-clause
Files: debian/*
-Copyright: 2018 Antonio Valentino <antonio.valentino at tiscali.it>
+Copyright: 2018-2019 Antonio Valentino <antonio.valentino at tiscali.it>
License: GPL-3+
License: GPL-3+
=====================================
debian/patches/0001-Python-3-compatibility.patch deleted
=====================================
@@ -1,61 +0,0 @@
-From: Antonio Valentino <antonio.valentino at tiscali.it>
-Date: Mon, 31 Dec 2018 17:15:28 +0000
-Subject: Python 3 compatibility
-
----
- pyninjotiff/ninjotiff.py | 17 +++++++++++------
- 1 file changed, 11 insertions(+), 6 deletions(-)
-
-diff --git a/pyninjotiff/ninjotiff.py b/pyninjotiff/ninjotiff.py
-index 16b8374..813f672 100644
---- a/pyninjotiff/ninjotiff.py
-+++ b/pyninjotiff/ninjotiff.py
-@@ -35,6 +35,8 @@ Edited by Christian Kliche (Ernst Basler + Partner) to replace pylibtiff with
- a modified version of tifffile.py (created by Christoph Gohlke)
- """
-
-+from __future__ import print_function
-+
- import calendar
- import logging
- import os
-@@ -176,7 +178,10 @@ class ProductConfigs(object):
- return sorted(self._products.keys())
-
- def read_config(self):
-- from ConfigParser import ConfigParser
-+ try:
-+ from ConfigParser import ConfigParser
-+ except ImportError:
-+ from configparser import ConfigParser
-
- def _eval(val):
- try:
-@@ -1060,9 +1065,9 @@ if __name__ == '__main__':
- try:
- filename = args[0]
- except IndexError:
-- print >> sys.stderr, """usage: python ninjotiff.py [<-p page-number>] [-c] <ninjotiff-filename>
-+ print("""usage: python ninjotiff.py [<-p page-number>] [-c] <ninjotiff-filename>
- -p <page-number>: print page number (default are all pages).
-- -c: print color maps (default is not to print color maps)."""
-+ -c: print color maps (default is not to print color maps).""", file=sys.stderr)
- sys.exit(2)
-
- pages = read_tags(filename)
-@@ -1070,12 +1075,12 @@ if __name__ == '__main__':
- try:
- pages = [pages[page_no]]
- except IndexError:
-- print >>sys.stderr, "Invalid page number '%d'" % page_no
-+ print("Invalid page number '%d'" % page_no, file=sys.stderr)
- sys.exit(2)
- for page in pages:
- names = sorted(page.keys())
-- print ""
-+ print("")
- for name in names:
- if not print_color_maps and name == "color_map":
- continue
-- print name, page[name]
-+ print(name, page[name])
=====================================
debian/patches/0002-Disable-pointless-warning.patch deleted
=====================================
@@ -1,33 +0,0 @@
-From: Antonio Valentino <antonio.valentino at tiscali.it>
-Date: Mon, 31 Dec 2018 17:33:47 +0000
-Subject: Disable pointless warning
-
----
- pyninjotiff/tifffile.py | 14 +++++++-------
- 1 file changed, 7 insertions(+), 7 deletions(-)
-
-diff --git a/pyninjotiff/tifffile.py b/pyninjotiff/tifffile.py
-index 3e0cf23..29ebadb 100644
---- a/pyninjotiff/tifffile.py
-+++ b/pyninjotiff/tifffile.py
-@@ -148,13 +148,13 @@ from xml.etree import cElementTree as etree
-
- import numpy
-
--try:
-- import _tifffile
--except ImportError:
-- warnings.warn(
-- "failed to import the optional _tifffile C extension module.\n"
-- "Loading of some compressed images will be slow.\n"
-- "Tifffile.c can be obtained at http://www.lfd.uci.edu/~gohlke/")
-+# try:
-+# import _tifffile
-+# except ImportError:
-+# warnings.warn(
-+# "failed to import the optional _tifffile C extension module.\n"
-+# "Loading of some compressed images will be slow.\n"
-+# "Tifffile.c can be obtained at http://www.lfd.uci.edu/~gohlke/")
-
- __version__ = '2014.08.24'
- __docformat__ = 'restructuredtext en'
=====================================
debian/patches/series deleted
=====================================
@@ -1,2 +0,0 @@
-0001-Python-3-compatibility.patch
-0002-Disable-pointless-warning.patch
=====================================
debian/rules
=====================================
@@ -5,7 +5,7 @@
#export DH_VERBOSE=1
export PYBUILD_NAME=pyninjotiff
-export PYBUILD_DISABLE=test
+export PYBUILD_BEFORE_TEST=cp -r {dir}/pyninjotiff/tests {build_dir}
%:
dh $@ --with python3 --buildsystem=pybuild
=====================================
debian/upstream/metadata
=====================================
@@ -1,6 +1,4 @@
----
Bug-Database: https://github.com/pytroll/pyninjotiff/issues
Bug-Submit: https://github.com/pytroll/pyninjotiff/issues/new
-Name: PyNinjoTiff
Repository: https://github.com/pytroll/pyninjotiff.git
Repository-Browse: https://github.com/pytroll/pyninjotiff
=====================================
pyninjotiff/ninjotiff.py
=====================================
@@ -1,39 +1,38 @@
# -*- coding: utf-8 -*-
-"""
-ninjotiff.py
-
-Created on Mon Apr 15 13:41:55 2013
-
-A big amount of the tiff writer are (PFE) from
-https://github.com/davidh-ssec/polar2grid by David Hoese
-
-License:
-Copyright (C) 2013 Space Science and Engineering Center (SSEC),
- University of Wisconsin-Madison.
- Lars Ørum Rasmussen, DMI.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-Original scripts and automation included as part of this package are
-distributed under the GNU GENERAL PUBLIC LICENSE agreement version 3.
-Binary executable files included as part of this software package are
-copyrighted and licensed by their respective organizations, and
-distributed consistent with their licensing terms.
-
-Edited by Christian Kliche (Ernst Basler + Partner) to replace pylibtiff with
-a modified version of tifffile.py (created by Christoph Gohlke)
-"""
+# ninjotiff.py
+#
+# Created on Mon Apr 15 13:41:55 2013
+#
+# A big amount of the tiff writer are (PFE) from
+# https://github.com/davidh-ssec/polar2grid by David Hoese
+#
+# License:
+# Copyright (C) 2013 Space Science and Engineering Center (SSEC),
+# University of Wisconsin-Madison.
+# Lars Ørum Rasmussen, DMI.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+# Original scripts and automation included as part of this package are
+# distributed under the GNU GENERAL PUBLIC LICENSE agreement version 3.
+# Binary executable files included as part of this software package are
+# copyrighted and licensed by their respective organizations, and
+# distributed consistent with their licensing terms.
+#
+# Edited by Christian Kliche (Ernst Basler + Partner) to replace pylibtiff with
+# a modified version of tifffile.py (created by Christoph Gohlke)
+"""Ninjotiff writing utility."""
import calendar
import logging
@@ -46,16 +45,16 @@ import numpy as np
from pyproj import Proj
from pyresample.utils import proj4_radius_parameters
-#import mpop.imageo.formats.writer_options as write_opts
from pyninjotiff import tifffile
log = logging.getLogger(__name__)
-#-------------------------------------------------------------------------
+
+# -------------------------------------------------------------------------
#
# Ninjo tiff tags from DWD
#
-#-------------------------------------------------------------------------
+# -------------------------------------------------------------------------
# Geotiff tags.
GTF_ModelPixelScale = 33550
GTF_ModelTiepoint = 33922
@@ -123,12 +122,34 @@ MODEL_PIXEL_SCALE_COUNT = int(os.environ.get(
"GEOTIFF_MODEL_PIXEL_SCALE_COUNT", 3))
-#-------------------------------------------------------------------------
+# -------------------------------------------------------------------------
#
# Read Ninjo products config file.
#
-#-------------------------------------------------------------------------
-def get_product_config(product_name, force_read=False):
+# -------------------------------------------------------------------------
+def get_writer_config(config_fname, prod, single_product_config, scn_metadata):
+ """Writer_config function for Trollflow_sat: calls the get_product_config function.
+
+ :Parameters:
+ config_fname: str
+ Name of the Ninjo product configuration file
+
+ prod: str
+ Name of Ninjo product.
+
+ single_product_config: dict
+ config params for the current product
+
+ scn_metadata: dict
+ Satpy satellite data
+ """
+ ninjo_product = prod
+ if 'ninjo_product_name' in single_product_config:
+ ninjo_product = single_product_config['ninjo_product_name']
+ return get_product_config(ninjo_product, True, config_fname)
+
+
+def get_product_config(product_name, force_read=False, config_filename=None):
"""Read Ninjo configuration entry for a given product name.
:Parameters:
@@ -145,74 +166,95 @@ def get_product_config(product_name, force_read=False):
* As an example, see *ninjotiff_products.cfg.template* in
MPOP's *etc* directory.
"""
- return ProductConfigs()(product_name, force_read)
+ return ProductConfigs()(product_name, force_read, config_filename)
class _Singleton(type):
- def __init__(cls, name_, bases_, dict_):
- super(_Singleton, cls).__init__(name_, bases_, dict_)
- cls.instance = None
+ def __init__(self, name_, bases_, dict_):
+ """Init the singleton."""
+ super(_Singleton, self).__init__(name_, bases_, dict_)
+ self.instance = None
- def __call__(cls, *args, **kwargs):
- if cls.instance is None:
- cls.instance = super(_Singleton, cls).__call__(*args, **kwargs)
- return cls.instance
+ def __call__(self, *args, **kwargs):
+ """Call the singleton."""
+ if self.instance is None:
+ self.instance = super(_Singleton, self).__call__(*args, **kwargs)
+ return self.instance
class ProductConfigs(object):
- __metaclass__ = _Singleton
+ """Product config."""
+
+ __metaclass__ = _Singleton # noqa
def __init__(self):
+ """Init the product config."""
self.read_config()
- def __call__(self, product_name, force_read=False):
+ def __call__(self, product_name, force_read=False, config_filename=None):
+ """Call the product config."""
if force_read:
- self.read_config()
- return self._products[product_name]
+ self.read_config(config_filename)
+ if product_name in self._products:
+ return self._products[product_name]
+ else:
+ return {}
@property
def product_names(self):
+ """Get the product names."""
return sorted(self._products.keys())
- def read_config(self):
- from ConfigParser import ConfigParser
+ def read_config(self, config_filename=None):
+ """Read the ninjo products config file."""
+ from six.moves.configparser import RawConfigParser
+ import ast
def _eval(val):
try:
- return eval(val)
- except:
+ return ast.literal_eval(val)
+ except (ValueError, SyntaxError):
return str(val)
- filename = self._find_a_config_file()
+ if config_filename is not None:
+ filename = self._find_a_config_file(config_filename)
+ else:
+ filename = self._find_a_config_file('ninjotiff_products.cfg')
log.info("Reading Ninjo config file: '%s'" % filename)
- cfg = ConfigParser()
- cfg.read(filename)
+ cfg = RawConfigParser()
products = {}
- for sec in cfg.sections():
- prd = {}
- for key, val in cfg.items(sec):
- prd[key] = _eval(val)
- products[sec] = prd
+ if filename is not None:
+ cfg.read(filename)
+ for sec in cfg.sections():
+ prd = {}
+ for key, val in cfg.items(sec):
+ prd[key] = _eval(val)
+ products[sec] = prd
self._products = products
@staticmethod
- def _find_a_config_file():
- name_ = 'ninjotiff_products.cfg'
- home_ = os.path.dirname(os.path.abspath(__file__))
- penv_ = os.environ.get('PPP_CONFIG_DIR', '')
- for fname_ in [os.path.join(x, name_) for x in (home_, penv_)]:
- if os.path.isfile(fname_):
- return fname_
- raise ValueError("Could not find a Ninjo tiff config file")
+ def _find_a_config_file(fname):
+ # if config file (fname) is not found as absolute path: look for the
+ # config file in the PPP_CONFIG_DIR or current dir
+ name_ = os.path.abspath(os.path.expanduser(fname))
+ if os.path.isfile(name_):
+ return name_
+ else:
+ home_ = os.path.dirname(os.path.abspath(__file__))
+ penv_ = os.environ.get('PPP_CONFIG_DIR', '')
+ for fname_ in [os.path.join(x, name_) for x in (home_, penv_)]:
+ if os.path.isfile(fname_):
+ return fname_
+ # raise ValueError("Could not find a Ninjo tiff config file")
-#-------------------------------------------------------------------------
+# -------------------------------------------------------------------------
#
# Write Ninjo Products
#
-#-------------------------------------------------------------------------
+# -------------------------------------------------------------------------
def _get_physic_value(physic_unit):
# return Ninjo's physics unit and value.
if physic_unit.upper() in ('K', 'KELVIN'):
@@ -281,9 +323,11 @@ def _get_satellite_altitude(filename):
return None
-def _finalize(img, dtype=np.uint8, value_range_measurement_unit=None, data_is_scaled_01=True):
- """Finalize a mpop GeoImage for Ninjo. Specialy take care of phycical scale
- and offset.
+def _finalize(img, dtype=np.uint8, value_range_measurement_unit=None,
+ data_is_scaled_01=True, fill_value=None):
+ """Finalize a mpop GeoImage for Ninjo.
+
+ Specialy take care of phycical scale and offset.
:Parameters:
img : mpop.imageo.img.GeoImage
@@ -308,14 +352,34 @@ def _finalize(img, dtype=np.uint8, value_range_measurement_unit=None, data_is_sc
**Notes**:
physic_val = image*scale + offset
Example values for value_range_measurement_unit are (0, 125) or (40.0, -87.5)
+
+ ***Warning***
+ Only the 'L' and 'RGB' cases are compatible with xarray.XRImage.
+ They still have to be tested thoroughly.
"""
if img.mode == 'L':
# PFE: mpop.satout.cfscene
- data = img.channels[0]
- fill_value = np.iinfo(dtype).min
- log.debug("Transparent pixel are forced to be %d" % fill_value)
+ if isinstance(img, np.ma.MaskedArray):
+ data = img.channels[0]
+ else:
+ # TODO: check what is the correct fill value for NinJo!
+ if fill_value is not None:
+ log.debug("Forcing fill value to %s", fill_value)
+ # Go back to the masked_array for compatibility
+ # with the following part of the code.
+ if (np.issubdtype(img.data[0].dtype, np.integer)
+ and '_FillValue' in img.data[0].attrs):
+ nodata_value = img.data[0].attrs['_FillValue']
+ data = img.data[0].values
+ data = np.ma.array(data, mask=(data == nodata_value))
+ else:
+ data = img.data[0].to_masked_array()
+
+ fill_value = fill_value if fill_value is not None else np.iinfo(dtype).min
+
log.debug("Before scaling: %.2f, %.2f, %.2f" %
(data.min(), data.mean(), data.max()))
+
if np.ma.count_masked(data) == data.size:
# All data is masked
data = np.ones(data.shape, dtype=dtype) * fill_value
@@ -328,26 +392,20 @@ def _finalize(img, dtype=np.uint8, value_range_measurement_unit=None, data_is_sc
# value_range_measurement_unit[0] and 1.0 as
# value_range_measurement_unit[1]
- # Make room for transparent pixel.
- scale_fill_value = (
- (np.iinfo(dtype).max) / (np.iinfo(dtype).max + 1.0))
- img = deepcopy(img)
- img.channels[0] *= scale_fill_value
-
- img.channels[0] += 1 / (np.iinfo(dtype).max + 1.0)
+ # Make room for the transparent pixel value.
+ data = data.clip(0, 1)
+ data *= (np.iinfo(dtype).max - 1)
+ data += 1
- channels, fill_value = img._finalize(dtype)
- data = channels[0]
-
- scale = ((value_range_measurement_unit[1] -
- value_range_measurement_unit[0]) /
- (np.iinfo(dtype).max))
+ scale = ((value_range_measurement_unit[1]
+ - value_range_measurement_unit[0])
+ / (np.iinfo(dtype).max - 1))
# Handle the case where all data has the same value.
scale = scale or 1
offset = value_range_measurement_unit[0]
mask = data.mask
-
+ data = np.round(data.data).astype(dtype)
offset -= scale
if fill_value is None:
@@ -397,8 +455,17 @@ def _finalize(img, dtype=np.uint8, value_range_measurement_unit=None, data_is_sc
return data, scale, offset, fill_value
elif img.mode == 'RGB':
- channels, fill_value = img._finalize(dtype)
- if fill_value is None:
+ if isinstance(img, np.ma.MaskedArray):
+ channels, fill_value = img._finalize(dtype)
+ else:
+ data, mode = img.finalize(fill_value=fill_value, dtype=dtype)
+ # Go back to the masked_array for compatibility with
+ # the rest of the code.
+ channels = data.to_masked_array()
+ # Is this fill_value ok or what should it be?
+ fill_value = (0, 0, 0, 0)
+
+ if isinstance(img, np.ma.MaskedArray) and fill_value is None:
mask = (np.ma.getmaskarray(channels[0]) &
np.ma.getmaskarray(channels[1]) &
np.ma.getmaskarray(channels[2]))
@@ -411,6 +478,8 @@ def _finalize(img, dtype=np.uint8, value_range_measurement_unit=None, data_is_sc
return data, 1.0, 0.0, fill_value[0]
elif img.mode == 'RGBA':
+ if not isinstance(img, np.ma.MaskedArray):
+ raise NotImplementedError("The 'RGBA' case has not been updated to xarray")
channels, fill_value = img._finalize(dtype)
fill_value = fill_value or (0, 0, 0, 0)
data = np.dstack((channels[0].filled(fill_value[0]),
@@ -420,6 +489,8 @@ def _finalize(img, dtype=np.uint8, value_range_measurement_unit=None, data_is_sc
return data, 1.0, 0.0, fill_value[0]
elif img.mode == 'P':
+ if not isinstance(img, np.ma.MaskedArray):
+ raise NotImplementedError("The 'P' case has not been updated to xarray")
fill_value = 0
data = img.channels[0]
if isinstance(data, np.ma.core.MaskedArray):
@@ -430,11 +501,11 @@ def _finalize(img, dtype=np.uint8, value_range_measurement_unit=None, data_is_sc
return data, 1.0, 0.0, fill_value
else:
- raise ValueError("Don't known how til handle image mode '%s'" %
+ raise ValueError("Don't know how to handle image mode '%s'" %
str(img.mode))
-def save(img, filename, ninjo_product_name=None, writer_options=None,
+def save(img, filename, ninjo_product_name=None, writer_options=None, data_is_scaled_01=True,
**kwargs):
"""Ninjo TIFF writer.
@@ -460,7 +531,6 @@ def save(img, filename, ninjo_product_name=None, writer_options=None,
* min value will be reserved for transparent color.
* If possible mpop.imageo.image's standard finalize will be used.
"""
-
if writer_options:
# add writer_options
kwargs.update(writer_options)
@@ -473,21 +543,31 @@ def save(img, filename, ninjo_product_name=None, writer_options=None,
if nbits == 16:
dtype = np.uint16
+ fill_value = None
+ if 'fill_value' in kwargs and kwargs['fill_value'] is not None:
+ fill_value = int(kwargs['fill_value'])
+
try:
value_range_measurement_unit = (float(kwargs["ch_min_measurement_unit"]),
float(kwargs["ch_max_measurement_unit"]))
except KeyError:
value_range_measurement_unit = None
- data_is_scaled_01 = bool(kwargs.get("data_is_scaled_01", True))
+ # In case we are working on a trollimage.xrimage.XRImage,
+ # a conversion to the previously used masked_array is needed
data, scale, offset, fill_value = _finalize(img,
dtype=dtype,
data_is_scaled_01=data_is_scaled_01,
- value_range_measurement_unit=value_range_measurement_unit,)
+ value_range_measurement_unit=value_range_measurement_unit,
+ fill_value=fill_value,)
- area_def = img.info['area']
- time_slot = img.info['start_time']
+ if isinstance(img, np.ma.MaskedArray):
+ area_def = img.info['area']
+ time_slot = img.info['start_time']
+ else:
+ area_def = img.data.area
+ time_slot = img.data.start_time
# Some Ninjo tiff names
kwargs['gradient'] = scale
@@ -505,14 +585,13 @@ def save(img, filename, ninjo_product_name=None, writer_options=None,
g += [0] * (256 - len(g))
b += [0] * (256 - len(b))
kwargs['cmap'] = r, g, b
-
write(data, filename, area_def, ninjo_product_name, **kwargs)
def write(image_data, output_fn, area_def, product_name=None, **kwargs):
- """Generic Ninjo TIFF writer.
+ """Write a Generic Ninjo TIFF.
- If 'prodcut_name' is given, it will load corresponding Ninjo tiff metadata
+ If 'product_name' is given, it will load corresponding Ninjo tiff metadata
from '${PPP_CONFIG_DIR}/ninjotiff.cfg'. Else, all Ninjo tiff metadata should
be passed by '**kwargs'. A mixture is allowed, where passed arguments
overwrite config file.
@@ -531,9 +610,7 @@ def write(image_data, output_fn, area_def, product_name=None, **kwargs):
kwargs : dict
See _write
"""
-
-
- proj = Proj(area_def.proj_dict)
+ proj = Proj(area_def.proj_dict)
upper_left = proj(
area_def.area_extent[0],
area_def.area_extent[3],
@@ -541,8 +618,8 @@ def write(image_data, output_fn, area_def, product_name=None, **kwargs):
lower_right = proj(
area_def.area_extent[2],
area_def.area_extent[1],
- inverse=True)
-
+ inverse=True)
+
if len(image_data.shape) == 3:
if image_data.shape[2] == 4:
shape = (area_def.y_size, area_def.x_size, 4)
@@ -585,7 +662,11 @@ def write(image_data, output_fn, area_def, product_name=None, **kwargs):
kwargs['altitude'] = altitude
if product_name:
- options = deepcopy(get_product_config(product_name))
+ # If ninjo_product_file in kwargs, load ninjo_product_file as config file
+ if 'ninjo_product_file' in kwargs:
+ options = deepcopy(get_product_config(product_name, True, kwargs['ninjo_product_file']))
+ else:
+ options = deepcopy(get_product_config(product_name))
else:
options = {}
@@ -601,9 +682,8 @@ def write(image_data, output_fn, area_def, product_name=None, **kwargs):
options['ref_lat2'] = 0
if 'lon_0' in area_def.proj_dict:
options['central_meridian'] = area_def.proj_dict['lon_0']
-
- a,b = proj4_radius_parameters(area_def.proj_dict)
+ a, b = proj4_radius_parameters(area_def.proj_dict)
options['radius_a'] = a
options['radius_b'] = b
options['origin_lon'] = upper_left[0]
@@ -621,11 +701,12 @@ def write(image_data, output_fn, area_def, product_name=None, **kwargs):
#
# -----------------------------------------------------------------------------
def _write(image_data, output_fn, write_rgb=False, **kwargs):
- """Proudly Found Elsewhere (PFE) https://github.com/davidh-ssec/polar2grid
+ """Create a NinJo compatible TIFF file.
+
+ Proudly Found Elsewhere (PFE) https://github.com/davidh-ssec/polar2grid
by David Hoese.
- Create a NinJo compatible TIFF file with the tags used
- by the DWD's version of NinJo. Also stores the image as tiles on disk
+ Also stores the image as tiles on disk
and creates a multi-resolution/pyramid/overview set of images
(deresolution: 2,4,8,16).
@@ -769,6 +850,7 @@ def _write(image_data, output_fn, write_rgb=False, **kwargs):
origin_lat = float(kwargs.pop("origin_lat"))
origin_lon = float(kwargs.pop("origin_lon"))
image_dt = kwargs.pop("image_dt")
+ zero_seconds = kwargs.pop("zero_seconds", False)
projection = str(kwargs.pop("projection"))
meridian_west = float(kwargs.pop("meridian_west", 0.0))
meridian_east = float(kwargs.pop("meridian_east", 0.0))
@@ -842,7 +924,13 @@ def _write(image_data, output_fn, write_rgb=False, **kwargs):
file_dt = datetime.utcnow()
file_epoch = calendar.timegm(file_dt.timetuple())
- image_epoch = calendar.timegm(image_dt.timetuple())
+ if zero_seconds:
+ log.debug("Applying zero seconds correction")
+ image_dt_corr = datetime(image_dt.year, image_dt.month, image_dt.day,
+ image_dt.hour, image_dt.minute)
+ else:
+ image_dt_corr = image_dt
+ image_epoch = calendar.timegm(image_dt_corr.timetuple())
compression = _eval_or_default("compression", int, 6)
@@ -1045,6 +1133,7 @@ def read_tags(filename):
pages.append(tags)
return pages
+
if __name__ == '__main__':
import sys
import getopt
@@ -1060,9 +1149,9 @@ if __name__ == '__main__':
try:
filename = args[0]
except IndexError:
- print >> sys.stderr, """usage: python ninjotiff.py [<-p page-number>] [-c] <ninjotiff-filename>
+ print("""usage: python ninjotiff.py [<-p page-number>] [-c] <ninjotiff-filename>
-p <page-number>: print page number (default are all pages).
- -c: print color maps (default is not to print color maps)."""
+ -c: print color maps (default is not to print color maps).""", sys.stderr)
sys.exit(2)
pages = read_tags(filename)
@@ -1070,12 +1159,12 @@ if __name__ == '__main__':
try:
pages = [pages[page_no]]
except IndexError:
- print >>sys.stderr, "Invalid page number '%d'" % page_no
+ print("Invalid page number '%d'" % page_no, sys.stderr)
sys.exit(2)
for page in pages:
names = sorted(page.keys())
- print ""
+ print("")
for name in names:
if not print_color_maps and name == "color_map":
continue
- print name, page[name]
+ print(name, page[name])
=====================================
pyninjotiff/ninjotiff_config-file_satpy_example.py
=====================================
@@ -0,0 +1,24 @@
+import os
+from satpy import Scene
+from datetime import datetime
+from satpy.utils import debug_on
+import pyninjotiff
+from glob import glob
+from pyresample.utils import load_area
+import copy
+debug_on()
+
+
+chn = "IR_108"
+ninjoRegion = load_area("areas.def", "nrEURO3km")
+
+filenames = glob("data/*__")
+global_scene = Scene(reader="hrit_msg", filenames=filenames)
+global_scene.load([chn])
+local_scene = global_scene.resample(ninjoRegion)
+local_scene.save_dataset(chn, filename="msg.tif", writer='ninjotiff',
+ # ninjo product name to look for in .cfg file
+ ninjo_product_name="IR_108",
+ # custom configuration file for ninjo tiff products
+ # if not specified PPP_CONFIG_DIR is used as config file directory
+ ninjo_product_file="/config_dir/ninjotiff_products.cfg")
=====================================
pyninjotiff/ninjotiff_satpy_example
=====================================
@@ -3,19 +3,23 @@ from satpy import Scene
from datetime import datetime
from satpy.utils import debug_on
import pyninjotiff
+from glob import glob
+from pyresample.utils import load_area
+import copy
debug_on()
-chn=10.8
-time_slot = datetime(2017, 1, 27, 7, 45)
-global_data = Scene(platform_name="Meteosat-10", sensor="seviri", reader="hrit_msg", start_time=time_slot)
+chn = "IR_108"
+ninjoRegion = load_area("areas.def", "nrEURO3km")
-global_data.load([chn])
-local_scene = global_data.resample("NinJoRegion")
+filenames = glob("data/*__")
+global_scene = Scene(reader="hrit_msg", filenames=filenames)
+global_scene.load([chn])
+local_scene = global_scene.resample(ninjoRegion)
local_scene.save_dataset(chn, filename="msg.tif", writer='ninjotiff',
- sat_id=1234,
- chan_id=5678,
+ sat_id=6300014,
+ chan_id=900015,
data_cat='GORN',
- data_source='EUMETSAT/MeteoSwiss',
+ data_source='EUMCAST',
physic_unit='K',
nbits=8)
=====================================
pyninjotiff/rgb_ninjotiff_satpy_example
=====================================
@@ -0,0 +1,23 @@
+import os
+from satpy import Scene
+from datetime import datetime
+from satpy.utils import debug_on
+import pyninjotiff
+from glob import glob
+from pyresample.utils import load_area
+debug_on()
+
+
+chn = "airmass"
+ninjoRegion = load_area("areas.def", "nrEURO3km")
+
+filenames = glob("data/*__")
+global_scene = Scene(reader="hrit_msg", filenames=filenames)
+global_scene.load([chn])
+local_scene = global_scene.resample(ninjoRegion)
+local_scene.save_dataset(chn, filename="airmass.tif", writer='ninjotiff',
+ sat_id=6300014,
+ chan_id=6500015,
+ data_cat='GPRN',
+ data_source='EUMCAST',
+ nbits=8)
=====================================
pyninjotiff/tests/test_ninjotiff.py
=====================================
@@ -0,0 +1,456 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2019 Martin Raspaud
+
+# Author(s):
+
+# Martin Raspaud <martin.raspaud at smhi.se>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+"""Test the ninjotiff writing."""
+
+import numpy as np
+import datetime
+import tempfile
+import xarray as xr
+import dask.array as da
+import colorsys
+import pytest
+
+TIME = datetime.datetime.utcnow()
+DELETE_FILES = True
+
+
+class FakeImage(object):
+ """Fake Image object for testing purposes."""
+
+ def __init__(self, data):
+ """Initialize the image."""
+ self.mode = ''.join(data.bands.values)
+ self.data = data
+
+ def finalize(self, fill_value=None, dtype=None):
+ if dtype is None:
+ dtype = np.uint8
+ if np.issubdtype(self.data.dtype, np.floating) and np.issubdtype(dtype, np.integer):
+ res = self.data.clip(0, 1) * np.iinfo(dtype).max
+ res = res.astype(dtype)
+ else:
+ res = self.data
+ return [res.astype(dtype)]
+
+
+class FakeArea(object):
+ def __init__(self, proj_dict, extent, y_size, x_size):
+ self.proj_dict = proj_dict
+ self.area_extent = extent
+ self.x_size, self.y_size = x_size, y_size
+ self.pixel_size_x = (extent[2] - extent[0]) / x_size
+ self.pixel_size_y = (extent[3] - extent[1]) / y_size
+
+
+def test_write_bw():
+ """Test saving a BW image."""
+ from pyninjotiff.ninjotiff import save
+ from pyninjotiff.tifffile import TiffFile
+
+ area = FakeArea({'ellps': 'WGS84', 'lat_0': 90.0, 'lat_ts': 60.0, 'lon_0': 0.0, 'proj': 'stere'},
+ (-1000000.0, -4500000.0, 2072000.0, -1428000.0),
+ 1024, 1024)
+ scale = 1.0 / 120
+ offset = 0.0
+ attrs = dict([('resolution', 1050),
+ ('polarization', None),
+ ('platform_name', 'NOAA-18'),
+ ('sensor', 'avhrr-3'),
+ ('units', '%'),
+ ('name', '1'),
+ ('level', None),
+ ('modifiers', ()),
+ ('wavelength', (10.3, 10.8, 11.3)),
+ ('calibration', 'brightness_temperature'),
+ ('start_time', TIME - datetime.timedelta(minutes=5)),
+ ('end_time', TIME),
+ ('area', area),
+ ('ancillary_variables', []),
+ ('enhancement_history', [{'offset': offset, 'scale': scale}])])
+
+ kwargs = {'ch_min_measurement_unit': np.array([0]),
+ 'ch_max_measurement_unit': np.array([120]),
+ 'compute': True, 'fill_value': None, 'sat_id': 6300014,
+ 'chan_id': 100015, 'data_cat': 'PORN', 'data_source': 'SMHI',
+ 'physic_unit': '%', 'nbits': 8}
+
+ data = da.tile(da.repeat(da.arange(4, chunks=1024) /
+ 3.0, 256), 1024).reshape((1, 1024, 1024))
+ data = xr.DataArray(data, coords={'bands': ['L']}, dims=[
+ 'bands', 'y', 'x'], attrs=attrs)
+ img = FakeImage(data)
+ with tempfile.NamedTemporaryFile(delete=DELETE_FILES) as tmpfile:
+ filename = tmpfile.name
+ if not DELETE_FILES:
+ print(filename)
+ save(img, filename, data_is_scaled_01=True, **kwargs)
+ tif = TiffFile(filename)
+ res = tif[0].asarray()
+ assert(np.allclose(res[0, 0, ::256],
+ np.array([256, 22016, 43520, 65280])))
+
+
+def test_write_bw_inverted_ir():
+ """Test saving a BW image."""
+ from pyninjotiff.ninjotiff import save
+ from pyninjotiff.tifffile import TiffFile
+
+ area = FakeArea({'ellps': 'WGS84', 'lat_0': 90.0, 'lat_ts': 60.0, 'lon_0': 0.0, 'proj': 'stere'},
+ (-1000000.0, -4500000.0, 2072000.0, -1428000.0),
+ 1024, 1024)
+ scale = 1.0 / 120
+ offset = 70.0 / 120
+ attrs = dict([('resolution', 1050),
+ ('polarization', None),
+ ('platform_name', 'NOAA-18'),
+ ('sensor', 'avhrr-3'),
+ ('units', 'K'),
+ ('name', '4'),
+ ('level', None),
+ ('modifiers', ()),
+ ('wavelength', (10.3, 10.8, 11.3)),
+ ('calibration', 'brightness_temperature'),
+ ('start_time', TIME - datetime.timedelta(minutes=15)),
+ ('end_time', TIME - datetime.timedelta(minutes=10)),
+ ('area', area),
+ ('ancillary_variables', []),
+ ('enhancement_history', [{'offset': offset, 'scale': scale}])])
+
+ kwargs = {'ch_min_measurement_unit': np.array([-70]),
+ 'ch_max_measurement_unit': np.array([50]),
+ 'compute': True, 'fill_value': None, 'sat_id': 6300014,
+ 'chan_id': 900015, 'data_cat': 'PORN', 'data_source': 'SMHI',
+ 'physic_unit': 'C', 'nbits': 8}
+
+ data = da.tile(da.repeat(da.arange(4, chunks=1024) /
+ 3.0, 256), 1024).reshape((1, 1024, 1024))
+ data = xr.DataArray(data, coords={'bands': ['L']}, dims=[
+ 'bands', 'y', 'x'], attrs=attrs)
+ img = FakeImage(data)
+ with tempfile.NamedTemporaryFile(delete=DELETE_FILES) as tmpfile:
+ filename = tmpfile.name
+ if not DELETE_FILES:
+ print(filename)
+ save(img, filename, data_is_scaled_01=True, **kwargs)
+ tif = TiffFile(filename)
+ res = tif[0].asarray()
+ assert(np.allclose(res[0, 0, ::256],
+ np.array([65024, 43264, 21760, 0])))
+
+
+def test_write_bw_fill():
+ """Test saving a BW image with transparency."""
+ from pyninjotiff.ninjotiff import save
+ from pyninjotiff.tifffile import TiffFile
+
+ area = FakeArea({'ellps': 'WGS84', 'lat_0': 90.0, 'lat_ts': 60.0, 'lon_0': 0.0, 'proj': 'stere'},
+ (-1000000.0, -4500000.0, 2072000.0, -1428000.0),
+ 1024, 1024)
+ scale = 1.0 / 120
+ offset = 0.0
+ attrs = dict([('resolution', 1050),
+ ('polarization', None),
+ ('platform_name', 'NOAA-18'),
+ ('sensor', 'avhrr-3'),
+ ('units', '%'),
+ ('name', '1'),
+ ('level', None),
+ ('modifiers', ()),
+ ('wavelength', (10.3, 10.8, 11.3)),
+ ('calibration', 'brightness_temperature'),
+ ('start_time', TIME - datetime.timedelta(minutes=25)),
+ ('end_time', TIME - datetime.timedelta(minutes=20)),
+ ('area', area),
+ ('ancillary_variables', []),
+ ('enhancement_history', [{'offset': offset, 'scale': scale}])])
+
+ kwargs = {'ch_min_measurement_unit': np.array([0]),
+ 'ch_max_measurement_unit': np.array([120]),
+ 'compute': True, 'fill_value': None, 'sat_id': 6300014,
+ 'chan_id': 100015, 'data_cat': 'PORN', 'data_source': 'SMHI',
+ 'physic_unit': '%', 'nbits': 8}
+
+ data1 = da.tile(da.repeat(da.arange(4, chunks=1024) /
+ 3.0, 256), 256).reshape((1, 256, 1024))
+ datanan = da.ones((1, 256, 1024), chunks=1024) * np.nan
+ data2 = da.tile(da.repeat(da.arange(4, chunks=1024) /
+ 3.0, 256), 512).reshape((1, 512, 1024))
+ data = da.concatenate((data1, datanan, data2), axis=1)
+ data = xr.DataArray(data, coords={'bands': ['L']}, dims=[
+ 'bands', 'y', 'x'], attrs=attrs)
+ img = FakeImage(data)
+ with tempfile.NamedTemporaryFile(delete=DELETE_FILES) as tmpfile:
+ filename = tmpfile.name
+ if not DELETE_FILES:
+ print(filename)
+ save(img, filename, data_is_scaled_01=True, **kwargs)
+ tif = TiffFile(filename)
+ res = tif[0].asarray()
+ assert(np.allclose(res[0, 0, ::256],
+ np.array([256, 22016, 43520, 65280])))
+
+
+def test_write_bw_inverted_ir_fill():
+ """Test saving a BW image with transparency."""
+ from pyninjotiff.ninjotiff import save
+ from pyninjotiff.tifffile import TiffFile
+
+ area = FakeArea({'ellps': 'WGS84', 'lat_0': 90.0, 'lat_ts': 60.0, 'lon_0': 0.0, 'proj': 'stere'},
+ (-1000000.0, -4500000.0, 2072000.0, -1428000.0),
+ 1024, 1024)
+ scale = 1.0 / 120
+ offset = 70.0 / 120
+ attrs = dict([('resolution', 1050),
+ ('polarization', None),
+ ('platform_name', 'NOAA-18'),
+ ('sensor', 'avhrr-3'),
+ ('units', 'K'),
+ ('name', '4'),
+ ('level', None),
+ ('modifiers', ()),
+ ('wavelength', (10.3, 10.8, 11.3)),
+ ('calibration', 'brightness_temperature'),
+ ('start_time', TIME - datetime.timedelta(minutes=35)),
+ ('end_time', TIME - datetime.timedelta(minutes=30)),
+ ('area', area),
+ ('ancillary_variables', []),
+ ('enhancement_history', [{'offset': offset, 'scale': scale}])])
+
+ kwargs = {'ch_min_measurement_unit': np.array([-70]),
+ 'ch_max_measurement_unit': np.array([50]),
+ 'compute': True, 'fill_value': None, 'sat_id': 6300014,
+ 'chan_id': 900015, 'data_cat': 'PORN', 'data_source': 'SMHI',
+ 'physic_unit': 'C', 'nbits': 8}
+
+ data1 = da.tile(da.repeat(da.arange(4, chunks=1024) /
+ 3.0, 256), 256).reshape((1, 256, 1024))
+ datanan = da.ones((1, 256, 1024), chunks=1024) * np.nan
+ data2 = da.tile(da.repeat(da.arange(4, chunks=1024) /
+ 3.0, 256), 512).reshape((1, 512, 1024))
+ data = da.concatenate((data1, datanan, data2), axis=1)
+ data = xr.DataArray(data, coords={'bands': ['L']}, dims=[
+ 'bands', 'y', 'x'], attrs=attrs)
+ img = FakeImage(data)
+ with tempfile.NamedTemporaryFile(delete=DELETE_FILES) as tmpfile:
+ filename = tmpfile.name
+ if not DELETE_FILES:
+ print(filename)
+ save(img, filename, data_is_scaled_01=True, **kwargs)
+ tif = TiffFile(filename)
+ res = tif[0].asarray()
+ assert(np.allclose(res[0, 0, ::256],
+ np.array([65024, 43264, 21760, 0])))
+
+
+def test_write_rgb():
+ """Test saving a non-trasparent RGB."""
+ from pyninjotiff.ninjotiff import save
+ from pyninjotiff.tifffile import TiffFile
+
+ area = FakeArea({'ellps': 'WGS84', 'lat_0': 90.0, 'lat_ts': 60.0, 'lon_0': 0.0, 'proj': 'stere'},
+ (-1000000.0, -4500000.0, 2072000.0, -1428000.0),
+ 1024, 1024)
+
+ x_size, y_size = 1024, 1024
+ arr = np.zeros((3, y_size, x_size))
+ radius = min(x_size, y_size) / 2.0
+ centre = x_size / 2, y_size / 2
+
+ for x in range(x_size):
+ for y in range(y_size):
+ rx = x - centre[0]
+ ry = y - centre[1]
+ s = ((x - centre[0])**2.0 + (y - centre[1])**2.0)**0.5 / radius
+ if s <= 1.0:
+ h = ((np.arctan2(ry, rx) / np.pi) + 1.0) / 2.0
+ rgb = colorsys.hsv_to_rgb(h, s, 1.0)
+ arr[:, y, x] = np.array(rgb)
+
+ attrs = dict([('platform_name', 'NOAA-18'),
+ ('resolution', 1050),
+ ('polarization', None),
+ ('level', None),
+ ('sensor', 'avhrr-3'),
+ ('ancillary_variables', []),
+ ('area', area),
+ ('start_time', TIME - datetime.timedelta(minutes=45)),
+ ('end_time', TIME - datetime.timedelta(minutes=40)),
+ ('wavelength', None),
+ ('optional_datasets', []),
+ ('standard_name', 'overview'),
+ ('name', 'overview'),
+ ('prerequisites', [0.6, 0.8, 10.8]),
+ ('optional_prerequisites', []),
+ ('calibration', None),
+ ('modifiers', None),
+ ('mode', 'RGB'),
+ ('enhancement_history', [{'scale': np.array([1, 1, -1]), 'offset': np.array([0, 0, 1])},
+ {'scale': np.array([0.0266347, 0.03559078, 0.01329783]),
+ 'offset': np.array([-0.02524969, -0.01996642, 3.8918446])},
+ {'gamma': 1.6}])])
+
+ kwargs = {'compute': True, 'fill_value': None, 'sat_id': 6300014,
+ 'chan_id': 6500015, 'data_cat': 'PPRN', 'data_source': 'SMHI', 'nbits': 8}
+ data = da.from_array(arr.clip(0, 1), chunks=1024)
+ data = xr.DataArray(data, coords={'bands': ['R', 'G', 'B']}, dims=[
+ 'bands', 'y', 'x'], attrs=attrs)
+
+ from trollimage.xrimage import XRImage
+ img = XRImage(data)
+
+ with tempfile.NamedTemporaryFile(delete=DELETE_FILES) as tmpfile:
+ filename = tmpfile.name
+ if not DELETE_FILES:
+ print(filename)
+ save(img, filename, data_is_scaled_01=False, **kwargs)
+ tif = TiffFile(filename)
+ res = tif[0].asarray()
+ for idx in range(3):
+ np.testing.assert_allclose(res[:, :, idx], np.round(
+ arr[idx, :, :] * 255).astype(np.uint8))
+
+
+def test_write_rgb_with_a():
+ """Test saving a transparent RGB."""
+ from pyninjotiff.ninjotiff import save
+ from pyninjotiff.tifffile import TiffFile
+
+ area = FakeArea({'ellps': 'WGS84', 'lat_0': 90.0, 'lat_ts': 60.0, 'lon_0': 0.0, 'proj': 'stere'},
+ (-1000000.0, -4500000.0, 2072000.0, -1428000.0),
+ 1024, 1024)
+
+ x_size, y_size = 1024, 1024
+ arr = np.zeros((3, y_size, x_size))
+ radius = min(x_size, y_size) / 2.0
+ centre = x_size / 2, y_size / 2
+
+ for x in range(x_size):
+ for y in range(y_size):
+ rx = x - centre[0]
+ ry = y - centre[1]
+ s = ((x - centre[0])**2.0 + (y - centre[1])**2.0)**0.5 / radius
+ if s <= 1.0:
+ h = ((np.arctan2(ry, rx) / np.pi) + 1.0) / 2.0
+ rgb = colorsys.hsv_to_rgb(h, s, 1.0)
+ arr[:, y, x] = np.array(rgb)
+ else:
+ arr[:, y, x] = np.nan
+
+ attrs = dict([('platform_name', 'NOAA-18'),
+ ('resolution', 1050),
+ ('polarization', None),
+ ('start_time', TIME - datetime.timedelta(minutes=55)),
+ ('end_time', TIME - datetime.timedelta(minutes=50)),
+ ('level', None),
+ ('sensor', 'avhrr-3'),
+ ('ancillary_variables', []),
+ ('area', area),
+ ('wavelength', None),
+ ('optional_datasets', []),
+ ('standard_name', 'overview'),
+ ('name', 'overview'),
+ ('prerequisites', [0.6, 0.8, 10.8]),
+ ('optional_prerequisites', []),
+ ('calibration', None),
+ ('modifiers', None),
+ ('mode', 'RGB'),
+ ('enhancement_history', [{'scale': np.array([1, 1, -1]), 'offset': np.array([0, 0, 1])},
+ {'scale': np.array([0.0266347, 0.03559078, 0.01329783]),
+ 'offset': np.array([-0.02524969, -0.01996642, 3.8918446])},
+ {'gamma': 1.6}])])
+
+ kwargs = {'compute': True, 'fill_value': None, 'sat_id': 6300014,
+ 'chan_id': 6500015, 'data_cat': 'PPRN', 'data_source': 'SMHI', 'nbits': 8}
+ data = da.from_array(arr.clip(0, 1), chunks=1024)
+
+ data = xr.DataArray(data, coords={'bands': ['R', 'G', 'B']}, dims=[
+ 'bands', 'y', 'x'], attrs=attrs)
+ from trollimage.xrimage import XRImage
+ img = XRImage(data)
+ with tempfile.NamedTemporaryFile(delete=DELETE_FILES) as tmpfile:
+ filename = tmpfile.name
+ if not DELETE_FILES:
+ print(filename)
+ save(img, filename, data_is_scaled_01=True, **kwargs)
+ tif = TiffFile(filename)
+ res = tif[0].asarray()
+ for idx in range(3):
+ np.testing.assert_allclose(res[:, :, idx], np.round(
+ np.nan_to_num(arr[idx, :, :]) * 255).astype(np.uint8))
+ np.testing.assert_allclose(res[:, :, 3] == 0, np.isnan(arr[0, :, :]))
+
+
+ at pytest.mark.skip(reason="this is no implemented yet.")
+def test_write_rgb_classified():
+ """Test saving a transparent RGB."""
+ from pyninjotiff.ninjotiff import save
+ from pyninjotiff.tifffile import TiffFile
+
+ area = FakeArea({'ellps': 'WGS84', 'lat_0': 90.0, 'lat_ts': 60.0, 'lon_0': 0.0, 'proj': 'stere'},
+ (-1000000.0, -4500000.0, 2072000.0, -1428000.0),
+ 1024, 1024)
+
+ x_size, y_size = 1024, 1024
+ arr = np.zeros((3, y_size, x_size))
+
+ attrs = dict([('platform_name', 'NOAA-18'),
+ ('resolution', 1050),
+ ('polarization', None),
+ ('start_time', TIME - datetime.timedelta(minutes=55)),
+ ('end_time', TIME - datetime.timedelta(minutes=50)),
+ ('level', None),
+ ('sensor', 'avhrr-3'),
+ ('ancillary_variables', []),
+ ('area', area),
+ ('wavelength', None),
+ ('optional_datasets', []),
+ ('standard_name', 'overview'),
+ ('name', 'overview'),
+ ('prerequisites', [0.6, 0.8, 10.8]),
+ ('optional_prerequisites', []),
+ ('calibration', None),
+ ('modifiers', None),
+ ('mode', 'P')])
+
+ kwargs = {'compute': True, 'fill_value': None, 'sat_id': 6300014,
+ 'chan_id': 1700015, 'data_cat': 'PPRN', 'data_source': 'SMHI', 'nbits': 8}
+
+ data1 = da.tile(da.repeat(da.arange(4, chunks=1024), 256), 256).reshape((1, 256, 1024))
+ datanan = da.ones((1, 256, 1024), chunks=1024) * 4
+ data2 = da.tile(da.repeat(da.arange(4, chunks=1024), 256), 512).reshape((1, 512, 1024))
+ data = da.concatenate((data1, datanan, data2), axis=1)
+ data = xr.DataArray(data, coords={'bands': ['P']}, dims=['bands', 'y', 'x'], attrs=attrs)
+
+ from trollimage.xrimage import XRImage
+ img = XRImage(data)
+ with tempfile.NamedTemporaryFile(delete=DELETE_FILES) as tmpfile:
+ filename = tmpfile.name
+ if not DELETE_FILES:
+ print(filename)
+ save(img, filename, data_is_scaled_01=True, **kwargs)
+ tif = TiffFile(filename)
+ res = tif[0].asarray()
+ for idx in range(3):
+ np.testing.assert_allclose(res[:, :, idx], np.round(
+ np.nan_to_num(arr[idx, :, :]) * 255).astype(np.uint8))
+ np.testing.assert_allclose(res[:, :, 3] == 0, np.isnan(arr[0, :, :]))
=====================================
pyninjotiff/tifffile.py
=====================================
@@ -337,7 +337,7 @@ class TiffWriter(object):
def save(self, data, photometric=None, planarconfig=None, resolution=None,
description=None, volume=False, writeshape=False, compress=0,
- colormap=None, extrasamples_type=1, tile_width=None,
+ colormap=None, extrasamples_type=1, tile_width=None,
tile_length=None, extratags=()):
"""Write image data to TIFF file.
@@ -711,8 +711,8 @@ class TiffWriter(object):
# reset and use compress sizes
strip_byte_counts = []
for plane in data[pageindex]:
- for ty in xrange(0, tiles_y):
- for tx in xrange(0, tiles_x):
+ for ty in range(0, tiles_y):
+ for tx in range(0, tiles_x):
# allocate fixed size tile filled with zeros
tile = numpy.zeros((tile_width * tile_length,
shape[-1]), data.dtype)
@@ -724,7 +724,7 @@ class TiffWriter(object):
itw = min(tile_width,
shape[3] - tx*tile_width)
ioffs = tx*tile_width
- for tl in xrange(0, itl):
+ for tl in range(0, itl):
# copy data to tile line
ir = ty*tile_length+tl
tile[tl*tile_width:tl*tile_width+itw] \
@@ -4989,4 +4989,3 @@ if sys.version_info[0] > 2:
if __name__ == "__main__":
sys.exit(main())
-
=====================================
pyninjotiff/version.py
=====================================
@@ -22,4 +22,4 @@
"""Version file."""
-__version__ = "v0.1.0"
+__version__ = "v0.2.0"
=====================================
setup.cfg
=====================================
@@ -0,0 +1,2 @@
+[flake8]
+max-line-length = 120
=====================================
setup.py
=====================================
@@ -31,7 +31,7 @@ version = imp.load_source('pyninjotiff.version', 'pyninjotiff/version.py')
setup(name="pyninjotiff",
version=version.__version__,
- description='Pytroll imaging library',
+ description='Python Ninjo TIFF writing library',
author='Martin Raspaud',
author_email='martin.raspaud at smhi.se',
classifiers=["Development Status :: 5 - Production/Stable",
@@ -44,6 +44,6 @@ setup(name="pyninjotiff",
url="https://github.com/pytroll/pyninjotiff",
packages=['pyninjotiff'],
zip_safe=False,
- install_requires=['numpy >=1.6', 'six'],
+ install_requires=['numpy >=1.6', 'six', 'pyproj', 'pyresample'],
# test_suite='pyninjotiff.tests.suite',
)
View it on GitLab: https://salsa.debian.org/debian-gis-team/pyninjotiff/compare/8e6d804c56c1ac3abf37a8c08cd47b5739430f24...64a19fe966789a6558e1fd54c56b727f154c3b82
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/pyninjotiff/compare/8e6d804c56c1ac3abf37a8c08cd47b5739430f24...64a19fe966789a6558e1fd54c56b727f154c3b82
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20190921/b761d2a4/attachment-0001.html>
More information about the Pkg-grass-devel
mailing list