[Git][debian-gis-team/pyspectral][upstream] New upstream version 0.10.0+ds
Antonio Valentino
gitlab at salsa.debian.org
Sun Jun 28 09:09:44 BST 2020
Antonio Valentino pushed to branch upstream at Debian GIS Project / pyspectral
Commits:
865fc37f by Antonio Valentino at 2020-06-28T07:57:42+00:00
New upstream version 0.10.0+ds
- - - - -
11 changed files:
- CHANGELOG.md
- RELEASING.md
- doc/platforms_supported.rst
- pyspectral/near_infrared_reflectance.py
- pyspectral/radiance_tb_conversion.py
- pyspectral/tests/test_reflectance.py
- pyspectral/utils.py
- + rsr_convert_scripts/fci_rsr.py
- rsr_convert_scripts/metimage_rsr.py
- rsr_convert_scripts/seviri_rsr.py
- rsr_convert_scripts/slstr_rsr.py
Changes:
=====================================
CHANGELOG.md
=====================================
@@ -1,3 +1,27 @@
+## Version v0.10.0 (2020/06/24)
+
+### Issues Closed
+
+* [Issue 96](https://github.com/pytroll/pyspectral/issues/96) - Spectral response function for SLSTR on Sentinel 3B ([PR 104](https://github.com/pytroll/pyspectral/pull/104))
+
+In this release 1 issue was closed.
+
+### Pull Requests Merged
+
+#### Bugs fixed
+
+* [PR 105](https://github.com/pytroll/pyspectral/pull/105) - Use original channel data on the night side for NIR emissive
+
+#### Features added
+
+* [PR 108](https://github.com/pytroll/pyspectral/pull/108) - Add option to specify sun-zenith angle threshold applied
+* [PR 107](https://github.com/pytroll/pyspectral/pull/107) - Add support for FCI
+* [PR 105](https://github.com/pytroll/pyspectral/pull/105) - Use original channel data on the night side for NIR emissive
+
+In this release 4 pull requests were closed.
+
+
+
## Version v0.9.5 (2020/02/04)
### Issues Closed
=====================================
RELEASING.md
=====================================
@@ -9,7 +9,7 @@ prerequisites: `pip install setuptools twine`
4. run `loghub` and update the `CHANGELOG.md` file:
```
-loghub pytroll/pyspectral -u <username> -st v0.8.3 -plg bug "Bugs fixed" -plg enhancement "Features added" -plg documentation "Documentation changes"
+loghub pytroll/pyspectral --token <personal access token (see https://github.com/settings/tokens)> -st v<previous version> -plg bug "Bugs fixed" -plg enhancement "Features added" -plg documentation "Documentation changes"
```
Don't forget to commit!
@@ -17,7 +17,7 @@ Don't forget to commit!
5. Create a tag with the new version number, starting with a 'v', eg:
```
-git tag v0.8.4 -m "Version 0.8.4"
+git tag -a v0.8.4 -m "Version 0.8.4"
```
See [semver.org](http://semver.org/) on how to write a version number.
=====================================
doc/platforms_supported.rst
=====================================
@@ -61,6 +61,9 @@ have been included in PySpectral.
* - Sentinel-3A slstr
- `rsr_slstr_Sentinel-3A.h5`
- ESA-Sentinel-SLSTR_
+ * - Sentinel-3B slstr
+ - `rsr_slstr_Sentinel-3B.h5`
+ - ESA-Sentinel-SLSTR_
* - Sentinel-3A olci
- `rsr_olci_Sentinel-3A.h5`
- ESA-Sentinel-OLCI_
@@ -88,6 +91,13 @@ have been included in PySpectral.
* - Metop-SG-A1 MetImage
- `rsr_metimage_Metop-SG-A1.h5`
- NWPSAF-MetImage_
+ * - Meteosat-12 fci
+ - `rsr_fci_Meteosat-12.h5`
+ - NWPSAF-Meteosat-12-fci_
+ * - MTG-I1 fci (NB! Identical to Meteosat-12 fci)
+ - `rsr_fci_MTG-I1.h5`
+ - NWPSAF-Meteosat-12-fci_
+
.. _Eumetsat: https://www.eumetsat.int/website/home/Data/Products/Calibration/MSGCalibration/index.html
.. _GSICS: https://www.star.nesdis.noaa.gov/smcd/GCC/instrInfo-srf.php
@@ -95,11 +105,12 @@ have been included in PySpectral.
.. _JMA: http://www.data.jma.go.jp/mscweb/en/himawari89/space_segment/spsg_ahi.html#srf
.. _ESA-Envisat: http://envisat.esa.int/handbooks/aatsr/aux-files/consolidatedsrfs.xls
.. _ESA-Sentinel-OLCI: https://sentinel.esa.int/documents/247904/322304/OLCI+SRF+%28NetCDF%29/15cfd7a6-b7bc-4051-87f8-c35d765ae43a
-.. _ESA-Sentinel-SLSTR: https://sentinel.esa.int/documents/247904/322305/SLSTR_FM02_Spectral_Responses_Necdf_zip/3a4482b8-6e44-47f3-a8f2-79c000663976
+.. _ESA-Sentinel-SLSTR: https://sentinel.esa.int/web/sentinel/technical-guides/sentinel-3-slstr/instrument/measured-spectral-response-function-data
.. _ESA-Sentinel-MSI: https://earth.esa.int/documents/247904/685211/S2-SRF_COPE-GSEG-EOPG-TN-15-0007_3.0.xlsx
.. _NASA-Landsat-OLI: https://landsat.gsfc.nasa.gov/wp-content/uploads/2013/06/Ball_BA_RSR.v1.1-1.xlsx
.. _NESDIS: https://ncc.nesdis.noaa.gov/J1VIIRS/J1VIIRSSpectralResponseFunctions.php
.. _CMA: http://www.cma.gov.cn/en2014/
.. _NWPSAF-MetImage: https://nwpsaf.eu/downloads/rtcoef_rttov12/ir_srf/rtcoef_metopsg_1_metimage_srf.html
.. _NWPSAF-GeoKompsat-2A-ami: https://nwpsaf.eu/downloads/rtcoef_rttov12/ir_srf/rtcoef_gkompsat2_1_ami_srf.html
+.. _NWPSAF-Meteosat-12-fci: https://nwpsaf.eu/downloads/rtcoef_rttov12/ir_srf/rtcoef_mtg_1_fci_srf.html
.. _NSMC-fy4a: http://fy4.nsmc.org.cn/portal/cn/fycv/srf.html
=====================================
pyspectral/near_infrared_reflectance.py
=====================================
@@ -1,7 +1,7 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
-# Copyright (c) 2014-2019 Adam.Dybbroe
+# Copyright (c) 2014-2020 Adam.Dybbroe
# Author(s):
@@ -32,24 +32,24 @@ window channel (usually around 11-12 microns).
import os
import numpy as np
try:
- from dask.array import where, logical_or, asanyarray, array
+ from dask.array import where, logical_or, asanyarray, array, isnan
except ImportError:
- from numpy import where, logical_or, asanyarray, array
+ from numpy import where, logical_or, asanyarray, array, isnan
from pyspectral.solar import (SolarIrradianceSpectrum,
TOTAL_IRRADIANCE_SPECTRUM_2000ASTM)
from pyspectral.utils import BANDNAMES, get_bandname_from_wavelength
from pyspectral.utils import TB2RAD_DIR
+from pyspectral.utils import WAVE_LENGTH
from pyspectral.radiance_tb_conversion import RadTbConverter
from pyspectral.config import get_config
-
import logging
LOG = logging.getLogger(__name__)
-
EPSILON = 0.005
TB_MIN = 150.
TB_MAX = 360.
+TERMINATOR_LIMIT = 85.0
class Calculator(RadTbConverter):
@@ -65,9 +65,11 @@ class Calculator(RadTbConverter):
The relfectance calculated is without units and should be between 0 and 1.
"""
- def __init__(self, platform_name, instrument, band, **kwargs):
+ def __init__(self, platform_name, instrument, band,
+ detector='det-1', wavespace=WAVE_LENGTH,
+ solar_flux=None, sunz_threshold=TERMINATOR_LIMIT):
"""Initialize the Class instance."""
- super(Calculator, self).__init__(platform_name, instrument, band, **kwargs)
+ super(Calculator, self).__init__(platform_name, instrument, band, detector=detector, wavespace=wavespace)
from numbers import Number
self.bandname = None
@@ -87,10 +89,14 @@ class Calculator(RadTbConverter):
options = get_config()
- self.solar_flux = kwargs.get('solar_flux', None)
+ self.solar_flux = solar_flux
if self.solar_flux is None:
self._get_solarflux()
+ # The sun-zenith angle limit in degrees defining how far towards the
+ # terminator we try derive a
+ self.detector = detector
+ self.sunz_threshold = sunz_threshold
self._rad3x = None
self._rad3x_t11 = None
self._solar_radiance = None
@@ -99,11 +105,6 @@ class Calculator(RadTbConverter):
self._e3x = None
self.lutfile = None
- if 'detector' in kwargs:
- self.detector = kwargs['detector']
- else:
- self.detector = 'det-1'
-
platform_sensor = platform_name + '-' + instrument
if platform_sensor in options and 'tb2rad_lut_filename' in options[platform_sensor]:
if isinstance(options[platform_sensor]['tb2rad_lut_filename'], dict):
@@ -171,6 +172,8 @@ class Calculator(RadTbConverter):
try:
# Emissive part:
self._e3x = self._rad3x_t11 * (1 - self._r3x)
+ # Use the original channel data on the night side
+ self._e3x = where(isnan(self._e3x), self._rad3x, self._e3x)
# Unsure how much sense it makes to apply the co2 correction term here!?
# FIXME!
# self._e3x *= self._rad3x_correction
@@ -252,19 +255,23 @@ class Calculator(RadTbConverter):
self._rad3x_t11 = self.tb2radiance(tb_therm, lut=lut)['radiance']
thermal_emiss_one = self._rad3x_t11 * self.rsr_integral
- l_nir = self.tb2radiance(tb_nir, lut=lut)['radiance'] * self.rsr_integral
+ l_nir = self.tb2radiance(tb_nir, lut=lut)['radiance']
+ self._rad3x = l_nir.copy()
+ l_nir *= self.rsr_integral
if thermal_emiss_one.ravel().shape[0] < 10:
LOG.info('thermal_emiss_one = %s', str(thermal_emiss_one))
if l_nir.ravel().shape[0] < 10:
LOG.info('l_nir = %s', str(l_nir))
- sunzmask = (sun_zenith < 0.0) | (sun_zenith > 88.0)
- sunz = sun_zenith.clip(0, 88.0)
-
+ LOG.debug("Apply sun-zenith angle clipping between 0 and %5.2f", self.sunz_threshold)
+ sunzmask = (sun_zenith < 0.0) | (sun_zenith > self.sunz_threshold)
+ # Could do a more smooth transition here:ยง
+ # FIXME!
+ sunz = sun_zenith.clip(0, self.sunz_threshold)
mu0 = np.cos(np.deg2rad(sunz))
+
# mu0 = np.where(np.less(mu0, 0.1), 0.1, mu0)
- self._rad3x = l_nir
self._solar_radiance = self.solar_flux * mu0 / np.pi
# CO2 correction to the 3.9 radiance, only if tbs of a co2 band around
=====================================
pyspectral/radiance_tb_conversion.py
=====================================
@@ -1,7 +1,7 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
-# Copyright (c) 2014-2019 Adam.Dybbroe
+# Copyright (c) 2014-2020 Adam.Dybbroe
# Author(s):
@@ -99,7 +99,8 @@ class RadTbConverter(object):
"""
- def __init__(self, platform_name, instrument, band, **options):
+ def __init__(self, platform_name, instrument, band, detector='det-1', wavespace=WAVE_LENGTH,
+ tb_resolution=0.1):
"""Initialize the Class instance.
E.g.:
@@ -116,7 +117,7 @@ class RadTbConverter(object):
self.bandwavelength = None
self.band = band
- self.wavespace = options.get('wavespace', WAVE_LENGTH)
+ self.wavespace = wavespace
if self.wavespace not in [WAVE_LENGTH, WAVE_NUMBER]:
raise AttributeError('Wave space not {0} or {1}!'.format(WAVE_LENGTH,
WAVE_NUMBER))
@@ -124,8 +125,8 @@ class RadTbConverter(object):
self._wave_unit = 'm'
self._wave_si_scale = 1.0
- self.detector = options.get('detector', 'det-1')
- self.tb_resolution = options.get('tb_resolution', 0.1)
+ self.detector = detector
+ self.tb_resolution = tb_resolution
self.tb_scale = 1. / self.tb_resolution
self.blackbody_function = BLACKBODY_FUNC[self.wavespace]
=====================================
pyspectral/tests/test_reflectance.py
=====================================
@@ -1,11 +1,11 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
-# Copyright (c) 2013-2019 Adam.Dybbroe
+# Copyright (c) 2013-2020 Pytroll
# Author(s):
-# Adam.Dybbroe <a000680 at c14526.ad.smhi.se>
+# Adam.Dybbroe <adam.dybbroe at smhi.se>
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -145,6 +145,8 @@ class TestReflectance(unittest.TestCase):
refl37 = Calculator('Suomi-NPP', 'viirs', 3.7)
self.assertEqual(refl37.bandwavelength, 3.7)
self.assertEqual(refl37.bandname, 'ch20')
+ # Default sunz-threshold used to stay on day side and away from terminator:
+ self.assertEqual(refl37.sunz_threshold, 85.0)
with patch('pyspectral.radiance_tb_conversion.RelativeSpectralResponse') as mymock:
instance = mymock.return_value
@@ -154,12 +156,32 @@ class TestReflectance(unittest.TestCase):
refl37 = Calculator('EOS-Aqua', 'modis', '20')
+ refl37_sz88 = Calculator('EOS-Aqua', 'modis', '20', sunz_threshold=88.0)
+ self.assertEqual(refl37_sz88.sunz_threshold, 88.0)
+ self.assertAlmostEqual(refl37_sz88.bandwavelength, 3.780282, 5)
+ self.assertEqual(refl37_sz88.bandname, '20')
+
sunz = np.array([80.])
tb3 = np.array([290.])
tb4 = np.array([282.])
refl = refl37.reflectance_from_tbs(sunz, tb3, tb4)
np.testing.assert_allclose(refl[0], 0.251245010648, 6)
+ sunz = np.array([85.])
+ tb3 = np.array([290.])
+ tb4 = np.array([282.])
+ refl = refl37.reflectance_from_tbs(sunz, tb3, tb4)
+ np.testing.assert_allclose(refl[0], 1.12236884, 6)
+
+ sunz = np.array([85.1])
+ refl = refl37.reflectance_from_tbs(sunz, tb3, tb4)
+ self.assertTrue(np.isnan(refl[0]))
+
+ refl_sz88 = refl37_sz88.reflectance_from_tbs(sunz, tb3, tb4)
+ np.testing.assert_allclose(refl_sz88[0], 1.2064644, 6)
+ sunz = np.array([86.0])
+ self.assertTrue(np.isnan(refl[0]))
+
tb3x = refl37.emissive_part_3x()
np.testing.assert_allclose(tb3x, 276.213054, 6)
=====================================
pyspectral/utils.py
=====================================
@@ -1,7 +1,7 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
-# Copyright (c) 2014-2019 Pytroll
+# Copyright (c) 2014-2020 Pytroll
# Author(s):
@@ -128,6 +128,24 @@ BANDNAMES['ami'] = {'VI004': 'ch1',
'IR133': 'ch16'
}
+BANDNAMES['fci'] = {'vis_04': 'ch1',
+ 'vis_05': 'ch2',
+ 'vis_06': 'ch3',
+ 'vis_08': 'ch4',
+ 'vis_09': 'ch5',
+ 'nir_13': 'ch6',
+ 'nir_16': 'ch7',
+ 'nir_22': 'ch8',
+ 'ir_38': 'ch9',
+ 'wv_63': 'ch10',
+ 'wv_73': 'ch11',
+ 'ir_87': 'ch12',
+ 'ir_97': 'ch13',
+ 'ir_105': 'ch14',
+ 'ir_123': 'ch15',
+ 'ir_133': 'ch16'
+ }
+
INSTRUMENTS = {'NOAA-19': 'avhrr/3',
'NOAA-18': 'avhrr/3',
'NOAA-17': 'avhrr/3',
@@ -152,13 +170,16 @@ INSTRUMENTS = {'NOAA-19': 'avhrr/3',
'FY-3B': 'virr',
'Feng-Yun 3D': 'mersi-2',
'FY-4A': 'agri',
- 'GEO-KOMPSAT-2A': 'ami'
+ 'GEO-KOMPSAT-2A': 'ami',
+ 'MTG-I1': 'fci'
}
-HTTP_PYSPECTRAL_RSR = "https://zenodo.org/record/3461164/files/pyspectral_rsr_data.tgz"
+HTTP_PYSPECTRAL_RSR = "https://zenodo.org/record/3824535/files/pyspectral_rsr_data.tgz"
+
+
RSR_DATA_VERSION_FILENAME = "PYSPECTRAL_RSR_VERSION"
-RSR_DATA_VERSION = "v1.0.10"
+RSR_DATA_VERSION = "v1.0.15"
ATM_CORRECTION_LUT_VERSION = {}
ATM_CORRECTION_LUT_VERSION['antarctic_aerosol'] = {'version': 'v1.0.1',
=====================================
rsr_convert_scripts/fci_rsr.py
=====================================
@@ -0,0 +1,87 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2020 Pytroll Community
+
+# Author(s):
+
+# Adam.Dybbroe <adam.dybbroe at smhi.se>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+"""Read the MTG FCI relative spectral response functions.
+
+Data from EUMETSAT NWP-SAF:
+https://nwpsaf.eu/downloads/rtcoef_rttov12/ir_srf/rtcoef_mtg_1_fci_srf.html
+"""
+
+import os
+import logging
+import numpy as np
+from pyspectral.utils import convert2hdf5 as tohdf5
+from pyspectral.raw_reader import InstrumentRSR
+
+LOG = logging.getLogger(__name__)
+
+FCI_BAND_NAMES = ['ch1', 'ch2', 'ch3', 'ch4', 'ch5', 'ch6', 'ch7', 'ch8', 'ch9', 'ch10',
+ 'ch11', 'ch12', 'ch13', 'ch14', 'ch15', 'ch16']
+
+
+class FciRSR(InstrumentRSR):
+ """Container for the MTG FCI RSR data."""
+
+ def __init__(self, bandname, platform_name):
+ """Setup the MTG FCI RSR data container."""
+ super(FciRSR, self).__init__(bandname, platform_name, FCI_BAND_NAMES)
+
+ self.instrument = 'fci'
+
+ self._get_options_from_config()
+ self._get_bandfilenames()
+
+ LOG.debug("Filenames: %s", str(self.filenames))
+ if self.filenames[bandname] and os.path.exists(self.filenames[bandname]):
+ self.requested_band_filename = self.filenames[bandname]
+ self._load()
+
+ else:
+ LOG.warning("Couldn't find an existing file for this band: %s",
+ str(self.bandname))
+
+ # To be compatible with VIIRS....
+ self.filename = self.requested_band_filename
+
+ def _load(self, scale=10000.0):
+ """Load the FCI RSR data for the band requested."""
+ data = np.genfromtxt(self.requested_band_filename,
+ unpack=True,
+ names=['wavenumber',
+ 'response'],
+ skip_header=4)
+
+ # Data are wavenumbers in cm-1:
+ wavelength = 1. / data['wavenumber'] * scale
+ response = data['response']
+
+ self.rsr = {'wavelength': wavelength[::-1], 'response': response[::-1]}
+
+
+def main():
+ """Main function creating the internal Pyspectral hdf5 output for FCI."""
+ for platform_name in ["Meteosat-12", 'MTG-I1']:
+ tohdf5(FciRSR, platform_name, FCI_BAND_NAMES)
+
+
+if __name__ == "__main__":
+ main()
=====================================
rsr_convert_scripts/metimage_rsr.py
=====================================
@@ -21,8 +21,7 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""Read the preliminary MetImage relative spectral response functions.
-Data from the NWPSAF, These are very early theoretical and idealized versions,
-derived from specifications I assume.
+Data from the NWPSAF, These are simulated function.
"""
@@ -68,16 +67,17 @@ class MetImageRSR(InstrumentRSR):
self.unit = 'micrometer'
self.wavespace = 'wavelength'
- def _load(self, scale=1.0):
+ def _load(self, scale=0.001):
"""Load the MetImage RSR data for the band requested"""
data = np.genfromtxt(self.requested_band_filename,
unpack=True,
- names=['wavenumber',
+ names=['wavelength',
'response'],
- skip_header=4)
+ skip_header=0,
+ delimiter=',')
- # Data are wavenumbers in cm-1:
- wavelength = 1. / data['wavenumber'] * 10000.
+ # Data are in nanometer and need to transform to microns:
+ wavelength = data['wavelength'] * scale
response = data['response']
# The real MetImage has 24 detectors. However, for now we store the
=====================================
rsr_convert_scripts/seviri_rsr.py
=====================================
@@ -2,7 +2,7 @@
# -*- coding: utf-8 -*-
-# Copyright (c) 2013-2018 Adam.Dybbroe
+# Copyright (c) 2013-2018, 2020 Pytroll Community
# Author(s):
=====================================
rsr_convert_scripts/slstr_rsr.py
=====================================
@@ -23,7 +23,7 @@
"""
Sentinel-3 SLSTR spectral response function interface
-https://sentinel.esa.int/documents/247904/322305/SLSTR_FM02_Spectral_Responses_Necdf_zip/3a4482b8-6e44-47f3-a8f2-79c000663976
+https://sentinel.esa.int/web/sentinel/technical-guides/sentinel-3-slstr/instrument/measured-spectral-response-function-data
"""
@@ -81,7 +81,7 @@ class SlstrRSR(InstrumentRSR):
def main():
"""Main"""
- for platform_name in ['Sentinel-3A', ]:
+ for platform_name in ['Sentinel-3B', ]:
tohdf5(SlstrRSR, platform_name, SLSTR_BAND_NAMES)
View it on GitLab: https://salsa.debian.org/debian-gis-team/pyspectral/-/commit/865fc37f23b971c5ed660a0bbc5680745be30b2c
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/pyspectral/-/commit/865fc37f23b971c5ed660a0bbc5680745be30b2c
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20200628/d4270c19/attachment-0001.html>
More information about the Pkg-grass-devel
mailing list