[Git][debian-gis-team/pyspectral][master] 7 commits: Install files in pyspectral/etc

Antonio Valentino gitlab at salsa.debian.org
Mon Apr 22 10:29:51 BST 2019



Antonio Valentino pushed to branch master at Debian GIS Project / pyspectral


Commits:
4f862fe2 by Antonio Valentino at 2019-04-22T09:09:32Z
Install files in pyspectral/etc

- - - - -
f2019d73 by Antonio Valentino at 2019-04-22T09:09:47Z
New upstream version 0.8.7+ds
- - - - -
ae0689ab by Antonio Valentino at 2019-04-22T09:09:49Z
Update upstream source from tag 'upstream/0.8.7+ds'

Update to upstream version '0.8.7+ds'
with Debian dir 3125b67bd765921388a6ab2e82f48bda49e8b656
- - - - -
da90f934 by Antonio Valentino at 2019-04-22T09:10:42Z
New upstream release

- - - - -
3987ee08 by Antonio Valentino at 2019-04-22T09:14:17Z
Updated debian copyright

- - - - -
4dca6de1 by Antonio Valentino at 2019-04-22T09:16:18Z
Refresh all patches

- - - - -
cc7dfaaf by Antonio Valentino at 2019-04-22T09:28:50Z
Set distribution to unstable

- - - - -


23 changed files:

- CHANGELOG.md
- README.md
- RELEASING.md
- bin/composite_rsr_plot.py
- debian/changelog
- debian/copyright
- debian/patches/0001-Fix-tests.patch
- debian/python3-pyspectral.install
- doc/37_reflectance.rst
- doc/platforms_supported.rst
- doc/usage.rst
- pyspectral/blackbody.py
- pyspectral/etc/pyspectral.yaml
- pyspectral/near_infrared_reflectance.py
- pyspectral/radiance_tb_conversion.py
- pyspectral/raw_reader.py
- pyspectral/rayleigh.py
- pyspectral/tests/__init__.py
- pyspectral/tests/test_reflectance.py
- pyspectral/utils.py
- pyspectral/version.py
- + rsr_convert_scripts/cocts_rsr.py
- + rsr_convert_scripts/metimage_rsr.py


Changes:

=====================================
CHANGELOG.md
=====================================
@@ -1,3 +1,26 @@
+## Version <RELEASE_VERSION> (2019/04/09)
+
+### Issues Closed
+
+* [Issue 66](https://github.com/pytroll/pyspectral/issues/66) - Throws a warning about non-existing directory - storing/reading cached radiance-tb look-up-tables ([PR 67](https://github.com/pytroll/pyspectral/pull/67))
+* [Issue 61](https://github.com/pytroll/pyspectral/issues/61) - can this program be used for user-defined sensor or rsr? ([PR 62](https://github.com/pytroll/pyspectral/pull/62))
+* [Issue 58](https://github.com/pytroll/pyspectral/issues/58) - Use dask instead of numpy masked arrays ([PR 59](https://github.com/pytroll/pyspectral/pull/59))
+
+In this release 3 issues were closed.
+
+### Pull Requests Merged
+
+#### Bugs fixed
+
+* [PR 64](https://github.com/pytroll/pyspectral/pull/64) - Fix interp function in rayleigh correction to be serializable
+
+#### Features added
+
+* [PR 59](https://github.com/pytroll/pyspectral/pull/59) - Daskify NIR reflectance calculations ([58](https://github.com/pytroll/pyspectral/issues/58))
+
+In this release 2 pull requests were closed.
+
+
 ## Version <RELEASE_VERSION> (2018/12/04)
 
 ### Issues Closed


=====================================
README.md
=====================================
@@ -7,7 +7,6 @@ PySpectral
 [![Coverage Status](https://coveralls.io/repos/github/pytroll/pyspectral/badge.svg?branch=master)](https://coveralls.io/github/pytroll/pyspectral?branch=master)
 [![Code Health](https://landscape.io/github/pytroll/pyspectral/master/landscape.png)](https://landscape.io/github/pytroll/pyspectral/master)
 [![PyPI version](https://badge.fury.io/py/pyspectral.svg)](https://badge.fury.io/py/pyspectral)
-[![Research software impact](http://depsy.org/api/package/pypi/pyspectral/badge.svg)](http://depsy.org/package/python/pyspectral)
 [![Code Climate](https://codeclimate.com/github/pytroll/pyspectral/badges/gpa.svg)](https://codeclimate.com/github/pytroll/pyspectral)
 [![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/pytroll/pyspectral/badges/quality-score.png?b=master)](https://scrutinizer-ci.com/g/pytroll/pyspectral/?branch=master)
 


=====================================
RELEASING.md
=====================================
@@ -17,7 +17,7 @@ Don't forget to commit!
 5. Create a tag with the new version number, starting with a 'v', eg:
 
 ```
-git tag v0.8.4 -m "Version 0.8.4
+git tag v0.8.4 -m "Version 0.8.4"
 ```
 
 See [semver.org](http://semver.org/) on how to write a version number.


=====================================
bin/composite_rsr_plot.py
=====================================
@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
-# Copyright (c) 2016-2018 Adam.Dybbroe
+# Copyright (c) 2016-2019 Adam.Dybbroe
 
 # Author(s):
 
@@ -74,6 +74,9 @@ def get_arguments():
     parser.add_argument("-x", "--xlimits", nargs=2,
                         help=("x-axis boundaries for plot"),
                         default=None, type=float)
+    parser.add_argument("-y", "--ylimits", nargs=2,
+                        help=("y-axis boundaries for plot"),
+                        default=None, type=float)
     parser.add_argument("-t", "--minimum_response",
                         help=("Minimum response: Any response lower than " +
                               "this will be ignored when plotting"),
@@ -115,6 +118,7 @@ if __name__ == "__main__":
     sensors = args.sensor
     minimum_response = args.minimum_response
     xlimits = args.xlimits
+    ylimits = args.ylimits
     title = args.title
     if not title:
         title = 'Relative Spectral Responses'
@@ -211,6 +215,13 @@ if __name__ == "__main__":
 
     plt.xlim((wmin, wmax))
 
+    wmin, wmax = plt.ylim()
+    if ylimits:
+        wmin = ylimits[0]
+        wmax = ylimits[1]
+
+    plt.ylim((wmin, wmax))
+
     plt.title(title)
     plt.legend(loc='lower right')
     if filename:


=====================================
debian/changelog
=====================================
@@ -1,3 +1,13 @@
+pyspectral (0.8.7+ds-1) unstable; urgency=medium
+
+  * New upstream release.
+  * Install files in pyspectral/etc.
+  * Updated debian/copyright.
+  * debian/patches:
+    - refresh all patches
+
+ -- Antonio Valentino <antonio.valentino at tiscali.it>  Mon, 22 Apr 2019 09:28:29 +0000
+
 pyspectral (0.8.6+ds-1) unstable; urgency=medium
 
   * Initial version (Closes: #917351)


=====================================
debian/copyright
=====================================
@@ -16,7 +16,7 @@ Files-Excluded: pyspectral/data/modis
  doc/_static/mersi2_rsr_band_0040_0070_missingbands.png
 
 Files: *
-Copyright: 2013-2018 Adam Dybbroe
+Copyright: 2013-2019 Adam Dybbroe
                      PyTroll Community
                      Pytroll
 License: GPL-3+


=====================================
debian/patches/0001-Fix-tests.patch
=====================================
@@ -8,7 +8,7 @@ Disable tests that require the network or external data
  1 file changed, 4 insertions(+), 4 deletions(-)
 
 diff --git a/pyspectral/tests/__init__.py b/pyspectral/tests/__init__.py
-index a4e8886..826d17b 100644
+index b930fc2..686ce89 100644
 --- a/pyspectral/tests/__init__.py
 +++ b/pyspectral/tests/__init__.py
 @@ -53,10 +53,10 @@ def suite():


=====================================
debian/python3-pyspectral.install
=====================================
@@ -1,3 +1,4 @@
 usr/lib/python3*/dist-packages/*/*.py
+usr/lib/python3*/dist-packages/*/etc/*
 usr/lib/python3*/dist-packages/*/data/*
 usr/lib/python3*/dist-packages/*.egg-info/*


=====================================
doc/37_reflectance.rst
=====================================
@@ -45,7 +45,7 @@ expressed in :math:`W/m^2 sr^{-1} \mu m^{-1}`, or using SI units :math:`W/m^2 sr
   >>> tb37 = np.array([298.07385254, 297.15478516, 294.43276978, 281.67633057, 273.7923584])
   >>> viirs = RadTbConverter('Suomi-NPP', 'viirs', 'M12')
   >>> rad37 = viirs.tb2radiance(tb37)
-  >>> print([np.round(rad, 7) for rad in rad37['radiance'].data])
+  >>> print([np.round(rad, 7) for rad in rad37['radiance']])
   [369717.4765726, 355110.5207853, 314684.2788726, 173143.5424898, 116408.0007877]
   >>> rad37['unit']
   'W/m^2 sr^-1 m^-1'
@@ -58,7 +58,7 @@ In order to get the total radiance over the band one has to multiply with the eq
   >>> tb37 = np.array([298.07385254, 297.15478516, 294.43276978, 281.67633057, 273.7923584])
   >>> viirs = RadTbConverter('Suomi-NPP', 'viirs', 'M12')
   >>> rad37 = viirs.tb2radiance(tb37, normalized=False)
-  >>> print([np.round(rad, 8) for rad in rad37['radiance'].data])
+  >>> print([np.round(rad, 8) for rad in rad37['radiance']])
   [0.07037968, 0.06759909, 0.05990352, 0.03295972, 0.02215951]
   >>> rad37['unit']
   'W/m^2 sr^-1'
@@ -127,7 +127,7 @@ where :math:`S(\lambda)` is the spectral solar irradiance.
   >>> viirs = RelativeSpectralResponse('Suomi-NPP', 'viirs')
   >>> solar_irr = SolarIrradianceSpectrum(TOTAL_IRRADIANCE_SPECTRUM_2000ASTM, dlambda=0.005)
   >>> sflux = solar_irr.inband_solarflux(viirs.rsr['M12'])
-  >>> print(round(sflux, 7))
+  >>> print(np.round(sflux, 7))
   2.2428119
 
 Derive the reflective part of the observed 3.7 micron radiance
@@ -197,10 +197,10 @@ In Python this becomes:
   >>> tb37 = np.array([298.07385254, 297.15478516, 294.43276978, 281.67633057, 273.7923584])
   >>> tb11 = np.array([271.38806152, 271.38806152, 271.33453369, 271.98553467, 271.93609619])
   >>> m12r = refl_m12.reflectance_from_tbs(sunz, tb37, tb11)
-  >>> print(m12r.mask)
+  >>> print(np.any(np.isnan(m12r)))
   False
-  >>> print([round(refl, 8) for refl in m12r.data])
-  [0.21432927, 0.20285153, 0.17063976, 0.05408903, 0.00838111]
+  >>> print([np.round(refl, 7) for refl in m12r])
+  [0.2143293, 0.2028515, 0.1706398, 0.054089, 0.0083811]
   
 We can try decompose equation :eq:`refl37` above using the example of VIIRS M12 band:
 
@@ -215,19 +215,19 @@ We can try decompose equation :eq:`refl37` above using the example of VIIRS M12
   >>> rad11 = viirs.tb2radiance(tb11, normalized=False)
   >>> sflux = 2.242817881698326
   >>> nomin = rad37['radiance'] - rad11['radiance']
-  >>> print(nomin.mask)
+  >>> print(np.isnan(nomin))
   [False False False False False]
-  >>> print([np.round(val, 8) for val in nomin.data])
+  >>> print([np.round(val, 8) for val in nomin])
   [0.05083677, 0.04805618, 0.0404157, 0.01279279, 0.00204485]
   >>> denom = np.cos(np.deg2rad(sunz))/np.pi * sflux - rad11['radiance']
-  >>> print(denom.mask)
+  >>> print(np.isnan(denom))
   [False False False False False]
-  >>> print([np.round(val, 8) for val in denom.data])
+  >>> print([np.round(val, 8) for val in denom])
   [0.23646313, 0.23645682, 0.23650559, 0.23582015, 0.2358661]
   >>> res = nomin/denom
-  >>> print(res.mask)
+  >>> print(np.isnan(res))
   [False False False False False]
-  >>> print([round(val, 8) for val in res.data])
+  >>> print([np.round(val, 8) for val in res])
   [0.21498817, 0.2032345, 0.17088689, 0.05424807, 0.00866955]
 
 
@@ -252,8 +252,8 @@ Using the example of the VIIRS M12 band from above this gives the following spec
   >>> tb11 = np.array([271.38806152, 271.38806152, 271.33453369, 271.98553467, 271.93609619])
   >>> m12r = refl_m12.reflectance_from_tbs(sunz, tb37, tb11)
   >>> tb = refl_m12.emissive_part_3x()
-  >>> ['{tb:6.3f}'.format(tb=round(t, 4)) for t in tb.data]
+  >>> ['{tb:6.3f}'.format(tb=np.round(t, 4)) for t in tb]
   ['266.996', '267.262', '267.991', '271.033', '271.927']
   >>> rad = refl_m12.emissive_part_3x(tb=False)
-  >>> ['{rad:6.3f}'.format(rad=round(r, 4)) for r in rad.data]
-  ['80285.149', '81458.022', '84749.639', '99761.400', '104582.030']
+  >>> ['{rad:6.3f}'.format(rad=np.round(r, 4)) for r in rad]
+  ['80285.149', '81458.022', '84749.639', '99761.401', '104582.030']


=====================================
doc/platforms_supported.rst
=====================================
@@ -76,7 +76,12 @@ have been included in PySpectral.
     * - FY-3D mersi-2
       - `rsr_mersi-2_FY-3D.h5`
       - CMA_ (Acquired via personal contact)
-
+    * - HY-1C cocts
+      - `rsr_cocts_HY-1C.h5`
+      - (Acquired via personal contact)
+    * - Metop-SG-A1 MetImage
+      - `rsr_metimage_Metop-SG-A1.h5`
+      - NWPSAF-MetImage_
 
 .. _Eumetsat: https://www.eumetsat.int/website/home/Data/Products/Calibration/MSGCalibration/index.html
 .. _GSICS: https://www.star.nesdis.noaa.gov/smcd/GCC/instrInfo-srf.php
@@ -89,4 +94,5 @@ have been included in PySpectral.
 .. _NASA-Landsat-OLI: https://landsat.gsfc.nasa.gov/wp-content/uploads/2013/06/Ball_BA_RSR.v1.1-1.xlsx
 .. _NESDIS: https://ncc.nesdis.noaa.gov/J1VIIRS/J1VIIRSSpectralResponseFunctions.php
 .. _CMA: http://www.cma.gov.cn/en2014/
+.. _NWPSAF-MetImage: https://nwpsaf.eu/downloads/rtcoef_rttov12/ir_srf/rtcoef_metopsg_1_metimage_srf.html
 


=====================================
doc/usage.rst
=====================================
@@ -22,9 +22,11 @@ Now, you can work with the data as you wish, make some simple plot for instance:
   >>> import matplotlib.pyplot as plt
   >>> dummy = plt.figure(figsize=(10, 5))
   >>> import numpy as np
-  >>> resp = np.ma.masked_less_equal(olci.rsr['Oa01']['det-1']['response'], 0.015)
-  >>> wvl = np.ma.masked_array(olci.rsr['Oa01']['det-1']['wavelength'], resp.mask)
-  >>> dummy = plt.plot(wvl.compressed(), resp.compressed())
+  >>> rsr = olci.rsr['Oa01']['det-1']['response']
+  >>> resp = np.where(rsr < 0.015, np.nan, rsr)
+  >>> wl_ = olci.rsr['Oa01']['det-1']['wavelength']
+  >>> wvl = np.where(np.isnan(resp), np.nan, wl_)
+  >>> dummy = plt.plot(wvl, resp)
   >>> plt.show() # doctest: +SKIP
 
   .. image:: _static/olci_ch1.png


=====================================
pyspectral/blackbody.py
=====================================
@@ -182,6 +182,7 @@ def planck(wave, temp, wavelength=True):
 
     denom = np.exp(exp_arg) - 1
     rad = nom / denom
+    rad = np.where(rad.mask, np.nan, rad.data)
     radshape = rad.shape
     if wln.shape[0] == 1:
         if temperature.shape[0] == 1:


=====================================
pyspectral/etc/pyspectral.yaml
=====================================
@@ -4,7 +4,12 @@
 #rsr_dir: ~/.local/share/pyspectral/
 #rayleigh_dir: ~/.local/share/pyspectral/
 
+# Here you may specify where (which path) to cache the radiance-tb lut for the
+# 3.9 (or 3.7 or 3.8) channel. If nothing is specified (default) the lut file
+# will be found in the directory path determined by tempfile.gettempdir()
+# (usually /tmp on a Linux system):
 #tb2rad_dir: /path/to/radiance/tb/lut/data
+#
 
 # On default relative spectral responses and short wave atmospheric correction
 # LUTs are downloaded from internet:
@@ -17,331 +22,331 @@ download_from_internet: True
 # can specify their file names and directory paths here.
 # Adam Dybbroe, 2018-03-20
 
-path1: '/home/a000680/data/SpectralResponses/jpss1-viirs/J1_VIIRS_Detector_RSR_V2'
-path2: '/home/a000680/data/SpectralResponses/jpss1-viirs/J1_VIIRS_V1_RSR_used_in_V2'
+# path1: '/home/a000680/data/SpectralResponses/jpss1-viirs/J1_VIIRS_Detector_RSR_V2'
+# path2: '/home/a000680/data/SpectralResponses/jpss1-viirs/J1_VIIRS_V1_RSR_used_in_V2'
 
-seviri:
-  path: /path/to/original/seviri/rsr/data/MSG_SEVIRI_Spectral_Response_Characterisation.XLS
-  filename: MSG_SEVIRI_Spectral_Response_Characterisation.XLS
+# seviri:
+#   path: /path/to/original/seviri/rsr/data/MSG_SEVIRI_Spectral_Response_Characterisation.XLS
+#   filename: MSG_SEVIRI_Spectral_Response_Characterisation.XLS
 
-Meteosat-8-seviri:
-  # You can provide any file name as you wish but it has to end with ".npz":
-  tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_meteosat9_seviri_ir3.9.npz
+# Meteosat-8-seviri:
+#   # You can provide any file name as you wish but it has to end with ".npz":
+#   tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_meteosat9_seviri_ir3.9.npz
 
-Meteosat-9-seviri:
-  tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_meteosat10_seviri_ir3.9.npz
+# Meteosat-9-seviri:
+#   tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_meteosat10_seviri_ir3.9.npz
 
-Meteosat-10-seviri:
-  tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_meteosat10_seviri_ir3.9.npz
+# Meteosat-10-seviri:
+#   tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_meteosat10_seviri_ir3.9.npz
 
-Meteosat-11-seviri:
-  tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_meteosat10_seviri_ir3.9.npz
+# Meteosat-11-seviri:
+#   tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_meteosat10_seviri_ir3.9.npz
 
-EOS-Terra-modis:
-  path: /home/a000680/data/SpectralResponses/modis/terra/L1B_RSR_LUT
-  tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_terra_modis_ir3.7.npz
+# EOS-Terra-modis:
+#   path: /home/a000680/data/SpectralResponses/modis/terra/L1B_RSR_LUT
+#   tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_terra_modis_ir3.7.npz
 
-EOS-Aqua-modis:
-  path: /home/a000680/data/SpectralResponses/modis/aqua
-  tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_aqua_modis_ir3.7.npz
+# EOS-Aqua-modis:
+#   path: /home/a000680/data/SpectralResponses/modis/aqua
+#   tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_aqua_modis_ir3.7.npz
 
 
-NOAA-20-viirs:
-  # J1_VIIRS_RSR_M1_Detector_Fused_V2.txt
-  # J1_VIIRS_V1_RSR_used_in_V2/J1_VIIRS_RSR_M10_Det_V1.txt
-  # J1_VIIRS_V1_RSR_used_in_V2/J1_VIIRS_RSR_M8_Det_V1.txt
+# NOAA-20-viirs:
+#   # J1_VIIRS_RSR_M1_Detector_Fused_V2.txt
+#   # J1_VIIRS_V1_RSR_used_in_V2/J1_VIIRS_RSR_M10_Det_V1.txt
+#   # J1_VIIRS_V1_RSR_used_in_V2/J1_VIIRS_RSR_M8_Det_V1.txt
 
-  rootdir: /path/to/original/jpss1/viirs/data
-  tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_noaa20_viirs_ir3.7.npz
+#   rootdir: /path/to/original/jpss1/viirs/data
+#   tb2rad_lut_filename: /path/to/radiance/tb/lut/data/tb2rad_lut_noaa20_viirs_ir3.7.npz
   
-  section1:
-    filename: J1_VIIRS_Detector_RSR_V2/J1_VIIRS_RSR_{bandname}_Detector_Fused_V2.txt
-    bands: [M1, M2, M3, M4, M5, M6, M7]
+#   section1:
+#     filename: J1_VIIRS_Detector_RSR_V2/J1_VIIRS_RSR_{bandname}_Detector_Fused_V2.txt
+#     bands: [M1, M2, M3, M4, M5, M6, M7]
 
-  section2:
-    filename: J1_VIIRS_Detector_RSR_V2/J1_VIIRS_RSR_{bandname}_Detector_Fused_V2.txt
-    bands: [I1, I2]
+#   section2:
+#     filename: J1_VIIRS_Detector_RSR_V2/J1_VIIRS_RSR_{bandname}_Detector_Fused_V2.txt
+#     bands: [I1, I2]
 
-  section3:
-    filename: J1_VIIRS_V1_RSR_used_in_V2/J1_VIIRS_RSR_M8_Det_V1.txt
-    bands: [M8]
+#   section3:
+#     filename: J1_VIIRS_V1_RSR_used_in_V2/J1_VIIRS_RSR_M8_Det_V1.txt
+#     bands: [M8]
     
-  section4:
-    filename: J1_VIIRS_Detector_RSR_V2.1/J1_VIIRS_RSR_M9_Det_V2.1.txt
-    bands: [M9]
+#   section4:
+#     filename: J1_VIIRS_Detector_RSR_V2.1/J1_VIIRS_RSR_M9_Det_V2.1.txt
+#     bands: [M9]
   
-  section5:
-    filename: J1_VIIRS_V1_RSR_used_in_V2/J1_VIIRS_RSR_{bandname}_Det_V1.txt
-    bands: [M10, M11, M12, M14, M15]
+#   section5:
+#     filename: J1_VIIRS_V1_RSR_used_in_V2/J1_VIIRS_RSR_{bandname}_Det_V1.txt
+#     bands: [M10, M11, M12, M14, M15]
 
-  section6:
-    filename: J1_VIIRS_Detector_RSR_V2/J1_VIIRS_RSR_M13_Det_V2.txt
-    bands: [M13]
+#   section6:
+#     filename: J1_VIIRS_Detector_RSR_V2/J1_VIIRS_RSR_M13_Det_V2.txt
+#     bands: [M13]
 
-  section7:
-    filename: J1_VIIRS_V1_RSR_used_in_V2/J1_VIIRS_RSR_M16A_Det_V1.txt
-    bands: [M16]
+#   section7:
+#     filename: J1_VIIRS_V1_RSR_used_in_V2/J1_VIIRS_RSR_M16A_Det_V1.txt
+#     bands: [M16]
 
-  section8:
-    filename: J1_VIIRS_V1_RSR_used_in_V2/J1_VIIRS_RSR_{bandname}_Det_V1.txt
-    bands: [I3, I4, I5]
+#   section8:
+#     filename: J1_VIIRS_V1_RSR_used_in_V2/J1_VIIRS_RSR_{bandname}_Det_V1.txt
+#     bands: [I3, I4, I5]
 
-  section9:
-    filename: J1_VIIRS_Detector_RSR_V2/J1_VIIRS_RSR_DNBLGS_Detector_Fused_V2S.txt
-    bands: [DNB]
+#   section9:
+#     filename: J1_VIIRS_Detector_RSR_V2/J1_VIIRS_RSR_DNBLGS_Detector_Fused_V2S.txt
+#     bands: [DNB]
 
 
-Suomi-NPP-viirs:
+# Suomi-NPP-viirs:
 
-  rootdir: /path/to/original/snpp/viirs/data
-  tb2rad_lut_filename:
-    m12: /path/to/radiance/tb/lut/data/tb2rad_lut_snpp_viirs_m12.npz
-    i4: /path/to/radiance/tb/lut/data/tb2rad_lut_snpp_viirs_i4.npz
+#   rootdir: /path/to/original/snpp/viirs/data
+#   tb2rad_lut_filename:
+#     m12: /path/to/radiance/tb/lut/data/tb2rad_lut_snpp_viirs_m12.npz
+#     i4: /path/to/radiance/tb/lut/data/tb2rad_lut_snpp_viirs_i4.npz
   
-  section1:
-    filename: GT_F1_SC_RSR_Release1.0_Best_Mbands/GT_F1_SC_RSR_Release1.0_Best_{bandname}_V2.71_ib_oob.txt
-    bands: [M1, M2, M3, M4, M5, M6, M7]
-
-  section2:
-    filename: GT_F1_CFPA_RSR_Release1.0_Best/GT_F1_CFPA_RSR_Release1.0_Best_{bandname}_v3_ib_oob.txt
-    bands: [M8, M9, M10, M11, M12, M13, M14, M15]
-
-  section3:
-    filename: GT_F1_CFPA_RSR_Release1.0_Best/GT_F1_CFPA_RSR_Release1.0_Best_M16A_v3_ib_oob.txt
-    bands: [M16]
-
-  section4:
-    filename: GT_F1_CFPA_RSR_Release1.0_Best/GT_F1_CFPA_RSR_Release1.0_Best_{bandname}_S0_v3_ib_oob.txt
-    bands: [I3, I4, I5]
-
-  section5:
-    filename: GT_F1_SC_RSR_Release1.0_Best_Ibands/GT_F1_SC_RSR_Release1.0_Best_{bandname}_V2.72_ib_oob.txt
-    bands: [I1, I2]
-
-  section6:
-    filename: tmp/GT_F1_DNB_RSR_Release1-1_Best/GT_F1_DNB_RSR_Release1.1_Best_DNBLGS_v3r1_ib_oob.txt
-    bands: [DNB]
-
-
-Envisat-aatsr:
-  path: /path/to/original/envisat/aatsr/data
-
-Sentinel-3A-slstr:
-  path: /path/to/original/sentinel-3a/slstr/data
-  ch1: SLSTR_FM02_S1_20150122.nc
-  ch2: SLSTR_FM02_S2_20150122.nc
-  ch3: SLSTR_FM02_S3_20150122.nc
-  ch4: SLSTR_FM02_S4_20150122.nc
-  ch5: SLSTR_FM02_S5_20150122.nc
-  ch6: SLSTR_FM02_S6_20150122.nc
-  ch7: SLSTR_FM02_S7_20150122.nc
-  ch8: SLSTR_FM02_S8_20150122.nc
-  ch9: SLSTR_FM02_S9_20150122.nc
-
-Sentinel-3A-olci:
-  path: /path/to/original/sentinel-3a/olci/data
-
-
-Sentinel-2A-msi:
-  path: /path/to/original/sentinel-2a/msi/data/S2-SRF_COPE-GSEG-EOPG-TN-15-0007_3.0.xlsx
-Sentinel-2B-msi:
-  path: /path/to/original/sentinel-2b/msi/data/S2-SRF_COPE-GSEG-EOPG-TN-15-0007_3.0.xlsx
-
-Himawari-8-ahi:
-  path: /path/to/original/ahi/data
-  ch1: AHI_Sep2013_CH01.SRF
-  ch2: AHI_Sep2013_CH02.SRF
-  ch3: AHI_Sep2013_CH03.SRF
-  ch4: AHI_Sep2013_CH04.SRF
-  ch5: AHI_Sep2013_CH05.SRF
-  ch6: AHI_Sep2013_CH06.SRF
-  ch7: AHI_Sep2013_CH07.SRF
-  ch8: AHI_Sep2013_CH08.SRF
-  ch9: AHI_Sep2013_CH09.SRF
-  ch10: AHI_Sep2013_CH10.SRF
-  ch11: AHI_Sep2013_CH11.SRF
-  ch12: AHI_Sep2013_CH12.SRF
-  ch13: AHI_Sep2013_CH13.SRF
-  ch14: AHI_Sep2013_CH14.SRF
-  ch15: AHI_Sep2013_CH15.SRF
-  ch16: AHI_Sep2013_CH16.SRF
-
-GOES-16-abi:
-  path: /path/to/original/goes16/abi/data
-  ch1: GOES-R_ABI_PFM_SRF_CWG_ch1.txt
-  ch2: GOES-R_ABI_PFM_SRF_CWG_ch2.txt
-  ch3: GOES-R_ABI_PFM_SRF_CWG_ch3.txt
-  ch4: GOES-R_ABI_PFM_SRF_CWG_ch4.txt
-  ch5: GOES-R_ABI_PFM_SRF_CWG_ch5.txt
-  ch6: GOES-R_ABI_PFM_SRF_CWG_ch6.txt
-  ch7: GOES-R_ABI_PFM_SRF_CWG_ch7.txt
-  ch8: GOES-R_ABI_PFM_SRF_CWG_ch8.txt
-  ch9: GOES-R_ABI_PFM_SRF_CWG_ch9.txt
-  ch10: GOES-R_ABI_PFM_SRF_CWG_ch10.txt
-  ch11: GOES-R_ABI_PFM_SRF_CWG_ch11.txt
-  ch12: GOES-R_ABI_PFM_SRF_CWG_ch12.txt
-  ch13: GOES-R_ABI_PFM_SRF_CWG_ch13.txt
-  ch14: GOES-R_ABI_PFM_SRF_CWG_ch14.txt
-  ch15: GOES-R_ABI_PFM_SRF_CWG_ch15.txt
-  ch16: GOES-R_ABI_PFM_SRF_CWG_ch16.txt
-
-GOES-17-abi:
-  path: /path/to/original/goes17/abi/data
-  ch1: GOES-R_ABI_FM2_SRF_CWG_ch1.txt
-  ch2: GOES-R_ABI_FM2_SRF_CWG_ch2.txt
-  ch3: GOES-R_ABI_FM2_SRF_CWG_ch3.txt
-  ch4: GOES-R_ABI_FM2_SRF_CWG_ch4.txt
-  ch5: GOES-R_ABI_FM2_SRF_CWG_ch5.txt
-  ch6: GOES-R_ABI_FM2_SRF_CWG_ch6.txt
-  ch7: GOES-R_ABI_FM2_SRF_CWG_ch7.txt
-  ch8: GOES-R_ABI_FM2_SRF_CWG_ch8.txt
-  ch9: GOES-R_ABI_FM2_SRF_CWG_ch9.txt
-  ch10: GOES-R_ABI_FM2_SRF_CWG_ch10.txt
-  ch11: GOES-R_ABI_FM2_SRF_CWG_ch11.txt
-  ch12: GOES-R_ABI_FM2_SRF_CWG_ch12.txt
-  ch13: GOES-R_ABI_FM2_SRF_CWG_ch13.txt
-  ch14: GOES-R_ABI_FM2_SRF_CWG_ch14.txt
-  ch15: GOES-R_ABI_FM2_SRF_CWG_ch15.txt
-  ch16: GOES-R_ABI_FM2_SRF_CWG_ch16.txt
-
-FY-3D-mersi-2:
-  path: /path/to/original/fy3d/mersi2/data
-  ch1:  FY3D_MERSI_SRF_CH01_Pub.txt
-  ch2:  FY3D_MERSI_SRF_CH02_Pub.txt
-  ch3:  FY3D_MERSI_SRF_CH03_Pub.txt
-  ch4:  FY3D_MERSI_SRF_CH04_Pub.txt
-  ch5:  FY3D_MERSI_SRF_CH05_Pub.txt
-  ch6:  FY3D_MERSI_SRF_CH06_Pub.txt
-  ch7:  FY3D_MERSI_SRF_CH07_Pub.txt
-  ch8:  FY3D_MERSI_SRF_CH08_Pub.txt
-  ch9:  FY3D_MERSI_SRF_CH09_Pub.txt
-  ch10: FY3D_MERSI_SRF_CH10_Pub.txt
-  ch11: FY3D_MERSI_SRF_CH11_Pub.txt
-  ch12: FY3D_MERSI_SRF_CH12_Pub.txt
-  ch13: FY3D_MERSI_SRF_CH13_Pub.txt
-  ch14: FY3D_MERSI_SRF_CH14_Pub.txt
-  ch15: FY3D_MERSI_SRF_CH15_Pub.txt
-  ch16: FY3D_MERSI_SRF_CH16_Pub.txt
-  ch17: FY3D_MERSI_SRF_CH17_Pub.txt
-  ch18: FY3D_MERSI_SRF_CH18_Pub.txt
-  ch19: FY3D_MERSI_SRF_CH19_Pub.txt
-  ch20: FY3D_MERSI_SRF_CH20_Pub.txt
-  ch21: FY3D_MERSI_SRF_CH21_Pub.txt
-  ch22: FY3D_MERSI_SRF_CH22_Pub.txt
-  ch23: FY3D_MERSI_SRF_CH23_Pub.txt
-  ch24: FY3D_MERSI_SRF_CH24_Pub.txt
-  ch25: FY3D_MERSI_SRF_CH25_Pub.txt
-
-
-NOAA-19-avhrr/3:
-  path: /path/to/original/noaa19/avhrr/data
-  ch1: NOAA_19_A308C001.txt
-  ch2: NOAA_19_A308C002.txt
-  ch3a: NOAA_19_A308C03A.txt
-  ch3b: NOAA_19_A308C03B.txt
-  ch4: NOAA_19_A308C004.txt
-  ch5: NOAA_19_A308C005.txt
-
-NOAA-18-avhrr/3:
-  path: /path/to/original/noaa18/avhrr/data
-  ch1: NOAA_18_A306C001.txt
-  ch2: NOAA_18_A306C002.txt
-  ch3a: NOAA_18_A306C03A.txt
-  ch3b: NOAA_18_A306C03B.txt
-  ch4: NOAA_18_A306C004.txt
-  ch5: NOAA_18_A306C005.txt
-
-NOAA-17-avhrr/3:
-  path: /path/to/original/noaa17/avhrr/data
-  ch1: NOAA_17_A304C001.txt
-  ch2: NOAA_17_A304C002.txt
-  ch3a: NOAA_17_A304C03A.txt
-  ch3b: NOAA_17_A304C03B.txt
-  ch4: NOAA_17_A304C004.txt
-  ch5: NOAA_17_A304C005.txt
-
-NOAA-16-avhrr/3:
-  path: /path/to/original/noaa16/avhrr/data
-  ch1: NOAA_16_A301C001.txt
-  ch2: NOAA_16_A301C002.txt
-  ch3a: NOAA_16_A301C03A.txt
-  ch3b: NOAA_16_A301C03B.txt
-  ch4: NOAA_16_A301C004.txt
-  ch5: NOAA_16_A301C005.txt
-
-NOAA-15-avhrr/3:
-  path: /path/to/original/noaa15/avhrr/data
-  ch1: NOAA_15_A302xx01.txt
-  ch2: NOAA_15_A302xx02.txt
-  ch3a: NOAA_15_A302xx3A.txt
-  ch3b: NOAA_15_A302xx3B.txt
-  ch4: NOAA_15_A302xx04.txt
-  ch5: NOAA_15_A302xx05.txt
-
-NOAA-14-avhrr/2:
-  path: /path/to/original/noaa14/avhrr/data
-  ch1: NOAA_14_A204C001.txt
-  ch2: NOAA_14_A204C002.txt
-  ch3: NOAA_14_A204C003.txt
-  ch4: NOAA_14_A204C004.txt
-  ch5: NOAA_14_A204C005.txt
-
-NOAA-12-avhrr/2:
-  path: /path/to/original/noaa12/avhrr/data
-  ch1: NOAA_12_A205C001.txt
-  ch2: NOAA_12_A205C002.txt
-  ch3: NOAA_12_A205C003.txt
-  ch4: NOAA_12_A205C004.txt
-  ch5: NOAA_12_A205C005.txt
-
-NOAA-11-avhrr/2:
-  path: /path/to/original/noaa11/avhrr/data
-  ch1: NOAA_11_A203C001.txt
-  ch2: NOAA_11_A203C002.txt
-  ch3: NOAA_11_A203C003.txt
-  ch4: NOAA_11_A203C004.txt
-  ch5: NOAA_11_A203C005.txt
-
-NOAA-9-avhrr/2:
-  path: /path/to/original/noaa9/avhrr/data
-  ch1: NOAA_9_A202C001.txt
-  ch2: NOAA_9_A202C002.txt
-  ch3: NOAA_9_A202C003.txt
-  ch4: NOAA_9_A202C004.txt
-  ch5: NOAA_9_A202C005.txt
-
-NOAA-7-avhrr/2:
-  path: /path/to/original/noaa7/avhrr/data
-  ch1: NOAA_7_A201C001.txt
-  ch2: NOAA_7_A201C002.txt
-  ch3: NOAA_7_A201C003.txt
-  ch4: NOAA_7_A201C004.txt
-  ch5: NOAA_7_A201C005.txt
-
-NOAA-10-avhrr/1:
-  path: /path/to/original/noaa10/avhrr/data
-  ch1: NOAA_10_A101C001.txt
-  ch2: NOAA_10_A101C002.txt
-  ch3: NOAA_10_A101C003.txt
-  ch4: NOAA_10_A101C004.txt
-
-NOAA-8-avhrr/1:
-  path: /path/to/original/noaa8/avhrr/data
-  ch1: NOAA_8_A102C001.txt
-  ch2: NOAA_8_A102C002.txt
-  ch3: NOAA_8_A102C003.txt
-  ch4: NOAA_8_A102C004.txt
-
-NOAA-6-avhrr/1:
-  path: /path/to/original/noaa6/avhrr/data
-  ch1: NOAA_6_A103C001.txt
-  ch2: NOAA_6_A103C002.txt
-  ch3: NOAA_6_A103C003.txt
-  ch4: NOAA_6_A103C004.txt
-
-TIROS-N-avhrr/1:
-  path: /path/to/original/tirosn/avhrr/data
-  ch1: TIROS_N_PFMC001.txt
-  ch2: TIROS_N_PFMC002.txt
-  ch3: TIROS_N_PFMC003.txt
-  ch4: TIROS_N_APFMC004.txt
+#   section1:
+#     filename: GT_F1_SC_RSR_Release1.0_Best_Mbands/GT_F1_SC_RSR_Release1.0_Best_{bandname}_V2.71_ib_oob.txt
+#     bands: [M1, M2, M3, M4, M5, M6, M7]
+
+#   section2:
+#     filename: GT_F1_CFPA_RSR_Release1.0_Best/GT_F1_CFPA_RSR_Release1.0_Best_{bandname}_v3_ib_oob.txt
+#     bands: [M8, M9, M10, M11, M12, M13, M14, M15]
+
+#   section3:
+#     filename: GT_F1_CFPA_RSR_Release1.0_Best/GT_F1_CFPA_RSR_Release1.0_Best_M16A_v3_ib_oob.txt
+#     bands: [M16]
+
+#   section4:
+#     filename: GT_F1_CFPA_RSR_Release1.0_Best/GT_F1_CFPA_RSR_Release1.0_Best_{bandname}_S0_v3_ib_oob.txt
+#     bands: [I3, I4, I5]
+
+#   section5:
+#     filename: GT_F1_SC_RSR_Release1.0_Best_Ibands/GT_F1_SC_RSR_Release1.0_Best_{bandname}_V2.72_ib_oob.txt
+#     bands: [I1, I2]
+
+#   section6:
+#     filename: tmp/GT_F1_DNB_RSR_Release1-1_Best/GT_F1_DNB_RSR_Release1.1_Best_DNBLGS_v3r1_ib_oob.txt
+#     bands: [DNB]
+
+
+# Envisat-aatsr:
+#   path: /path/to/original/envisat/aatsr/data
+
+# Sentinel-3A-slstr:
+#   path: /path/to/original/sentinel-3a/slstr/data
+#   ch1: SLSTR_FM02_S1_20150122.nc
+#   ch2: SLSTR_FM02_S2_20150122.nc
+#   ch3: SLSTR_FM02_S3_20150122.nc
+#   ch4: SLSTR_FM02_S4_20150122.nc
+#   ch5: SLSTR_FM02_S5_20150122.nc
+#   ch6: SLSTR_FM02_S6_20150122.nc
+#   ch7: SLSTR_FM02_S7_20150122.nc
+#   ch8: SLSTR_FM02_S8_20150122.nc
+#   ch9: SLSTR_FM02_S9_20150122.nc
+
+# Sentinel-3A-olci:
+#   path: /path/to/original/sentinel-3a/olci/data
+
+
+# Sentinel-2A-msi:
+#   path: /path/to/original/sentinel-2a/msi/data/S2-SRF_COPE-GSEG-EOPG-TN-15-0007_3.0.xlsx
+# Sentinel-2B-msi:
+#   path: /path/to/original/sentinel-2b/msi/data/S2-SRF_COPE-GSEG-EOPG-TN-15-0007_3.0.xlsx
+
+# Himawari-8-ahi:
+#   path: /path/to/original/ahi/data
+#   ch1: AHI_Sep2013_CH01.SRF
+#   ch2: AHI_Sep2013_CH02.SRF
+#   ch3: AHI_Sep2013_CH03.SRF
+#   ch4: AHI_Sep2013_CH04.SRF
+#   ch5: AHI_Sep2013_CH05.SRF
+#   ch6: AHI_Sep2013_CH06.SRF
+#   ch7: AHI_Sep2013_CH07.SRF
+#   ch8: AHI_Sep2013_CH08.SRF
+#   ch9: AHI_Sep2013_CH09.SRF
+#   ch10: AHI_Sep2013_CH10.SRF
+#   ch11: AHI_Sep2013_CH11.SRF
+#   ch12: AHI_Sep2013_CH12.SRF
+#   ch13: AHI_Sep2013_CH13.SRF
+#   ch14: AHI_Sep2013_CH14.SRF
+#   ch15: AHI_Sep2013_CH15.SRF
+#   ch16: AHI_Sep2013_CH16.SRF
+
+# GOES-16-abi:
+#   path: /path/to/original/goes16/abi/data
+#   ch1: GOES-R_ABI_PFM_SRF_CWG_ch1.txt
+#   ch2: GOES-R_ABI_PFM_SRF_CWG_ch2.txt
+#   ch3: GOES-R_ABI_PFM_SRF_CWG_ch3.txt
+#   ch4: GOES-R_ABI_PFM_SRF_CWG_ch4.txt
+#   ch5: GOES-R_ABI_PFM_SRF_CWG_ch5.txt
+#   ch6: GOES-R_ABI_PFM_SRF_CWG_ch6.txt
+#   ch7: GOES-R_ABI_PFM_SRF_CWG_ch7.txt
+#   ch8: GOES-R_ABI_PFM_SRF_CWG_ch8.txt
+#   ch9: GOES-R_ABI_PFM_SRF_CWG_ch9.txt
+#   ch10: GOES-R_ABI_PFM_SRF_CWG_ch10.txt
+#   ch11: GOES-R_ABI_PFM_SRF_CWG_ch11.txt
+#   ch12: GOES-R_ABI_PFM_SRF_CWG_ch12.txt
+#   ch13: GOES-R_ABI_PFM_SRF_CWG_ch13.txt
+#   ch14: GOES-R_ABI_PFM_SRF_CWG_ch14.txt
+#   ch15: GOES-R_ABI_PFM_SRF_CWG_ch15.txt
+#   ch16: GOES-R_ABI_PFM_SRF_CWG_ch16.txt
+
+# GOES-17-abi:
+#   path: /path/to/original/goes17/abi/data
+#   ch1: GOES-R_ABI_FM2_SRF_CWG_ch1.txt
+#   ch2: GOES-R_ABI_FM2_SRF_CWG_ch2.txt
+#   ch3: GOES-R_ABI_FM2_SRF_CWG_ch3.txt
+#   ch4: GOES-R_ABI_FM2_SRF_CWG_ch4.txt
+#   ch5: GOES-R_ABI_FM2_SRF_CWG_ch5.txt
+#   ch6: GOES-R_ABI_FM2_SRF_CWG_ch6.txt
+#   ch7: GOES-R_ABI_FM2_SRF_CWG_ch7.txt
+#   ch8: GOES-R_ABI_FM2_SRF_CWG_ch8.txt
+#   ch9: GOES-R_ABI_FM2_SRF_CWG_ch9.txt
+#   ch10: GOES-R_ABI_FM2_SRF_CWG_ch10.txt
+#   ch11: GOES-R_ABI_FM2_SRF_CWG_ch11.txt
+#   ch12: GOES-R_ABI_FM2_SRF_CWG_ch12.txt
+#   ch13: GOES-R_ABI_FM2_SRF_CWG_ch13.txt
+#   ch14: GOES-R_ABI_FM2_SRF_CWG_ch14.txt
+#   ch15: GOES-R_ABI_FM2_SRF_CWG_ch15.txt
+#   ch16: GOES-R_ABI_FM2_SRF_CWG_ch16.txt
+
+# FY-3D-mersi-2:
+#   path: /path/to/original/fy3d/mersi2/data
+#   ch1:  FY3D_MERSI_SRF_CH01_Pub.txt
+#   ch2:  FY3D_MERSI_SRF_CH02_Pub.txt
+#   ch3:  FY3D_MERSI_SRF_CH03_Pub.txt
+#   ch4:  FY3D_MERSI_SRF_CH04_Pub.txt
+#   ch5:  FY3D_MERSI_SRF_CH05_Pub.txt
+#   ch6:  FY3D_MERSI_SRF_CH06_Pub.txt
+#   ch7:  FY3D_MERSI_SRF_CH07_Pub.txt
+#   ch8:  FY3D_MERSI_SRF_CH08_Pub.txt
+#   ch9:  FY3D_MERSI_SRF_CH09_Pub.txt
+#   ch10: FY3D_MERSI_SRF_CH10_Pub.txt
+#   ch11: FY3D_MERSI_SRF_CH11_Pub.txt
+#   ch12: FY3D_MERSI_SRF_CH12_Pub.txt
+#   ch13: FY3D_MERSI_SRF_CH13_Pub.txt
+#   ch14: FY3D_MERSI_SRF_CH14_Pub.txt
+#   ch15: FY3D_MERSI_SRF_CH15_Pub.txt
+#   ch16: FY3D_MERSI_SRF_CH16_Pub.txt
+#   ch17: FY3D_MERSI_SRF_CH17_Pub.txt
+#   ch18: FY3D_MERSI_SRF_CH18_Pub.txt
+#   ch19: FY3D_MERSI_SRF_CH19_Pub.txt
+#   ch20: FY3D_MERSI_SRF_CH20_Pub.txt
+#   ch21: FY3D_MERSI_SRF_CH21_Pub.txt
+#   ch22: FY3D_MERSI_SRF_CH22_Pub.txt
+#   ch23: FY3D_MERSI_SRF_CH23_Pub.txt
+#   ch24: FY3D_MERSI_SRF_CH24_Pub.txt
+#   ch25: FY3D_MERSI_SRF_CH25_Pub.txt
+
+
+# NOAA-19-avhrr/3:
+#   path: /path/to/original/noaa19/avhrr/data
+#   ch1: NOAA_19_A308C001.txt
+#   ch2: NOAA_19_A308C002.txt
+#   ch3a: NOAA_19_A308C03A.txt
+#   ch3b: NOAA_19_A308C03B.txt
+#   ch4: NOAA_19_A308C004.txt
+#   ch5: NOAA_19_A308C005.txt
+
+# NOAA-18-avhrr/3:
+#   path: /path/to/original/noaa18/avhrr/data
+#   ch1: NOAA_18_A306C001.txt
+#   ch2: NOAA_18_A306C002.txt
+#   ch3a: NOAA_18_A306C03A.txt
+#   ch3b: NOAA_18_A306C03B.txt
+#   ch4: NOAA_18_A306C004.txt
+#   ch5: NOAA_18_A306C005.txt
+
+# NOAA-17-avhrr/3:
+#   path: /path/to/original/noaa17/avhrr/data
+#   ch1: NOAA_17_A304C001.txt
+#   ch2: NOAA_17_A304C002.txt
+#   ch3a: NOAA_17_A304C03A.txt
+#   ch3b: NOAA_17_A304C03B.txt
+#   ch4: NOAA_17_A304C004.txt
+#   ch5: NOAA_17_A304C005.txt
+
+# NOAA-16-avhrr/3:
+#   path: /path/to/original/noaa16/avhrr/data
+#   ch1: NOAA_16_A301C001.txt
+#   ch2: NOAA_16_A301C002.txt
+#   ch3a: NOAA_16_A301C03A.txt
+#   ch3b: NOAA_16_A301C03B.txt
+#   ch4: NOAA_16_A301C004.txt
+#   ch5: NOAA_16_A301C005.txt
+
+# NOAA-15-avhrr/3:
+#   path: /path/to/original/noaa15/avhrr/data
+#   ch1: NOAA_15_A302xx01.txt
+#   ch2: NOAA_15_A302xx02.txt
+#   ch3a: NOAA_15_A302xx3A.txt
+#   ch3b: NOAA_15_A302xx3B.txt
+#   ch4: NOAA_15_A302xx04.txt
+#   ch5: NOAA_15_A302xx05.txt
+
+# NOAA-14-avhrr/2:
+#   path: /path/to/original/noaa14/avhrr/data
+#   ch1: NOAA_14_A204C001.txt
+#   ch2: NOAA_14_A204C002.txt
+#   ch3: NOAA_14_A204C003.txt
+#   ch4: NOAA_14_A204C004.txt
+#   ch5: NOAA_14_A204C005.txt
+
+# NOAA-12-avhrr/2:
+#   path: /path/to/original/noaa12/avhrr/data
+#   ch1: NOAA_12_A205C001.txt
+#   ch2: NOAA_12_A205C002.txt
+#   ch3: NOAA_12_A205C003.txt
+#   ch4: NOAA_12_A205C004.txt
+#   ch5: NOAA_12_A205C005.txt
+
+# NOAA-11-avhrr/2:
+#   path: /path/to/original/noaa11/avhrr/data
+#   ch1: NOAA_11_A203C001.txt
+#   ch2: NOAA_11_A203C002.txt
+#   ch3: NOAA_11_A203C003.txt
+#   ch4: NOAA_11_A203C004.txt
+#   ch5: NOAA_11_A203C005.txt
+
+# NOAA-9-avhrr/2:
+#   path: /path/to/original/noaa9/avhrr/data
+#   ch1: NOAA_9_A202C001.txt
+#   ch2: NOAA_9_A202C002.txt
+#   ch3: NOAA_9_A202C003.txt
+#   ch4: NOAA_9_A202C004.txt
+#   ch5: NOAA_9_A202C005.txt
+
+# NOAA-7-avhrr/2:
+#   path: /path/to/original/noaa7/avhrr/data
+#   ch1: NOAA_7_A201C001.txt
+#   ch2: NOAA_7_A201C002.txt
+#   ch3: NOAA_7_A201C003.txt
+#   ch4: NOAA_7_A201C004.txt
+#   ch5: NOAA_7_A201C005.txt
+
+# NOAA-10-avhrr/1:
+#   path: /path/to/original/noaa10/avhrr/data
+#   ch1: NOAA_10_A101C001.txt
+#   ch2: NOAA_10_A101C002.txt
+#   ch3: NOAA_10_A101C003.txt
+#   ch4: NOAA_10_A101C004.txt
+
+# NOAA-8-avhrr/1:
+#   path: /path/to/original/noaa8/avhrr/data
+#   ch1: NOAA_8_A102C001.txt
+#   ch2: NOAA_8_A102C002.txt
+#   ch3: NOAA_8_A102C003.txt
+#   ch4: NOAA_8_A102C004.txt
+
+# NOAA-6-avhrr/1:
+#   path: /path/to/original/noaa6/avhrr/data
+#   ch1: NOAA_6_A103C001.txt
+#   ch2: NOAA_6_A103C002.txt
+#   ch3: NOAA_6_A103C003.txt
+#   ch4: NOAA_6_A103C004.txt
+
+# TIROS-N-avhrr/1:
+#   path: /path/to/original/tirosn/avhrr/data
+#   ch1: TIROS_N_PFMC001.txt
+#   ch2: TIROS_N_PFMC002.txt
+#   ch3: TIROS_N_PFMC003.txt
+#   ch4: TIROS_N_APFMC004.txt


=====================================
pyspectral/near_infrared_reflectance.py
=====================================
@@ -28,6 +28,11 @@ window channel (usually around 11-12 microns).
 
 import os
 import numpy as np
+try:
+    from dask.array import where, logical_or
+except ImportError:
+    from numpy import where, logical_or
+
 from pyspectral.solar import (SolarIrradianceSpectrum,
                               TOTAL_IRRADIANCE_SPECTRUM_2000ASTM)
 from pyspectral.utils import BANDNAMES, get_bandname_from_wavelength
@@ -195,15 +200,25 @@ class Calculator(RadTbConverter):
                      absorption correction will be applied.
 
         """
+        # Check for dask arrays
+        if hasattr(tb_near_ir, 'compute') or hasattr(tb_thermal, 'compute'):
+            compute = False
+        else:
+            compute = True
+        if hasattr(tb_near_ir, 'mask') or hasattr(tb_thermal, 'mask'):
+            is_masked = True
+        else:
+            is_masked = False
+
         if np.isscalar(tb_near_ir):
             tb_nir = np.array([tb_near_ir, ])
         else:
-            tb_nir = np.array(tb_near_ir)
+            tb_nir = np.asanyarray(tb_near_ir)
 
         if np.isscalar(tb_thermal):
             tb_therm = np.array([tb_thermal, ])
         else:
-            tb_therm = np.array(tb_thermal)
+            tb_therm = np.asanyarray(tb_thermal)
 
         if tb_therm.shape != tb_nir.shape:
             errmsg = 'Dimensions do not match! {0} and {1}'.format(
@@ -221,7 +236,7 @@ class Calculator(RadTbConverter):
             if np.isscalar(tb_ir_co2):
                 tbco2 = np.array([tb_ir_co2, ])
             else:
-                tbco2 = np.array(tb_ir_co2)
+                tbco2 = np.asanyarray(tb_ir_co2)
 
         if not self.rsr:
             raise NotImplementedError("Reflectance calculations without "
@@ -239,9 +254,8 @@ class Calculator(RadTbConverter):
         if l_nir.ravel().shape[0] < 10:
             LOG.info('l_nir = %s', str(l_nir))
 
-        sunz = np.ma.masked_outside(sun_zenith, 0.0, 88.0)
-        sunzmask = sunz.mask
-        sunz = sunz.filled(88.)
+        sunzmask = (sun_zenith < 0.0) | (sun_zenith > 88.0)
+        sunz = where(sunzmask, 88.0, sun_zenith)
 
         mu0 = np.cos(np.deg2rad(sunz))
         # mu0 = np.where(np.less(mu0, 0.1), 0.1, mu0)
@@ -262,12 +276,18 @@ class Calculator(RadTbConverter):
         mask = (self._solar_radiance - thermal_emiss_one *
                 self._rad3x_correction) < EPSILON
 
-        np.logical_or(sunzmask, mask, out=mask)
-        np.logical_or(mask, np.isnan(tb_nir), out=mask)
+        logical_or(sunzmask, mask, out=mask)
+        logical_or(mask, np.isnan(tb_nir), out=mask)
 
-        self._r3x = np.ma.masked_where(mask, data)
+        self._r3x = where(mask, np.nan, data)
 
         # Reflectances should be between 0 and 1, but values above 1 is
         # perfectly possible and okay! (Multiply by 100 to get reflectances
         # in percent)
-        return self._r3x
+        if hasattr(self._r3x, 'compute') and compute:
+            res = self._r3x.compute()
+        else:
+            res = self._r3x
+        if is_masked:
+            res = np.ma.masked_array(res, mask=np.isnan(res))
+        return res


=====================================
pyspectral/radiance_tb_conversion.py
=====================================
@@ -228,7 +228,7 @@ class RadTbConverter(object):
         tb_ = np.arange(TB_MIN, TB_MAX, self.tb_resolution)
         retv = self.tb2radiance(tb_, normalized=normalized)
         rad = retv['radiance']
-        np.savez(filepath, tb=tb_, radiance=rad.compressed())
+        np.savez(filepath, tb=tb_, radiance=rad)
 
     @staticmethod
     def read_tb2rad_lut(filepath):


=====================================
pyspectral/raw_reader.py
=====================================
@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
-# Copyright (c) 2016-2017 Adam.Dybbroe
+# Copyright (c) 2016-2019 Adam.Dybbroe
 
 # Author(s):
 


=====================================
pyspectral/rayleigh.py
=====================================
@@ -204,6 +204,12 @@ class Rayleigh(object):
         return self._rayl, self._wvl_coord, self._azid_coord,\
             self._satz_sec_coord, self._sunz_sec_coord
 
+    @staticmethod
+    def _do_interp(minterp, sunzsec, azidiff, satzsec):
+        interp_points2 = np.vstack((sunzsec.ravel(), 180 - azidiff.ravel(), satzsec.ravel()))
+        res = minterp(interp_points2)
+        return res.reshape(sunzsec.shape)
+
     def get_reflectance(self, sun_zenith, sat_zenith, azidiff, bandname, redband=None):
         """Get the reflectance from the three sun-sat angles"""
         # Get wavelength in nm for band:
@@ -277,19 +283,11 @@ class Rayleigh(object):
         minterp = MultilinearInterpolator(smin, smax, orders)
         minterp.set_values(f_3d_grid)
 
-        def _do_interp(minterp, sunzsec, azidiff, satzsec):
-            interp_points2 = np.vstack((sunzsec.ravel(),
-                                        180 - azidiff.ravel(),
-                                        satzsec.ravel()))
-            res = minterp(interp_points2)
-            return res.reshape(sunzsec.shape)
-
         if HAVE_DASK:
-            ipn = map_blocks(_do_interp, minterp, sunzsec, azidiff,
-                             satzsec, dtype=raylwvl.dtype,
-                             chunks=azidiff.chunks)
+            ipn = map_blocks(self._do_interp, minterp, sunzsec, azidiff,
+                             satzsec, dtype=raylwvl.dtype, chunks=azidiff.chunks)
         else:
-            ipn = _do_interp(minterp, sunzsec, azidiff, satzsec)
+            ipn = self._do_interp(minterp, sunzsec, azidiff, satzsec)
 
         LOG.debug("Time - Interpolation: {0:f}".format(time.time() - tic))
 


=====================================
pyspectral/tests/__init__.py
=====================================
@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
-# Copyright (c) 2013-2018 Adam.Dybbroe
+# Copyright (c) 2013-2019 Adam.Dybbroe
 
 # Author(s):
 


=====================================
pyspectral/tests/test_reflectance.py
=====================================
@@ -120,7 +120,7 @@ class TestReflectance(unittest.TestCase):
             refl37 = Calculator('EOS-Aqua', 'modis', '20')
 
         expected = 1.8563451e-07  # unit = 'm' (meter)
-        self.assertAlmostEqual(refl37.rsr_integral, expected)
+        np.testing.assert_allclose(refl37.rsr_integral, expected)
 
         with patch('pyspectral.radiance_tb_conversion.RelativeSpectralResponse') as mymock:
             instance = mymock.return_value
@@ -132,7 +132,7 @@ class TestReflectance(unittest.TestCase):
 
         expected = 13000.385  # SI units = 'm-1' (1/meter)
         res = refl37.rsr_integral
-        self.assertAlmostEqual(res / expected, 1.0, 6)
+        np.testing.assert_allclose(res / expected, 1.0, 6)
 
     def test_reflectance(self):
         """Test the derivation of the reflective part of a 3.7 micron band"""
@@ -162,28 +162,44 @@ class TestReflectance(unittest.TestCase):
         tb3 = np.array([290.])
         tb4 = np.array([282.])
         refl = refl37.reflectance_from_tbs(sunz, tb3, tb4)
-        self.assertAlmostEqual(refl.data[0], 0.251245010648, 6)
+        np.testing.assert_allclose(refl[0], 0.251245010648, 6)
 
         tb3x = refl37.emissive_part_3x()
-        self.assertAlmostEqual(tb3x, 276.213054, 6)
+        np.testing.assert_allclose(tb3x, 276.213054, 6)
 
         sunz = np.array([80.])
         tb3 = np.array([295.])
         tb4 = np.array([282.])
         refl = refl37.reflectance_from_tbs(sunz, tb3, tb4)
-        self.assertAlmostEqual(refl.data[0], 0.452497961, 6)
+        np.testing.assert_allclose(refl[0], 0.452497961, 6)
 
         tb3x = refl37.emissive_part_3x()
-        self.assertAlmostEqual(tb3x, 270.077268, 6)
+        np.testing.assert_allclose(tb3x, 270.077268, 6)
 
         sunz = np.array([50.])
         tb3 = np.array([300.])
         tb4 = np.array([285.])
         refl = refl37.reflectance_from_tbs(sunz, tb3, tb4)
-        self.assertAlmostEqual(refl.data[0], 0.1189217, 6)
+        np.testing.assert_allclose(refl[0], 0.1189217, 6)
 
         tb3x = refl37.emissive_part_3x()
-        self.assertAlmostEqual(tb3x, 282.455426, 6)
+        np.testing.assert_allclose(tb3x, 282.455426, 6)
+
+        sunz = np.array([50.])
+        tb3 = np.ma.masked_array([300.], mask=False)
+        tb4 = np.ma.masked_array([285.], mask=False)
+        refl = refl37.reflectance_from_tbs(sunz, tb3, tb4)
+        self.assertTrue(hasattr(refl, 'mask'))
+
+        try:
+            import dask.array as da
+            sunz = da.from_array([50.], chunks=10)
+            tb3 = da.from_array([300.], chunks=10)
+            tb4 = da.from_array([285.], chunks=10)
+            refl = refl37.reflectance_from_tbs(sunz, tb3, tb4)
+            self.assertTrue(hasattr(refl, 'compute'))
+        except ImportError:
+            pass
 
     def tearDown(self):
         """Clean up"""


=====================================
pyspectral/utils.py
=====================================
@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
-# Copyright (c) 2014-2018 Pytroll
+# Copyright (c) 2014-2019 Pytroll
 
 # Author(s):
 
@@ -123,9 +123,9 @@ INSTRUMENTS = {'NOAA-19': 'avhrr/3',
                'Feng-Yun 3D': 'mersi-2'
                }
 
-HTTP_PYSPECTRAL_RSR = "https://zenodo.org/record/1491277/files/pyspectral_rsr_data.tgz"
+HTTP_PYSPECTRAL_RSR = "https://zenodo.org/record/2617441/files/pyspectral_rsr_data.tgz"
 RSR_DATA_VERSION_FILENAME = "PYSPECTRAL_RSR_VERSION"
-RSR_DATA_VERSION = "v1.0.3"
+RSR_DATA_VERSION = "v1.0.5"
 
 ATM_CORRECTION_LUT_VERSION = {}
 ATM_CORRECTION_LUT_VERSION['antarctic_aerosol'] = {'version': 'v1.0.1',


=====================================
pyspectral/version.py
=====================================
@@ -46,9 +46,9 @@ def get_keywords():
     # setup.py/versioneer.py will grep for the variable names, so they must
     # each be defined on a line of their own. _version.py will just call
     # get_keywords().
-    git_refnames = " (tag: v0.8.6)"
-    git_full = "0de80ee05f736553de53f3ff1dd8e9853d3d801b"
-    git_date = "2018-12-04 08:50:14 +0100"
+    git_refnames = " (tag: v0.8.7)"
+    git_full = "4cd92661e6a5bfde3526a832e1b8eab9b741592d"
+    git_date = "2019-04-09 17:10:33 +0200"
     keywords = {"refnames": git_refnames, "full": git_full, "date": git_date}
     return keywords
 


=====================================
rsr_convert_scripts/cocts_rsr.py
=====================================
@@ -0,0 +1,90 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2019 Adam.Dybbroe
+
+# Author(s):
+
+#   Adam.Dybbroe <adam.dybbroe at smhi.se>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Read the HY-1C COCTS relative spectral responses. Data from 
+lwk1542 at hotmail.com
+
+NB! The two IR bands are NOT included.
+
+See issue 
+https://github.com/pytroll/pyspectral/issues/61
+"""
+import os
+import numpy as np
+from pyspectral.utils import INSTRUMENTS
+from pyspectral.utils import convert2hdf5 as tohdf5
+from pyspectral.raw_reader import InstrumentRSR
+
+import logging
+LOG = logging.getLogger(__name__)
+
+COCTS_BAND_NAMES = ['ch1', 'ch2', 'ch3', 'ch4', 'ch5', 'ch6', 'ch7', 'ch8']
+
+
+class COCTS_RSR(InstrumentRSR):
+
+    """Container for the HY-1C COCTS RSR data"""
+
+    def __init__(self, bandname, platform_name):
+
+        super(COCTS_RSR, self).__init__(bandname, platform_name, COCTS_BAND_NAMES)
+
+        self.instrument = INSTRUMENTS.get(platform_name, 'cocts')
+
+        self._get_options_from_config()
+        self._get_bandfilenames()
+
+        LOG.debug("Filenames: %s", str(self.filenames))
+        if self.filenames[bandname] and os.path.exists(self.filenames[bandname]):
+            self.requested_band_filename = self.filenames[bandname]
+            self._load(bandname)
+
+        else:
+            LOG.warning("Couldn't find an existing file for this band: %s",
+                        str(self.bandname))
+
+        # To be compatible with VIIRS....
+        self.filename = self.requested_band_filename
+
+    def _load(self, bandname, scale=0.001):
+        """Load the COCTS RSR data for the band requested.
+           Wavelength is given in nanometers.
+        """
+        data = np.genfromtxt(self.requested_band_filename,
+                             unpack=True,
+                             names=['wavelength', 'ch1', 'ch2', 'ch3', 'ch4', 'ch5', 'ch6', 'ch7', 'ch8'],
+                             skip_header=0)
+
+        wavelength = data['wavelength'] * scale
+        response = data[bandname]
+
+        self.rsr = {'wavelength': wavelength, 'response': response}
+
+
+def main():
+    """Main"""
+    for platform_name in ["HY-1C", ]:
+        tohdf5(COCTS_RSR, platform_name, COCTS_BAND_NAMES)
+
+
+if __name__ == "__main__":
+    main()


=====================================
rsr_convert_scripts/metimage_rsr.py
=====================================
@@ -0,0 +1,93 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2019 Adam.Dybbroe
+
+# Author(s):
+
+#   Adam.Dybbroe <adam.dybbroe at smhi.se>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Read the preliminary MetImage relative spectral response functions.
+Data from the NWPSAF, These are very early theoretical and idealized versions,
+derived from specifications I assume.
+
+"""
+
+import os
+import numpy as np
+from pyspectral.utils import convert2hdf5 as tohdf5
+from pyspectral.raw_reader import InstrumentRSR
+import logging
+
+LOG = logging.getLogger(__name__)
+
+METIMAGE_BAND_NAMES = ['ch1', 'ch2', 'ch3', 'ch4', 'ch5',
+                       'ch6', 'ch7', 'ch8', 'ch9', 'ch10',
+                       'ch11', 'ch12', 'ch13', 'ch14', 'ch15',
+                       'ch16', 'ch17', 'ch18', 'ch19', 'ch20']
+
+
+class MetImageRSR(InstrumentRSR):
+
+    """Container for the EPS-SG MetImage RSR data"""
+
+    def __init__(self, bandname, platform_name):
+
+        super(MetImageRSR, self).__init__(
+            bandname, platform_name, METIMAGE_BAND_NAMES)
+
+        self.instrument = 'metimage'
+        self._get_options_from_config()
+        self._get_bandfilenames()
+
+        LOG.debug("Filenames: %s", str(self.filenames))
+        if self.filenames[bandname] and os.path.exists(self.filenames[bandname]):
+            self.requested_band_filename = self.filenames[bandname]
+            self._load()
+
+        else:
+            LOG.warning("Couldn't find an existing file for this band: %s",
+                        str(self.bandname))
+
+        # To be compatible with VIIRS....
+        self.filename = self.requested_band_filename
+
+        self.unit = 'micrometer'
+        self.wavespace = 'wavelength'
+
+    def _load(self, scale=1.0):
+        """Load the MetImage RSR data for the band requested"""
+        data = np.genfromtxt(self.requested_band_filename,
+                             unpack=True,
+                             names=['wavenumber',
+                                    'response'],
+                             skip_header=4)
+
+        # Data are wavenumbers in cm-1:
+        wavelength = 1. / data['wavenumber'] * 10000.
+        response = data['response']
+
+        self.rsr = {'wavelength': wavelength, 'response': response}
+
+
+def main():
+    """Main"""
+    for platform_name in ["Metop-SG-A1", ]:
+        tohdf5(MetImageRSR, platform_name, METIMAGE_BAND_NAMES)
+
+
+if __name__ == "__main__":
+    main()



View it on GitLab: https://salsa.debian.org/debian-gis-team/pyspectral/compare/52c403932ae7d44e82e3ec1d20c99ef3307397fd...cc7dfaafaf35b076b061072e148afe99af1a9e84

-- 
View it on GitLab: https://salsa.debian.org/debian-gis-team/pyspectral/compare/52c403932ae7d44e82e3ec1d20c99ef3307397fd...cc7dfaafaf35b076b061072e148afe99af1a9e84
You're receiving this email because of your account on salsa.debian.org.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20190422/6a21bd8b/attachment-0001.html>


More information about the Pkg-grass-devel mailing list