[pyepr] 02/05: Imported Upstream version 0.9.2
Antonio Valentino
a_valentino-guest at moszumanska.debian.org
Sun Mar 22 18:28:17 UTC 2015
This is an automated email from the git hooks/post-receive script.
a_valentino-guest pushed a commit to branch master
in repository pyepr.
commit 4448e661b5ca2227ab0b6ce495e1fc19a388be80
Author: Antonio Valentino <antonio.valentino at tiscali.it>
Date: Sun Mar 22 17:55:45 2015 +0000
Imported Upstream version 0.9.2
---
README.txt | 2 +-
doc/NEWS.txt | 11 +-
doc/bands_example.txt | 6 +
doc/bitmask_example.txt | 19 ++-
doc/conf.py | 4 +-
doc/examples/export_gdalvrt.py | 185 +++++++++++++++++++++++++++
doc/gdal_export_example.txt | 235 +++++++++++++++++++++++++++++++++++
doc/index.txt | 7 +-
doc/interactive_use.txt | 17 +++
doc/ndvi_example.txt | 12 +-
doc/pydoctheme/static/pydoctheme.css | 20 ++-
doc/reference.txt | 37 ++++++
doc/tutorials.txt | 3 +
doc/update_example.txt | 31 ++---
doc/usermanual.txt | 166 ++++++++++++++++---------
src/epr.pyx | 4 +-
16 files changed, 666 insertions(+), 93 deletions(-)
diff --git a/README.txt b/README.txt
index ae05e3e..dd1895c 100644
--- a/README.txt
+++ b/README.txt
@@ -6,7 +6,7 @@ ENVISAT Product Reader Python API
:Author: Antonio Valentino
:Contact: antonio.valentino at tiscali.it
:Copyright: 2011-2015, Antonio Valentino <antonio.valentino at tiscali.it>
-:Version: 0.9.1
+:Version: 0.9.2
Introduction
diff --git a/doc/NEWS.txt b/doc/NEWS.txt
index 493a6e9..08946e2 100644
--- a/doc/NEWS.txt
+++ b/doc/NEWS.txt
@@ -2,10 +2,19 @@ Change history
==============
+PyEPR 0.9.2 (08/03/2015)
+------------------------
+
+* Improved string representation of fields in case of :data:`E_TID_STRING`
+ data type. Now bytes are decoded and represented as Python strings.
+* New tutorial :doc:`gdal_export_example`
+* Improved "Installation" and "Testing" sections of the user manual
+
+
PyEPR 0.9.1 (27/02/2015)
------------------------
-* Fix source distribution (missing epr-api C sources)
+* Fix source distribution (missing EPR API C sources)
PyEPR 0.9 (27/02/2015)
diff --git a/doc/bands_example.txt b/doc/bands_example.txt
index e59e92c..27df85f 100644
--- a/doc/bands_example.txt
+++ b/doc/bands_example.txt
@@ -1,6 +1,8 @@
Exporting band data
-------------------
+.. index:: ENVISAT, band, raster
+
This tutorial shows how to convert ENVISAT_ raster information from dataset
and generate flat binary rasters using PyEPR_.
@@ -196,3 +198,7 @@ powerful :class:`numpy.ndarray` interface.
.. _Python: https://www.python.org
.. _pythonic: http://www.cafepy.com/article/be_pythonic
+
+.. raw:: latex
+
+ \clearpage0+++++
diff --git a/doc/bitmask_example.txt b/doc/bitmask_example.txt
index 0232114..0b05919 100644
--- a/doc/bitmask_example.txt
+++ b/doc/bitmask_example.txt
@@ -1,6 +1,8 @@
Exporting bitmasks
-------------------
+..index:: bitmask, ENVISAT
+
This tutorial shows how to generate bit masks from ENVISAT_ flags information
as "raw" image using PyEPR_.
@@ -19,7 +21,6 @@ The program is invoked as follows:
.. _PyEPR: https://github.com/avalentino/pyepr
.. _`write_bitmask.c`: https://github.com/bcdev/epr-api/blob/master/src/examples/write_bitmask.c
-
The :download:`examples/write_bitmask.py` code consists in a single function
that also includes command line arguments handling:
@@ -29,7 +30,9 @@ that also includes command line arguments handling:
.. literalinclude:: examples/write_bitmask.py
:language: python
- :linenos:
+
+.. index:: EPR-API, module
+ pair: epr; module
In order to use the Python_ EPR API the :mod:`epr` module is imported:
@@ -40,6 +43,9 @@ In order to use the Python_ EPR API the :mod:`epr` module is imported:
.. literalinclude:: examples/write_bitmask.py
:lines: 27
+.. index:: product, open
+ pair: with; statement
+
As usual the ENVISAT_ product is opened using the :func:`epr.open` function
that returns an :class:`epr.Product` instance.
In this case the :func:`epr.open` is used together with a ``with`` statement
@@ -54,6 +60,8 @@ program exits the ``with`` block.
:language: python
:lines: 51-52
+.. index:: product
+
Scene size parameters are retrieved form the :class:`epr.Product` object
using the :meth:`epr.Product.get_scene_width` and
:meth:`epr.Product.get_scene_height` methods:
@@ -98,6 +106,8 @@ The :meth:`epr.Product.read_bitmask_raster` method receives in input the
of the bit-mask. In a bit-mask expression, any number of the flag-names
(found in the DDDB) can be composed with “(”, ”)”, “NOT”, “AND”, “OR”.
+.. index:: AND, OR, NOT, DDDB
+
Valid bit-mask expression are for example::
flags.LAND OR flags.CLOUD
@@ -122,3 +132,8 @@ of the :class:`epr.Raster` objects that exposes data via the
.. _Python: https://www.python.org
.. _ENVISAT: https://envisat.esa.int
+
+
+.. raw:: latex
+
+ \clearpage
diff --git a/doc/conf.py b/doc/conf.py
index d22d603..5272fce 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -68,9 +68,9 @@ copyright = u'2011-2015, Antonio Valentino'
# built documents.
#
# The short X.Y version.
-version = '0.9.1'
+version = '0.9.2'
# The full version, including alpha/beta/rc tags.
-release = version #+ 'dev'
+release = version # + 'dev'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
diff --git a/doc/examples/export_gdalvrt.py b/doc/examples/export_gdalvrt.py
new file mode 100755
index 0000000..a51b0a9
--- /dev/null
+++ b/doc/examples/export_gdalvrt.py
@@ -0,0 +1,185 @@
+#!/usr/bin/env python3
+
+import os
+import epr
+from osgeo import gdal
+
+
+epr_to_gdal_type = {
+ epr.E_TID_UNKNOWN: gdal.GDT_Unknown,
+ epr.E_TID_UCHAR: gdal.GDT_Byte,
+ epr.E_TID_CHAR: gdal.GDT_Byte,
+ epr.E_TID_USHORT: gdal.GDT_UInt16,
+ epr.E_TID_SHORT: gdal.GDT_Int16,
+ epr.E_TID_UINT: gdal.GDT_UInt32,
+ epr.E_TID_INT: gdal.GDT_Int32,
+ epr.E_TID_FLOAT: gdal.GDT_Float32,
+ epr.E_TID_DOUBLE: gdal.GDT_Float64,
+ #epr.E_TID_STRING: gdal.GDT_Unknown,
+ #epr.E_TID_SPARE: gdal.GDT_Unknown,
+ #epr.E_TID_TIME: gdal.GDT_Unknown,
+}
+
+
+def epr2gdal_band(band, vrt):
+ product = band.product
+ dataset = band.dataset
+ record = dataset.read_record(0)
+ field = record.get_field_at(band._field_index - 1)
+
+ ysize = product.get_scene_height()
+ xsize = product.get_scene_width()
+
+ if isinstance(vrt, gdal.Dataset):
+ if (vrt.RasterYSize, vrt.RasterXSize) != (ysize, xsize):
+ raise ValueError('dataset size do not match')
+ gdal_ds = vrt
+ elif os.path.exists(vrt):
+ gdal_ds = gdal.Open(vrt, gdal.GA_Update)
+ if gdal_ds is None:
+ raise RuntimeError('unable to open "{}"'.format(vrt))
+ driver = gdal_ds.GetDriver()
+ if driver.ShortName != 'VRT':
+ raise TypeError('unexpected GDAL driver ({}). '
+ 'VRT driver expected'.format(driver.ShortName))
+ else:
+ driver = gdal.GetDriverByName('VRT')
+ if driver is None:
+ raise RuntimeError('unable to get driver "VRT"')
+
+ gdal_ds = driver.Create(vrt, xsize, ysize, 0)
+ if gdal_ds is None:
+ raise RuntimeError('unable to create "{}" dataset'.format(vrt))
+
+ filename = os.pathsep.join(product.file_path.split('/')) # denormalize
+ offset = dataset.get_dsd().ds_offset + field.get_offset()
+ line_offset = record.tot_size
+ pixel_offset = epr.get_data_type_size(field.get_type())
+
+ if band.sample_model == epr.E_SMOD_1OF2:
+ pixel_offset *= 2
+ elif band.sample_model == epr.E_SMOD_2OF2:
+ offset += pixel_offset
+ pixel_offset *= 2
+
+ options = [
+ 'subClass=VRTRawRasterBand',
+ 'SourceFilename={}'.format(filename),
+ 'ImageOffset={}'.format(offset),
+ 'LineOffset={}'.format(line_offset),
+ 'PixelOffset={}'.format(pixel_offset),
+ 'ByteOrder=MSB',
+ ]
+
+ gtype = epr_to_gdal_type[field.get_type()]
+ ret = gdal_ds.AddBand(gtype, options=options)
+ if ret != gdal.CE_None:
+ raise RuntimeError(
+ 'unable to add VRTRawRasterBand to "{}"'.format(vrt))
+
+ gdal_band = gdal_ds.GetRasterBand(gdal_ds.RasterCount)
+ gdal_band.SetDescription(band.description)
+ metadata = {
+ 'name': band.get_name(),
+ 'dataset_name': dataset.get_name(),
+ 'dataset_description': dataset.description,
+ 'lines_mirrored': str(band.lines_mirrored),
+ 'sample_model': epr.get_sample_model_name(band.sample_model),
+ 'scaling_factor': str(band.scaling_factor),
+ 'scaling_offset': str(band.scaling_offset),
+ 'scaling_method': epr.get_scaling_method_name(band.scaling_method),
+ 'spectr_band_index': str(band.spectr_band_index),
+ 'unit': band.unit if band.unit else '',
+ 'bm_expr': band.bm_expr if band.bm_expr else '',
+ }
+ gdal_band.SetMetadata(metadata)
+
+ return gdal_ds
+
+
+def epr2gdal(product, vrt, overwrite_existing=False):
+ if isinstance(product, str):
+ filename = product
+ product = epr.open(filename)
+
+ ysize = product.get_scene_height()
+ xsize = product.get_scene_width()
+
+ if os.path.exists(vrt) and not overwrite_existing:
+ raise ValueError('unable to create "{0}". Already exists'.format(vrt))
+
+ driver = gdal.GetDriverByName('VRT')
+ if driver is None:
+ raise RuntimeError('unable to get driver "VRT"')
+
+ gdal_ds = driver.Create(vrt, xsize, ysize, 0)
+ if gdal_ds is None:
+ raise RuntimeError('unable to create "{}" dataset'.format(vrt))
+
+ metadata = {
+ 'id_string': product.id_string,
+ 'meris_iodd_version': str(product.meris_iodd_version),
+ 'dataset_names': ','.join(product.get_dataset_names()),
+ 'num_datasets': str(product.get_num_datasets()),
+ 'num_dsds': str(product.get_num_dsds()),
+ }
+ gdal_ds.SetMetadata(metadata)
+
+ mph = product.get_mph()
+ metadata = str(mph).replace(' = ', '=').split('\n')
+ gdal_ds.SetMetadata(metadata, 'MPH')
+
+ sph = product.get_sph()
+ metadata = str(sph).replace(' = ', '=').split('\n')
+ gdal_ds.SetMetadata(metadata, 'SPH')
+
+ for band in product.bands():
+ epr2gdal_band(band, gdal_ds)
+
+ # @TODO: set geographic info
+
+ return gdal_ds
+
+
+if __name__ == '__main__':
+ filename = 'MER_LRC_2PTGMV20000620_104318_00000104X000_00000_00000_0001.N1'
+ vrtfilename = os.path.splitext(filename)[0] + '.vrt'
+
+ gdal_ds = epr2gdal(filename, vrtfilename)
+
+ with epr.open(filename) as product:
+ band_index = product.get_band_names().index('water_vapour')
+ band = product.get_band('water_vapour')
+ eprdata = band.read_as_array()
+ unit = band.unit
+ lines_mirrored = band.lines_mirrored
+ scaling_offset = band.scaling_offset
+ scaling_factor = band.scaling_factor
+
+ gdal_band = gdal_ds.GetRasterBand(band_index + 1)
+ vrtdata = gdal_band.ReadAsArray()
+
+ if lines_mirrored:
+ vrtdata = vrtdata[:, ::-1]
+
+ vrtdata = vrtdata * scaling_factor + scaling_offset
+
+ print('Max absolute error:', abs(vrtdata - eprdata).max())
+
+ # plot
+ from matplotlib import pyplot as plt
+
+ plt.figure()
+ plt.subplot(2, 1, 1)
+ plt.imshow(eprdata)
+ plt.grid(True)
+ cb = plt.colorbar()
+ cb.set_label(unit)
+ plt.title('EPR data')
+ plt.subplot(2, 1, 2)
+ plt.imshow(vrtdata)
+ plt.grid(True)
+ cb = plt.colorbar()
+ cb.set_label(unit)
+ plt.title('VRT data')
+ plt.show()
diff --git a/doc/gdal_export_example.txt b/doc/gdal_export_example.txt
new file mode 100644
index 0000000..5560eff
--- /dev/null
+++ b/doc/gdal_export_example.txt
@@ -0,0 +1,235 @@
+GDAL_ export example
+--------------------
+
+.. index:: GDAL, export, VRT, ENVISAT, offset
+
+This tutorial explains how to use PyEPR_ to generate a file in GDAL Virtual
+Format (VRT) that can be used to access data with the powerful and popular
+GDAL_ library.
+GDAL_ already has support for ENVISAT_ products but this example is
+interesting for two reasons:
+
+* it exploits some low level feature (like e.g. offset management) that are
+ rarely used but that can be very useful in some cases
+* the generated VRT file uses raw raster access and it can be opened in
+ update mode to modify the ENVISAT_ product data.
+ This feature is not supported by the native ENVISAT_ driver of the GDAL_
+ library
+
+
+export_gdalvrt module
+~~~~~~~~~~~~~~~~~~~~~
+
+The complete code of the example is available in the
+:download:`examples/export_gdalvrt.py` file.
+It is organized so that it can also be imported as a module in any program.
+
+The :mod:`export_gdalvrt` module provides two functions:
+
+.. module:: export_gdalvrt
+
+
+.. function:: epr2gdal_band(band, vrt)
+
+ Takes in input an :class:`epr.Band` object and a VRT dataset
+ and add a GDAL_ band to the VRT dataset
+
+
+.. function:: epr2gdal(product, vrt, overwrite_existing=False)
+
+ Takes in input a PyEPR_ :class:`Product` (or a filename) and the
+ file name of the output VRT file and generates the VRT file itself
+ containing a band for each :class:`epr.Band` present in the original
+ :class:`epr.Product` and also associated metadata.
+
+
+The :func:`epr2gdal` function first creates the VRT dataset
+
+.. raw:: latex
+
+ \fvset{fontsize=\footnotesize}
+
+.. literalinclude:: examples/export_gdalvrt.py
+ :language: python
+ :lines: 101-117
+
+and then loops on all :class:`epr.Band`\ s of the PyEPR_ :class:`epr.Product`
+calling the :func:`epr2gdal_band` function on each of them:
+
+.. raw:: latex
+
+ \fvset{fontsize=\footnotesize}
+
+.. literalinclude:: examples/export_gdalvrt.py
+ :language: python
+ :lines: 136-137
+
+The :mod:`export_gdalvrt` module also provides a :data:`epr_to_gdal_type`
+mapping between EPR and GDAL data type identifiers.
+
+
+.. index:: VRTRawRasterBand
+
+Generating *VRTRawRasterBand*\ s
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The core of the example is the part of the code in the :func:`epr2gdal_band`
+function that generates the GDAL_ *VRTRawRasterBand*.
+It is a description of a raster file that the GDAL_ library uses for low level
+data access.
+Of course the entire machinery works because data in :class:`epr.Band`\s and
+:class:`epr.Dataset`\ s of ENVISAT_ products are stored as contiguous
+rasters.
+
+.. raw:: latex
+
+ \fvset{fontsize=\footnotesize}
+
+.. literalinclude:: examples/export_gdalvrt.py
+ :language: python
+ :lines: 54-78
+
+.. index:: offset
+
+The fundamental part is the computation of the:
+
+**ImageOffset**:
+
+ the offset in bytes to the beginning of the first pixel of data with
+ respect to the beginning of the file.
+
+ In the example it is computed using
+
+ * the :data:`epr.DSD.ds_offset` attribute, that represents the offset
+ in bytes of the :class:`epr.Dataset` from the beginning of the file,
+ and
+ * the :meth:`epr.Field.get_offset` method that returns the offset in
+ bytes of the :class:`epr.Field` containing :class:`epr.Band` data from
+ the beginning of the :class:`epr.Record`
+
+ ::
+
+ offset = dataset.get_dsd().ds_offset + field.get_offset()
+
+**LineOffset**:
+
+ the offset in bytes from the beginning of one *scanline* of data and the
+ next scanline of data.
+ In the example it is set to the :class:`epr.Record` size in bytes using
+ the :attr:`epr.Record.tot_size` attribute::
+
+ line_offset = record.tot_size
+
+**PixelOffset**:
+
+ the offset in bytes from the beginning of one pixel and the next on
+ the same line. Usually it corresponds to the size in bytes of the
+ elementary data type.
+ It is set using the :meth:`epr.Field.get_type` method and the
+ :func:`epr.get_data_type_size` function::
+
+ pixel_offset = epr.get_data_type_size(field.get_type())
+
+The band size in lines and columns of the GDAL_ bands is fixed at GDAL_
+dataset level when it is created:
+
+.. raw:: latex
+
+ \fvset{fontsize=\footnotesize}
+
+.. literalinclude:: examples/export_gdalvrt.py
+ :language: python
+ :lines: 50-52
+
+.. index:: dataset, band
+
+Please note that in case of :class:`epr.Dataset`\ s storing complex values,
+like in `MDS1` :class:`epr.Dataset` of ASAR IMS :class:`epr.Product`\ s,
+pixels of real and imaginary parts are interleaved, so to represent
+:class:`epr.Band`\ s of the two components the pixel offset have to be
+doubled and an additional offset (one pixel) must be added to the
+*ImageOffset* of the :class:`epr.Band` representing the imaginary part:
+
+.. raw:: latex
+
+ \fvset{fontsize=\footnotesize}
+
+.. literalinclude:: examples/export_gdalvrt.py
+ :language: python
+ :lines: 59-63
+
+.. note::
+
+ the PyEPR_ API does not supports complex :class:`Band`\ s.
+ :class:`epr.Dataset`\s containing complex data, like the `MDS1`
+ :class:`epr.Dataset` of ASAR IMS :class:`epr.Product`\ s, are associated
+ to two distinct :class:`epr.Band`\s containing the real (I) and the
+ imaginary (Q) component respectively.
+
+ GDAL_, instead, supports complex data types, so it is possible to map a
+ complex ENVISAT_ :class:`epr.Dataset` onto a single GDAL_ bands with
+ complex data type.
+
+ This case is not handled in the example.
+
+
+.. index:: metadata
+
+Metadata
+~~~~~~~~
+
+The :func:`epr2gdal_band` function also stores a small set of metadata for
+each :class:`epr.Band`:
+
+.. raw:: latex
+
+ \fvset{fontsize=\footnotesize}
+
+.. literalinclude:: examples/export_gdalvrt.py
+ :language: python
+ :lines: 80-95
+
+Metadata are also stored at GDAL_ dataset level by the :func:`epr2gdal`
+function:
+
+.. raw:: latex
+
+ \fvset{fontsize=\footnotesize}
+
+.. literalinclude:: examples/export_gdalvrt.py
+ :language: python
+ :lines: 119-126
+
+.. index:: MPH, SPH
+
+The :func:`epr2gdal` function also stores the contents of the *MPH* and the
+*SPH* records as GDAL_ dataset matadata in custom domains:
+
+.. raw:: latex
+
+ \fvset{fontsize=\footnotesize}
+
+.. literalinclude:: examples/export_gdalvrt.py
+ :language: python
+ :lines: 128-134
+
+
+Complete listing
+~~~~~~~~~~~~~~~~
+
+.. raw:: latex
+
+ \fvset{fontsize=\footnotesize}
+
+.. literalinclude:: examples/export_gdalvrt.py
+ :language: python
+
+
+.. _GDAL: http://www.gdal.org
+.. _PyEPR: https://github.com/avalentino/pyepr
+.. _ENVISAT: https://envisat.esa.int
+
+
+.. raw:: latex
+
+ \clearpage
diff --git a/doc/index.txt b/doc/index.txt
index 2d6d31c..605d346 100644
--- a/doc/index.txt
+++ b/doc/index.txt
@@ -37,6 +37,8 @@ ENVISAT Product Reader Python API
.. _ENVISAT: https://envisat.esa.int
.. _ESA: https://earth.esa.int
+ .. index:: ENVISAT, MERIS, AASTR, ASAR
+
Documentation
=============
@@ -58,7 +60,8 @@ ENVISAT Product Reader Python API
Online documentation for other PyEpr_ versions:
* `latest <https://pyepr.readthedocs.org/en/latest/>`_ development
- * `0.9.1 <https://pyepr.readthedocs.org/en/v0.9.1/>`_ (latest stable)
+ * `0.9.2 <https://pyepr.readthedocs.org/en/v0.9.2/>`_ (latest stable)
+ * `0.9.1 <https://pyepr.readthedocs.org/en/v0.9.1/>`_
* `0.9 <https://pyepr.readthedocs.org/en/v0.9/>`_
* `0.8.2 <https://pyepr.readthedocs.org/en/v0.8.2/>`_
* `0.8.1 <https://pyepr.readthedocs.org/en/v0.8.1/>`_
@@ -69,6 +72,8 @@ ENVISAT Product Reader Python API
* `0.6 <https://pyepr.readthedocs.org/en/v0.6/>`_
+.. index:: license
+
License
=======
diff --git a/doc/interactive_use.txt b/doc/interactive_use.txt
index a84edd7..d090e98 100644
--- a/doc/interactive_use.txt
+++ b/doc/interactive_use.txt
@@ -3,6 +3,10 @@ Interactive use of PyEPR_
.. highlight:: ipython
+.. index:: ipython, interactive, ENVISAT, ASAR, ESA, pylab, matplotlib
+ pair: interactive; shell
+ pair: sample; dataset
+
In this tutorial it is showed an example of how to use PyEPR_ interactively
to open, browse and display data of an ENVISAT_ ASAR_ product.
@@ -22,6 +26,9 @@ ESA_ web site.
.. _ESA: https://earth.esa.int
+.. index:: module
+ pair: epr; module
+
:mod:`epr` module and classes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -77,6 +84,8 @@ available classes and functions::
In [3]: epr.__version__, epr.EPR_C_API_VERSION
Out[3]: ('0.9.1', '2.3dev')
+.. index:: __version__
+
Docstrings are available for almost all classes, methods and functions in
the :mod:`epr` and they can be displayed using the :func:`help` python_
command or the ``?`` IPython_ shortcut as showed above.
@@ -110,6 +119,8 @@ complete commands or to display available functions and classes::
epr.E_TID_SPARE epr.sys
+.. index:: product
+
:class:`epr.Product` navigation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -219,6 +230,8 @@ get all desired information from the product::
range_spacing = 12.500000
+.. index:: iteration, iterable, record
+
Iterating over :mod:`epr` objects
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -244,6 +257,8 @@ the following::
beam_merge_alg_param: 4 elements
+.. index:: data, image
+
Image data
~~~~~~~~~~
@@ -302,6 +317,8 @@ Dealing with image data is simple as well::
.. _iterable: http://docs.python.org/glossary.html#term-iterable
+.. index:: close, product
+
Closing the epr.Product
~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/ndvi_example.txt b/doc/ndvi_example.txt
index 86a90be..55fb268 100644
--- a/doc/ndvi_example.txt
+++ b/doc/ndvi_example.txt
@@ -1,6 +1,8 @@
NDVI computation
----------------
+.. index:: NDVI, MERIS
+
This tutorial shows how to use PyEPR_ to open a MERIS_ L1B product, compute
the *Normalized Difference Vegetation Index* (NDVI) and store it into a flat
binary file.
@@ -27,7 +29,6 @@ The code have been kept very simple and it consists in a single function
\fvset{fontsize=\footnotesize}
.. literalinclude:: examples/write_ndvi.py
- :linenos:
The ENVISAT_ :class:`epr.Product` is opened using the :func:`epr.open`
function.
@@ -39,6 +40,9 @@ function.
.. literalinclude:: examples/write_ndvi.py
:lines: 49
+.. index:: context, open
+ pair: with; statement
+
As usual in modern python programs the *with* statement has been used to
ensure that the product is automatically closed as soon as the program exits
the block.
@@ -72,6 +76,8 @@ method:
:class:`epr.Raster` objects that allow to access data matrices with the
radiance values.
+.. index:: raster, memory
+
Before reading data into the :class:`epr.Raster` objects they have to be
instantiated specifying their size and data type in order to allow the library
to allocate the correct amount of memory.
@@ -189,3 +195,7 @@ used to write the NDVI of the pixel n the file in binary format.
.. _Python: https://www.python.org
.. _pythonic: http://www.cafepy.com/article/be_pythonic
+
+.. raw:: latex
+
+ \clearpage
diff --git a/doc/pydoctheme/static/pydoctheme.css b/doc/pydoctheme/static/pydoctheme.css
index 5e1bd14..50835bb 100644
--- a/doc/pydoctheme/static/pydoctheme.css
+++ b/doc/pydoctheme/static/pydoctheme.css
@@ -94,31 +94,31 @@ div.body div.seealso {
}
div.body a {
- color: #00608f;
+ color: #0072aa;
}
div.body a:visited {
- color: #30306f;
+ color: #6363bb;
}
div.body a:hover {
color: #00B0E4;
}
-tt, pre {
+tt, code, pre {
font-family: monospace, sans-serif;
font-size: 96.5%;
}
-div.body tt {
+div.body tt, div.body code {
border-radius: 3px;
}
-div.body tt.descname {
+div.body tt.descname, div.body code.descname {
font-size: 120%;
}
-div.body tt.xref, div.body a tt {
+div.body tt.xref, div.body a tt, div.body code.xref, div.body a code {
font-weight: normal;
}
@@ -168,3 +168,11 @@ div.footer {
div.footer a:hover {
color: #0095C4;
}
+
+.refcount {
+ color: #060;
+}
+
+.stableabi {
+ color: #229;
+}
diff --git a/doc/reference.txt b/doc/reference.txt
index a75e239..0fe4e0a 100644
--- a/doc/reference.txt
+++ b/doc/reference.txt
@@ -4,6 +4,9 @@ API Reference
.. module:: epr
:synopsis: Python bindings for ENVISAT Product Reader C API
+.. index:: bindings, ENVISAT, ESA, EPR-API
+ pair: epr; module
+
PyEPR_ provides Python_ bindings for the ENVISAT Product Reader C API
(`EPR API`_) for reading satellite data from ENVISAT_ ESA_ (European
Space Agency) mission.
@@ -266,6 +269,9 @@ Product
* __enter__
* __exit__
+ .. index:: __repr__, __str__, __enter__, __exit__
+ pair: special; methods
+
Dataset
~~~~~~~
@@ -380,6 +386,9 @@ Dataset
* __str__
* __iter__
+ .. index:: __repr__, __str__, __iter__
+ pair: special; methods
+
Record
~~~~~~
@@ -536,6 +545,9 @@ Record
* __str__
* __iter__
+ .. index:: __repr__, __str__, __iter__
+ pair: special; methods
+
Field
~~~~~
@@ -691,6 +703,9 @@ Field
* __ne__
* __len__ [#]_
+ .. index:: __repr__, __str__, __eq__, __ne__, __len__
+ pair: special; methods
+
.. rubric:: Footnotes
.. [#] if the field is a :data:`E_TID_STRING` field then the
@@ -761,6 +776,9 @@ DSD
* __eq__
* __ne__
+ .. index:: __repr__, __eq__, __ne__
+ pair: special; methods
+
Band
~~~~
@@ -1061,6 +1079,9 @@ Band
* __repr__
+ .. index:: __repr__
+ pair: special; methods
+
Raster
~~~~~~
@@ -1167,6 +1188,9 @@ Raster
* __repr__
+ .. index:: __repr__
+ pair: special; methods
+
EPRTime
~~~~~~~
@@ -1182,6 +1206,8 @@ EPRTime
.. attribute:: microseconds
+.. index:: function
+
Functions
---------
@@ -1301,6 +1327,8 @@ Functions
.. seealso:: the description of :meth:`Band.create_compatible_raster`
+.. index:: exception, error
+
Exceptions
----------
@@ -1364,6 +1392,9 @@ Data type identifiers
.. data:: E_TID_TIME
+.. index::
+ pair: sample; model
+
Sample Models
~~~~~~~~~~~~~
@@ -1374,6 +1405,9 @@ Sample Models
.. data:: E_SMOD_2TOF
+.. index::
+ pair: scaling; method
+
Scaling Methods
~~~~~~~~~~~~~~~
@@ -1386,8 +1420,11 @@ Scaling Methods
Linear pixel scaling
+ .. index:: linear
+
.. data:: E_SMID_LOG
Logarithmic pixel scaling
+ .. index:: logarithmic
diff --git a/doc/tutorials.txt b/doc/tutorials.txt
index e68d26b..70f87af 100644
--- a/doc/tutorials.txt
+++ b/doc/tutorials.txt
@@ -1,6 +1,8 @@
Tutorials
=========
+.. index:: tutorial
+
.. toctree::
:maxdepth: 2
@@ -9,4 +11,5 @@ Tutorials
bitmask_example
ndvi_example
update_example
+ gdal_export_example
diff --git a/doc/update_example.txt b/doc/update_example.txt
index ad08e02..8899731 100644
--- a/doc/update_example.txt
+++ b/doc/update_example.txt
@@ -3,6 +3,9 @@ Update :class:`Field` elements
.. highlight:: ipython
+.. index:: update, field, read-only, EPR-API, MERIS
+ pair: open; mode
+
The EPR C API has been designed to provide read-only features.
PyEPR_ provides and extra capability consisting in the possibility to
@@ -16,7 +19,6 @@ content at a specific position.
One can load water vapour and compute an histogram using the following
instructions:
-
.. raw:: latex
\fvset{fontsize=\footnotesize}
@@ -25,10 +27,10 @@ instructions:
:language: python
:lines: 10-15
+.. index:: matplotlib
The resulting histogram can be plot using Matplotlib_:
-
.. raw:: latex
\fvset{fontsize=\footnotesize}
@@ -37,19 +39,16 @@ The resulting histogram can be plot using Matplotlib_:
:language: python
:lines: 17-21
-
.. figure:: images/water_vapour_histogram_01.*
:width: 60%
Histogram of the original water vapour content
-
The actual values of the water vapour content :class:`Band` are computed
starting form data stored in the `Vapour_Content` :class:`Dataset` using
scaling factors contained in the `Scaling_Factor_GADS` :class:`Dataset`.
In particular :class:`Field`\ s `sf_wvapour` and `off_wvapour` are used::
-
In [21]: dataset = product.get_dataset('Scaling_Factor_GADS')
In [22]: print(dataset)
@@ -63,6 +62,8 @@ In particular :class:`Field`\ s `sf_wvapour` and `off_wvapour` are used::
off_wvapour = -0.100000
spare_1 = <<unknown data type>>
+.. index:: band
+ pair: scaling; factor
Now suppose that for some reason one needs to update the `sf_wvapour` scaling
factor for the water vapour content.
@@ -72,7 +73,6 @@ Changing the scaling factor, of course, will change all values in the
The change can be performed using the :meth:`Field.set_elem` and
:meth:`Field.set_elems` methods of :class:`Field` objects:
-
.. raw:: latex
\fvset{fontsize=\footnotesize}
@@ -81,12 +81,10 @@ The change can be performed using the :meth:`Field.set_elem` and
:language: python
:lines: 24-32
-
Now the `sf_wvapour` scaling factor has been changed and it is possible to
compute and display the histogram of modified data in the `water_vapour`
:class:`Band`:
-
.. raw:: latex
\fvset{fontsize=\footnotesize}
@@ -95,25 +93,21 @@ compute and display the histogram of modified data in the `water_vapour`
:language: python
:lines: 34-48
-
.. figure:: images/water_vapour_histogram_02.*
:width: 60%
Histogram of the water vapour content (original and modified)
-
Figure above shows the two different histograms, original data in blue and
modified data in red, demonstrating the effect of the change of the scaling
factor.
The new map of water vapour is showed in the following picture:
-
.. figure:: images/modified_water_vapour.*
Modified water vapour content map
-
.. important::
it is important to stress that it is necessary to close and re-open the
@@ -123,7 +117,6 @@ The new map of water vapour is showed in the following picture:
This is a limitation of the current implementation that could be removed
in future versions of the PyEPR_ package.
-
It has been showed that changing the `sf_wvapour` scaling factor modifies
all values of the `water_vapour` :class:`Band`.
@@ -135,7 +128,6 @@ The :class:`Dataset` size can be read form the :class:`Product`::
In [44]: product.get_scene_height(), product.get_scene_width()
Out[44]: (149, 281)
-
while information about the fields in each record can be retrieved
introspecting the :class:`Record` object::
@@ -147,13 +139,11 @@ introspecting the :class:`Record` object::
In [51]: record.get_field('wvapour_cont_pix')
Out[51]: epr.Field("wvapour_cont_pix") 281 uchar elements
-
So the name of the :class:`Field` we need to change is the `wvapour_cont_pix`,
and its index is `2`.
It is possible to change a small box inside the :class:`Dataset` as follows:
-
.. raw:: latex
\fvset{fontsize=\footnotesize}
@@ -162,7 +152,6 @@ It is possible to change a small box inside the :class:`Dataset` as follows:
:language: python
:lines: 60-68
-
Please note that when one modifies the content of a :class:`Dataset` he/she
should also take into account id the corresponding band has lines mirrored
or not::
@@ -172,11 +161,9 @@ or not::
In [60]: band.lines_mirrored
Out[60]: True
-
Finally the :class:`Product` can be re-opened to load and display the
modified :class:`Band`:
-
.. raw:: latex
\fvset{fontsize=\footnotesize}
@@ -185,12 +172,10 @@ modified :class:`Band`:
:language: python
:lines: 71-82
-
.. figure:: images/modified_water_vapour_with_box.*
Modified water vapour content map with zeroed box
-
Of course values in the box that has been set to zero in the :class:`Dataset`
are transformed according to the scaling factor and offset parameters
associated to `water_vapour` :class:`Band`.
@@ -203,3 +188,7 @@ The complete code of the example can be found at
.. _ENVISAT: https://envisat.esa.int
.. _Matplotlib: http://matplotlib.org
+
+.. raw:: latex
+
+ \clearpage
diff --git a/doc/usermanual.txt b/doc/usermanual.txt
index 21b05c9..87fe071 100644
--- a/doc/usermanual.txt
+++ b/doc/usermanual.txt
@@ -1,6 +1,9 @@
User Manual
===========
+.. index:: EPR-API, ENVISAT, MERIS, AASTR, ASAR, ESA
+ single: product; dataset; record; close
+ pair: epr; module
Quick start
-----------
@@ -16,7 +19,7 @@ or on a raw data layer. The raw data access makes it possible to read
any data field contained in a product file.
Full access to the Python EPR API is provided by the :mod:`epr` module that
-have to be imported by the client program e-g- as follows::
+have to be imported by the client program e.g. as follows::
import epr
@@ -38,6 +41,11 @@ The user can open a product in update mode ('rb+') and call the
:meth:`epr.Field.set_elem` and :meth:`epr.Field.set_elems` methods of
:class:`epr.Field` class to update its elements and write changes to disk.
+.. seealso:: `Update support`_ and :doc:`update_example` tutorial for details.
+
+.. index::
+ pair: open; mode
+
.. _PyEPR: https://github.com/avalentino/pyepr
.. _Python: https://www.python.org
@@ -46,6 +54,8 @@ The user can open a product in update mode ('rb+') and call the
.. _ESA: https://earth.esa.int
+.. index:: requirements, EPR-API, Python, numpy, cython, unittest2, gcc, extension
+
Requirements
------------
@@ -63,7 +73,7 @@ correctly installed and configured:
.. [#] PyEPR_ has been developed and tested with gcc_ 4.
.. [#] The source tarball of official releases also includes the C extension
- code generated by cython_ so users don's strictly need cython_ to
+ code generated by cython_ so users don't strictly need cython_ to
install PyEPR_.
It is only needed to re-generate the C extension code (e.g. if one
@@ -77,6 +87,9 @@ correctly installed and configured:
.. _unittest2: https://pypi.python.org/pypi/unittest2
+.. index:: download, PyPi, GitHub, project, git
+ pair: git; clone
+
Download
--------
@@ -100,52 +113,48 @@ To clone the git_ repository the following command can be used::
.. _git: http://git-scm.com
+.. index:: install, pip
+ pair: install; user
+ pair: install; option
+ pair: install; prefix
+
.. _installation:
Installation
------------
-The easier way to install PyEPR_ is using tools like pip_ or easy_install_::
-
- $ pip install numpy pyepr
+The easier way to install PyEPR_ is using tools like pip_::
-.. note::
-
- the :file:`setup.py` script does not use easy_install_ specific functions
- so it is unable to handle dependencies automatically.
-
- Also the :file:`setup.py` script uses numpy_ to retrieve the path of
- headers and libraries.
- For this reaseon numpy_ must be already installed when :file:`setup.py`
- is executed.
+ $ pip install pyepr
- In the above example the required numpy_ package is explicitly included in
- the list of packages to be installed.
+For a user specific installation please use::
- .. seealso:: Requirements_
-
-PyEPR_ uses the standard Python_ distutils_ so it can be installed from
-sources using the following command::
-
- $ python setup.py install
-
-For a user specific installation use::
-
- $ python setup.py install --user
+ $ pip install --user pyepr
To install PyEPR_ in a non-standard path::
- $ python setup.py install --prefix=<TARGET_PATH>
+ $ pip install --install-option="--prefix=<TARGET_PATH>" pyepr
just make sure that :file:`<TARGET_PATH>/lib/pythonX.Y/site-packages` is in
the :envvar:`PYTHONPATH`.
+.. index::
+ single: sources; setup.py
+ pair: standalone; mode
+ pair: EPR-API; sources
+ pair: dynamic; library
+ pair: git; repository
+
+PyEPR_ can be installed from sources using the following command::
+
+ $ python setup.py install
+
The :file:`setup.py` script by default checks for the availability of the
EPR C API source code in the :file:`<package-root>/epr-api-src` directory
and tries to build PyEPR in *standalone mode*, i.e. without linking an
external dynamic library of EPR-API.
-If no EPR C API sources are found then the :file:`setup.py` of PyEPR
+If no EPR C API sources are found then the :file:`setup.py` script
automatically tries to link the EPR-API dynamic library.
This can happen, for example, if the user is using a copy of the PyEPR
sources cloned from a git_ repository.
@@ -163,40 +172,43 @@ with the system `EPR API`_ C library::
$ python setup.py install --epr-api-src=None
.. _pip: https://pypi.python.org/pypi/pip
-.. _easy_install: https://pypi.python.org/pypi/setuptools#using-setuptools-and-easyinstall
-.. _distutils: https://docs.python.org/3/distutils
+.. index:: test, setup.py, download
+ pair: test; suite
+ pair: sample; product
+
Testing
-------
-PyEPR_ package comes with a complete test suite but in order to run it
-the ENVISAT sample product used for testing,
-MER_LRC_2PTGMV20000620_104318_00000104X000_00000_00000_0001.N1__,
-have to be downloaded from the ESA_ website, saved in the :file:`tests`
-directory and decompressed.
+PyEPR_ package comes with a complete test suite.
+The test suite can be run using the following command in the :file:`tests`
+directory::
-__ https://earth.esa.int/services/sample_products/meris/LRC/L2/MER_LRC_2PTGMV20000620_104318_00000104X000_00000_00000_0001.N1.gz
+ $ python test_all.py
-.. note::
+or from the package root directory::
- since PyEPR_ 0.9.0 the test product is downloaded and decompressed
- automatically by the test script if not already available.
- In this case a working Internet connection is expected to be available
- when the test suite is run.
+ $ python setup.py test
-On GNU Linux platforms the following shell commands can be used::
+The test script automatically downloads and decompresses the ENVISAT sample
+product necessary for testing,
+MER_LRC_2PTGMV20000620_104318_00000104X000_00000_00000_0001.N1__,
+if it is not already available in the :file:`tests` directory.
- $ cd pyepr-X.Y.Z/tests
- $ wget https://earth.esa.int/services/sample_products/meris/LRC/L2/\
- MER_LRC_2PTGMV20000620_104318_00000104X000_00000_00000_0001.N1.gz
- $ gunzip MER_LRC_2PTGMV20000620_104318_00000104X000_00000_00000_0001.N1.gz
+.. note::
-After installation the test suite can be run using the following command
-in the :file:`tests` directory::
+ please note that, unless the user already have a copy of the specified
+ sample product correctly installed, an **internet connection** is
+ necessary the first time that the test suite is run.
+
+ After the first run the sample product remains in the :file:`tests`
+ directory so the internet access is no longer necessary.
+
+__ https://earth.esa.int/services/sample_products/meris/LRC/L2/MER_LRC_2PTGMV20000620_104318_00000104X000_00000_00000_0001.N1.gz
- $ python test_all.py
+.. index:: EPR-API
Python vs C API
---------------
@@ -215,6 +227,9 @@ following sections.
.. _`C API`: https://rawgithub.com/bcdev/epr-api/master/docs/epr_c_api/index.html
+.. index:: memory, product
+ pair: allocation; de-allocation
+
Memory management
-----------------
@@ -238,6 +253,8 @@ For example: the C `EPR_DatasetId` structure has a field (`product_id`)
that points to the *product* descriptor `EPR_productId` to which it
belongs to.
+..index:: dataset, record
+
The reference to the parent product is used, for example, when one wants
to read a record using the `epr_read_record` function:
@@ -258,12 +275,12 @@ object instance::
dataset = product.get_dataset('MAIN_PROCESSING_PARAMS_ADS')
and then forget about the *product* instance it depends on.
-Even if the *product* variable goes out of scope and it is no more directly
-accessible in the program the *dataset* object keeps staying valid since
-it holds an internal reference to the *product* instance it depends on.
+Even if the *product* variable goes out of :index:`scope` and it is no more
+directly accessible in the program the *dataset* object keeps staying valid
+since it holds an internal reference to the *product* instance it depends on.
When *record* is destroyed automatically also the parent :class:`epr.Product`
-object is destroyed (assumed there is no other reference to it).
+object is destroyed (assumed there is no other :index:`reference` to it).
The entire machinery is completely automatic and transparent to the user.
@@ -273,6 +290,10 @@ The entire machinery is completely automatic and transparent to the user.
:meth:`epr.Product.close` any I/O operation on it and on other objects
(bands, datasets, etc) associated to it is no more possible.
+ .. index:: close
+
+
+.. index:: array, numpy, raster
Arrays
------
@@ -305,6 +326,11 @@ amount of data contained in ENVISAT_ products.
It read a slice of data from the :class:`epr.Band` and returns it as a
2D :class:`numpy.ndarray`.
+.. index:: read_as_aaray, data
+
+
+.. index:: enumeration
+ pair: module; constant
Enumerators
-----------
@@ -325,6 +351,8 @@ e_smid_log E_SMID_LOG
============ ============
+.. index:: logging, error, exception, EPR-API
+
Error handling and logging
--------------------------
@@ -334,9 +362,12 @@ exposed to python.
Internal library logging is completely silenced and errors are converted
to Python_ exceptions.
Where appropriate standard Python_ exception types are use in other cases
-custom exception types (e.g. :exc:`epr.EPRError`,
-:exc:`epr.EPRValueError`) are used.
+custom exception types (e.g. :exc:`epr.EPRError`, :exc:`epr.EPRValueError`)
+are used.
+
+.. index:: library, module, APR-API
+ pair: library; initialization
Library initialization
----------------------
@@ -346,6 +377,9 @@ performed internally the first time the :mod:`epr` module is imported
in Python_.
+.. index:: API
+ pair: high-level; API
+
High level API
--------------
@@ -373,6 +407,9 @@ Another example::
print(band)
+.. index:: __str__, __repr__, print_,
+ pair: special; methods
+
Special methods
---------------
@@ -399,6 +436,8 @@ giver the same result of::
Of course the :meth:`epr.Record.print_` method is more efficient for writing
to file.
+..index:: __iter__
+
Also :class:`epr.Dataset` and :class:`epr.Record` classes implement the
``__iter__`` `special method`_ for iterating over records and fields
respectively.
@@ -408,6 +447,8 @@ So it is possible to write code like the following::
for index, field in enumerate(record):
print(index, field)
+.. index:: __eq__
+
:class:`epr.DSD` and :class:`epr.Field` classes implement the ``__eq__``
and ``__ne__`` methods for objects comparison::
@@ -418,6 +459,8 @@ and ``__ne__`` methods for objects comparison::
print('field1:', field1)
print('field2:', field2)
+.. index:: __len__
+
:class:`epr.Field` object also implement the ``__len__`` special method
that returns the number of elements in the field::
@@ -434,6 +477,10 @@ that returns the number of elements in the field::
If the field contains a string then the string length is
returned.
+.. index:: __enter__, __exit__, context, with
+ pair: context; manager
+ pair: with; statement
+
Finally the :class:`epr.Product` class acts as a `context manager`_ (i.e. it
implements the ``__enter__`` and ``__exit__`` methods).
@@ -446,6 +493,9 @@ that ensure that the product is closed as soon as the program exits the
``with`` block.
+.. index:: update, ENVISAT
+ pair: open; mode
+
Update support
--------------
@@ -482,7 +532,11 @@ or also update all elements ol the :class:`epr.Field` in one shot::
For this reason the user may need to close and re-open the
:class:`epr.Product` in order to have all changes effectively applied.
- .. seealso:: :doc:`update_example`.
+ .. seealso:: :doc:`update_example`
+
+ .. index::
+ pair: scaling; factor
+
.. _`special method`: https://docs.python.org/3/reference/datamodel.html
.. _pythonic: http://www.cafepy.com/article/be_pythonic
diff --git a/src/epr.pyx b/src/epr.pyx
index 8db8d94..c57a129 100644
--- a/src/epr.pyx
+++ b/src/epr.pyx
@@ -41,7 +41,7 @@ in a product file.
'''
-__version__ = '0.9.1'
+__version__ = '0.9.2'
from libc cimport errno
from libc cimport stdio
@@ -953,7 +953,7 @@ cdef class Field(EprObject):
cdef int num_elems = 0
if type_ == e_tid_string:
- return '%s = "%s"' % (name, self.get_elem())
+ return '%s = "%s"' % (name, self.get_elem().decode('utf-8'))
elif type_ == e_tid_time:
days, seconds, microseconds = self.get_elem()
return '%s = {d=%d, j=%d, m=%d}' % (name,
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/pkg-grass/pyepr.git
More information about the Pkg-grass-devel
mailing list