[Git][debian-gis-team/fiona][upstream] New upstream version 1.9.3
Bas Couwenberg (@sebastic)
gitlab at salsa.debian.org
Tue Apr 11 04:44:27 BST 2023
Bas Couwenberg pushed to branch upstream at Debian GIS Project / fiona
Commits:
6f8de821 by Bas Couwenberg at 2023-04-11T05:25:54+02:00
New upstream version 1.9.3
- - - - -
12 changed files:
- CHANGES.txt
- README.rst
- docs/conf.py
- docs/index.rst
- docs/manual.rst
- fiona/__init__.py
- fiona/collection.py
- fiona/fio/load.py
- fiona/ogrext.pyx
- tests/test_fio_load.py
- tests/test_geojson.py
- tests/test_schema.py
Changes:
=====================================
CHANGES.txt
=====================================
@@ -3,6 +3,15 @@ Changes
All issue numbers are relative to https://github.com/Toblerity/Fiona/issues.
+1.9.3 (2023-04-10)
+------------------
+
+- Rasterio CRS objects are compatible with the Collection constructor and are
+ now accepted (#1248).
+- Enable append mode for fio-load (#1237).
+- Reading a GeoJSON with an empty array property can result in a segmentation
+ fault since version 1.9.0. This has been fixed (#1228).
+
1.9.2 (2023-03-20)
------------------
=====================================
README.rst
=====================================
@@ -2,8 +2,8 @@
Fiona
=====
-.. image:: https://github.com/Toblerity/Fiona/workflows/Linux%20CI/badge.svg?branch=master
- :target: https://github.com/Toblerity/Fiona/actions?query=branch%3Amaster
+.. image:: https://github.com/Toblerity/Fiona/workflows/Tests/badge.svg?branch=maint-1.9
+ :target: https://github.com/Toblerity/Fiona/actions?query=branch%3Amaint-1.9
Fiona streams simple feature data to and from GIS formats like GeoPackage and
Shapefile.
@@ -34,23 +34,28 @@ These wheels are mainly intended to make installation easy for simple
applications, not so much for production. They are not tested for compatibility
with all other binary wheels, conda packages, or QGIS, and omit many of GDAL's
optional format drivers. If you need, for example, GML support you will need to
-build and install Fiona from a source distribution.
+build and install Fiona from a source distribution. It is possible to install
+Fiona from source using pip and the `--no-binary` option. A specific GDAL
+installation can be selected by setting the GDAL_CONFIG environment variable.
+
+.. code-block:: console
+
+ pip install --no-binary fiona fiona
Many users find Anaconda and conda-forge a good way to install Fiona and get
access to more optional format drivers (like GML).
-Fiona 1.9 (coming soon) requires Python 3.7 or higher and GDAL 3.2 or higher.
+Fiona 1.9 requires Python 3.7 or higher and GDAL 3.2 or higher.
Python Usage
============
-Features are read from and written to file-like ``Collection`` objects
-returned from the ``fiona.open()`` function. Features are data classes modeled
-on the GeoJSON format. They don't have any spatial methods of their own, so if
-you want to do anything fancy with them you will need Shapely or something like
-it. Here is an example of using Fiona to read some features from one data file,
-change their geometry attributes using Shapely, and write them to a new data
-file.
+Features are read from and written to file-like ``Collection`` objects returned
+from the ``fiona.open()`` function. Features are data classes modeled on the
+GeoJSON format. They don't have any spatial methods of their own, so if you
+want to transform them you will need Shapely or something like it. Here is an
+example of using Fiona to read some features from one data file, change their
+geometry attributes using Shapely, and write them to a new data file.
.. code-block:: python
@@ -59,7 +64,9 @@ file.
from shapely.geometry import mapping, shape
# Open a file for reading. We'll call this the source.
- with fiona.open("tests/data/coutwildrnp.shp") as src:
+ with fiona.open(
+ "zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip"
+ ) as src:
# The file we'll write to must be initialized with a coordinate
# system, a format driver name, and a record schema. We can get
@@ -72,13 +79,13 @@ file.
# Open an output file, using the same format driver and coordinate
# reference system as the source. The profile mapping fills in the
# keyword parameters of fiona.open.
- with fiona.open("/tmp/example.gpkg", "w", **profile) as dst:
+ with fiona.open("centroids.gpkg", "w", **profile) as dst:
- # Process only the records intersecting a box.
- for f in src.filter(bbox=(-107.0, 37.0, -105.0, 39.0)):
+ # Process only the feature records intersecting a box.
+ for feat in src.filter(bbox=(-107.0, 37.0, -105.0, 39.0)):
# Get the feature's centroid.
- centroid_shp = shape(f.geometry).centroid
+ centroid_shp = shape(feat.geometry).centroid
new_geom = Geometry.from_dict(centroid_shp)
# Write the feature out.
=====================================
docs/conf.py
=====================================
@@ -24,7 +24,14 @@ import sys, os
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = ['sphinx.ext.autodoc']
+extensions = [
+ 'sphinx.ext.autodoc',
+ 'sphinx.ext.doctest',
+ 'sphinx.ext.intersphinx',
+ 'sphinx.ext.napoleon',
+ 'sphinx.ext.todo',
+ 'sphinx_click',
+]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
@@ -97,7 +104,8 @@ pygments_style = 'sphinx'
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#html_theme = 'default'
-html_theme = 'sphinxdoc'
+#html_theme = 'sphinxdoc'
+html_theme = 'sphinx_rtd_theme'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
=====================================
docs/index.rst
=====================================
@@ -1,10 +1,42 @@
-Fiona Documentation Contents
-============================
+===============================================
+Fiona: access to simple geospatial feature data
+===============================================
+
+Fiona streams simple feature data to and from GIS formats like GeoPackage and
+Shapefile. Simple features are record, or row-like, and have a single geometry
+attribute. Fiona can read and write real-world simple feature data using
+multi-layered GIS formats, zipped and in-memory virtual file systems, from
+files on your hard drive or in cloud storage. This project includes Python
+modules and a command line interface (CLI).
+
+Here's an example of streaming and filtering features from a zipped dataset on
+the web and saving them to a new layer in a new Geopackage file.
+
+.. code-block:: python
+
+ import fiona
+
+ with fiona.open(
+ "zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip"
+ ) as src:
+ profile = src.profile
+ profile["driver"] = "GPKG"
+
+ with fiona.open("example.gpkg", "w", layer="selection", **profile) as dst:
+ dst.writerecords(feat in src.filter(bbox=(-107.0, 37.0, -105.0, 39.0)))
+
+The same result can be achieved on the command line using a combination of
+fio-cat and fio-load.
+
+.. code-block:: console
+
+ fio cat zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip --bbox "-107.0,37.0,-105.0,39.0" \
+ | fio load -f GPKG --layer selection example.gpkg
.. toctree::
:maxdepth: 2
- README
+ Project Information <README>
Installation <install>
User Manual <manual>
API Documentation <modules>
=====================================
docs/manual.rst
=====================================
@@ -95,74 +95,69 @@ record) from one file to another, adding two attributes and making sure that
all polygons are facing "up". Orientation of polygons is significant in some
applications, extruded polygons in Google Earth for one. No other library (like
:py:mod:`Shapely`) is needed here, which keeps it uncomplicated. There's a
-:file:`test_uk` file in the Fiona repository for use in this and other
+:file:`coutwildrnp.zip` file in the Fiona repository for use in this and other
examples.
.. code-block:: python
- import datetime
- import logging
- import sys
+ import datetime
- import fiona
-
- logging.basicConfig(stream=sys.stderr, level=logging.INFO)
-
-
- def signed_area(coords):
- """Return the signed area enclosed by a ring using the linear time
- algorithm at http://www.cgafaq.info/wiki/Polygon_Area. A value >= 0
- indicates a counter-clockwise oriented ring.
- """
- xs, ys = map(list, zip(*coords))
- xs.append(xs[1])
- ys.append(ys[1])
- return sum(xs[i] * (ys[i + 1] - ys[i - 1]) for i in range(1, len(coords))) / 2.0
-
-
- with fiona.open("docs/data/test_uk.shp", "r") as source:
-
- # Copy the source schema and add two new properties.
- sink_schema = source.schema
- sink_schema["properties"]["s_area"] = "float"
- sink_schema["properties"]["timestamp"] = "datetime"
-
- # Create a sink for processed features with the same format and
- # coordinate reference system as the source.
- with fiona.open(
- "oriented-ccw.shp",
- "w",
- crs=source.crs,
- driver=source.driver,
- schema=sink_schema,
- ) as sink:
- for f in source:
- try:
- # If any feature's polygon is facing "down" (has rings
- # wound clockwise), its rings will be reordered to flip
- # it "up".
- g = f["geometry"]
- assert g["type"] == "Polygon"
- rings = g["coordinates"]
- sa = sum(signed_area(r) for r in rings)
-
- if sa < 0.0:
- rings = [r[::-1] for r in rings]
- g["coordinates"] = rings
- f["geometry"] = g
-
- # Add the signed area of the polygon and a timestamp
- # to the feature properties map.
- f["properties"].update(
- s_area=sa, timestamp=datetime.datetime.now().isoformat()
- )
-
- sink.write(f)
-
- except Exception as e:
- logging.exception("Error processing feature %s:", f["id"])
-
- # The sink file is written to disk and closed when its block ends.
+ import fiona
+ from fiona import Geometry, Feature, Properties
+
+
+ def signed_area(coords):
+ """Return the signed area enclosed by a ring using the linear time
+ algorithm at http://www.cgafaq.info/wiki/Polygon_Area. A value >= 0
+ indicates a counter-clockwise oriented ring.
+ """
+ xs, ys = map(list, zip(*coords))
+ xs.append(xs[1])
+ ys.append(ys[1])
+ return sum(xs[i] * (ys[i + 1] - ys[i - 1]) for i in range(1, len(coords))) / 2.0
+
+
+ with fiona.open(
+ "zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip"
+ ) as src:
+
+ # Copy the source schema and add two new properties.
+ dst_schema = src.schema
+ dst_schema["properties"]["signed_area"] = "float"
+ dst_schema["properties"]["timestamp"] = "datetime"
+
+ # Create a sink for processed features with the same format and
+ # coordinate reference system as the source.
+ with fiona.open(
+ "example.gpkg",
+ mode="w",
+ layer="oriented-ccw",
+ crs=src.crs,
+ driver="GPKG",
+ schema=dst_schema,
+ ) as dst:
+ for feat in src:
+ # If any feature's polygon is facing "down" (has rings
+ # wound clockwise), its rings will be reordered to flip
+ # it "up".
+ geom = feat.geometry
+ assert geom.type == "Polygon"
+ rings = geom.coordinates
+ sa = sum(signed_area(ring) for ring in rings)
+
+ if sa < 0.0:
+ rings = [r[::-1] for r in rings]
+ geom = Geometry(type=geom.type, coordinates=rings)
+
+ # Add the signed area of the polygon and a timestamp
+ # to the feature properties map.
+ props = Properties.from_dict(
+ **feat.properties,
+ signed_area=sa,
+ timestamp=datetime.datetime.now().isoformat()
+ )
+
+ dst.write(Feature(geometry=geom, properties=props))
Data Model
==========
@@ -216,17 +211,12 @@ Reading a GIS vector file begins by opening it in mode ``'r'`` using Fiona's
.. code-block:: pycon
- >>> import fiona
- >>> c = fiona.open('docs/data/test_uk.shp', 'r')
- >>> c
- <open Collection 'docs/data/test_uk.shp:test_uk', mode 'r' at 0x...>
- >>> c.closed
- False
-
-.. admonition:: API Change
-
- :py:func:`fiona.collection` is deprecated, but aliased to
- :py:func:`fiona.open` in version 0.9.
+ >>> import fiona
+ >>> colxn = fiona.open("zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip", "r")
+ >>> colxn
+ <open Collection '/vsizip/vsicurl/https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip:coutwildrnp', mode 'r' at 0x7f9555af8f50>
+ >>> collection.closed
+ False
Mode ``'r'`` is the default and will be omitted in following examples.
@@ -235,31 +225,31 @@ Fiona's :py:class:`~fiona.collection.Collection` is like a Python
.. code-block:: pycon
- >>> next(c)
- {'geometry': {'type': 'Polygon', 'coordinates': ...
- >>> len(list(c))
- 48
+ >>> next(iter(colxn))
+ {'geometry': {'type': 'Polygon', 'coordinates': ...
+ >>> len(list(colxn))
+ 67
Note that :py:func:`list` iterates over the entire collection, effectively
emptying it as with a Python :py:class:`file`.
.. code-block:: pycon
- >>> next(c)
- Traceback (most recent call last):
- ...
- StopIteration
- >>> len(list(c))
- 0
+ >>> next(iter(colxn))
+ Traceback (most recent call last):
+ ...
+ StopIteration
+ >>> len(list(colxn))
+ 0
Seeking the beginning of the file is not supported. You must reopen the
collection to get back to the beginning.
.. code-block:: pycon
- >>> c = fiona.open('docs/data/test_uk.shp')
- >>> len(list(c))
- 48
+ >>> colxn = fiona.open("zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip")
+ >>> len(list(colxn))
+ 67
.. admonition:: File Encoding
@@ -278,29 +268,15 @@ Features of a collection may also be accessed by index.
.. code-block:: pycon
- >>> import pprint
- >>> with fiona.open('docs/data/test_uk.shp') as src:
- ... pprint.pprint(src[1])
+ >>> with fiona.open("zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip") as colxn:
+ ... print(colxn[1])
...
- {'geometry': {'coordinates': [[(-4.663611, 51.158333),
- (-4.669168, 51.159439),
- (-4.673334, 51.161385),
- (-4.674445, 51.165276),
- (-4.67139, 51.185272),
- (-4.669445, 51.193054),
- (-4.665556, 51.195),
- (-4.65889, 51.195),
- (-4.656389, 51.192215),
- (-4.646389, 51.164444),
- (-4.646945, 51.160828),
- (-4.651668, 51.159439),
- (-4.663611, 51.158333)]],
- 'type': 'Polygon'},
- 'id': '1',
- 'properties': OrderedDict([('CAT', 232.0), ('FIPS_CNTRY', 'UK'), ('CNTRY_NAME', 'United Kingdom'), ('AREA', 244820.0), ('POP_CNTRY', 60270708.0)]),
- 'type': 'Feature'}
-
-Note that these indices are controlled by GDAL, and do not always follow Python conventions. They can start from 0, 1 (e.g. geopackages), or even other values, and have no guarantee of contiguity. Negative indices will only function correctly if indices start from 0 and are contiguous.
+ <fiona.model.Feature object at 0x7f954bfc5f50>
+
+Note that these indices are controlled by GDAL, and do not always follow Python
+conventions. They can start from 0, 1 (e.g. geopackages), or even other values,
+and have no guarantee of contiguity. Negative indices will only function
+correctly if indices start from 0 and are contiguous.
New in version 1.1.6
@@ -315,23 +291,23 @@ is a context guard, it is closed no matter what happens within the block.
.. code-block:: pycon
- >>> try:
- ... with fiona.open('docs/data/test_uk.shp') as c:
- ... print(len(list(c)))
- ... assert True is False
- ... except:
- ... print(c.closed)
- ... raise
- ...
- 48
- True
- Traceback (most recent call last):
+ >>> try:
+ ... with fiona.open("zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip") as colxn:
+ ... print(len(list(colxn)))
+ ... assert True is False
+ ... except Exception:
+ ... print(colxn.closed)
+ ... raise
...
- AssertionError
+ 67
+ True
+ Traceback (most recent call last):
+ ...
+ AssertionError
An exception is raised in the :keyword:`with` block above, but as you can see
-from the print statement in the :keyword:`except` clause :py:meth:`c.__exit__`
-(and thereby :py:meth:`c.close`) has been called.
+from the print statement in the :keyword:`except` clause :py:meth:`colxn.__exit__`
+(and thereby :py:meth:`colxn.close`) has been called.
.. important:: Always call :py:meth:`~fiona.collection.Collection.close` or
use :keyword:`with` and you'll never stumble over tied-up external resources,
@@ -348,44 +324,42 @@ a :py:class:`~fiona.collection.Collection` has a read-only
.. code-block:: pycon
- >>> c = fiona.open('docs/data/test_uk.shp')
- >>> c.driver
- 'ESRI Shapefile'
+ >>> colxn = fiona.open("zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip")
+ >>> colxn.driver
+ 'ESRI Shapefile'
The :dfn:`coordinate reference system` (CRS) of the collection's vector data is
accessed via a read-only :py:attr:`~fiona.collection.Collection.crs` attribute.
.. code-block:: pycon
- >>> c.crs
- {'no_defs': True, 'ellps': 'WGS84', 'datum': 'WGS84', 'proj': 'longlat'}
-
-The CRS is represented by a mapping of :program:`PROJ.4` parameters.
+ >>> colxn.crs
+ CRS.from_epsg(4326)
The :py:mod:`fiona.crs` module provides 3 functions to assist with these
mappings. :py:func:`~fiona.crs.to_string` converts mappings to PROJ.4 strings:
.. code-block:: pycon
- >>> from fiona.crs import to_string
- >>> print(to_string(c.crs))
- +datum=WGS84 +ellps=WGS84 +no_defs +proj=longlat
+ >>> from fiona.crs import to_string
+ >>> to_string(colxn.crs)
+ 'EPSG:4326'
:py:func:`~fiona.crs.from_string` does the inverse.
.. code-block:: pycon
- >>> from fiona.crs import from_string
- >>> from_string("+datum=WGS84 +ellps=WGS84 +no_defs +proj=longlat")
- {'no_defs': True, 'ellps': 'WGS84', 'datum': 'WGS84', 'proj': 'longlat'}
+ >>> from fiona.crs import from_string
+ >>> from_string("+datum=WGS84 +ellps=WGS84 +no_defs +proj=longlat")
+ CRS.from_epsg(4326)
:py:func:`~fiona.crs.from_epsg` is a shortcut to CRS mappings from EPSG codes.
.. code-block:: pycon
- >>> from fiona.crs import from_epsg
- >>> from_epsg(3857)
- {'init': 'epsg:3857', 'no_defs': True}
+ >>> from fiona.crs import from_epsg
+ >>> from_epsg(3857)
+ CRS.from_epsg(3857)
.. admonition:: No Validation
@@ -398,8 +372,8 @@ built in :py:func:`len` function.
.. code-block:: pycon
- >>> len(c)
- 48
+ >>> len(colxn)
+ 67
The :dfn:`minimum bounding rectangle` (MBR) or :dfn:`bounds` of the
collection's records is obtained via a read-only
@@ -407,8 +381,8 @@ collection's records is obtained via a read-only
.. code-block:: pycon
- >>> c.bounds
- (-8.621389, 49.911659, 1.749444, 60.844444)
+ >>> colxn.bounds
+ (-113.56424713134766, 37.0689811706543, -104.97087097167969, 41.99627685546875)
Finally, the schema of its record type (a vector file has a single type of
record, remember) is accessed via a read-only
@@ -418,14 +392,19 @@ dict with items having the same order as the fields in the data file.
.. code-block:: pycon
- >>> import pprint
- >>> pprint.pprint(c.schema)
- {'geometry': 'Polygon',
- 'properties': {'CAT': 'float:16',
- 'FIPS_CNTRY': 'str',
- 'CNTRY_NAME': 'str',
- 'AREA': 'float:15.2',
- 'POP_CNTRY': 'float:15.2'}}
+ >>> import pprint
+ >>> pprint.pprint(colxn.schema)
+ {'geometry': 'Polygon',
+ 'properties': {'AGBUR': 'str:80',
+ 'AREA': 'float:24.15',
+ 'FEATURE1': 'str:80',
+ 'FEATURE2': 'str:80',
+ 'NAME': 'str:80',
+ 'PERIMETER': 'float:24.15',
+ 'STATE': 'str:80',
+ 'STATE_FIPS': 'str:80',
+ 'URL': 'str:101',
+ 'WILDRNP020': 'int:10'}}
Keeping Schemas Simple
----------------------
@@ -437,11 +416,11 @@ keys of the collection's record mappings.
.. code-block:: pycon
- >>> rec = next(c)
- >>> set(rec.keys()) - set(c.schema.keys())
- {'id'}
- >>> set(rec['properties'].keys()) == set(c.schema['properties'].keys())
- True
+ >>> feat = next(iter(colxn))
+ >>> set(feat.keys()) - set(colxn.schema.keys())
+ {'id'}
+ >>> set(feat['properties'].keys()) == set(colxn.schema['properties'].keys())
+ True
The values of the schema mapping are either additional mappings or field type
names like 'Polygon', 'float', and 'str'. The corresponding Python types can
@@ -449,17 +428,17 @@ be found in a dictionary named :py:attr:`fiona.FIELD_TYPES_MAP`.
.. code-block:: pycon
- >>> pprint.pprint(fiona.FIELD_TYPES_MAP)
- {'List[str]': typing.List[str],
- 'bytes': <class 'bytes'>,
- 'date': <class 'fiona.rfc3339.FionaDateType'>,
- 'datetime': <class 'fiona.rfc3339.FionaDateTimeType'>,
- 'float': <class 'float'>,
- 'int': <class 'int'>,
- 'int32': <class 'int'>,
- 'int64': <class 'int'>,
- 'str': <class 'str'>,
- 'time': <class 'fiona.rfc3339.FionaTimeType'>}
+ >>> pprint.pprint(fiona.FIELD_TYPES_MAP)
+ {'List[str]': typing.List[str],
+ 'bytes': <class 'bytes'>,
+ 'date': <class 'fiona.rfc3339.FionaDateType'>,
+ 'datetime': <class 'fiona.rfc3339.FionaDateTimeType'>,
+ 'float': <class 'float'>,
+ 'int': <class 'int'>,
+ 'int32': <class 'int'>,
+ 'int64': <class 'int'>,
+ 'str': <class 'str'>,
+ 'time': <class 'fiona.rfc3339.FionaTimeType'>}
Field Types
-----------
@@ -470,11 +449,11 @@ may contain Unicode characters.
.. code-block:: pycon
- >>> type(rec['properties']['CNTRY_NAME'])
+ >>> type(feat.properties['NAME'])
<class 'str'>
- >>> c.schema['properties']['CNTRY_NAME']
+ >>> colxn.schema['properties']['NAME']
'str'
- >>> fiona.FIELD_TYPES_MAP[c.schema['properties']['CNTRY_NAME']]
+ >>> fiona.FIELD_TYPES_MAP[colxn.schema['properties']['NAME']]
<class 'str'>
String type fields may also indicate their maximum width. A value of 'str:25'
@@ -536,120 +515,83 @@ Note that one of the most common vector data formats, Esri's Shapefile, has no
that indicates 'Polygon' in its schema may yield either 'Polygon' or
'MultiPolygon' features.
-Records
-=======
+Features
+========
-A record you get from a collection is a Python :py:class:`dict` structured
-exactly like a GeoJSON Feature. Fiona records are self-describing; the names of
-its fields are contained within the data structure and the values in the fields
-are typed properly for the type of record. Numeric field values are instances
-of type :py:class:`int` and :py:class:`float`, for example, not strings.
-
-.. code-block:: pycon
-
- >>> pprint.pprint(rec)
- {'geometry': {'coordinates': [[(-4.663611, 51.158333),
- (-4.669168, 51.159439),
- (-4.673334, 51.161385),
- (-4.674445, 51.165276),
- (-4.67139, 51.185272),
- (-4.669445, 51.193054),
- (-4.665556, 51.195),
- (-4.65889, 51.195),
- (-4.656389, 51.192215),
- (-4.646389, 51.164444),
- (-4.646945, 51.160828),
- (-4.651668, 51.159439),
- (-4.663611, 51.158333)]],
- 'type': 'Polygon'},
- 'id': '1',
- 'properties': {'CAT': 232.0,
- 'FIPS_CNTRY': 'UK',
- 'CNTRY_NAME': 'United Kingdom',
- 'AREA': 244820.0,
- 'POP_CNTRY': 60270708.0}}
+A record you get from a collection is structured like a GeoJSON Feature. Fiona
+records are self-describing; the names of its fields are contained within the
+data structure and the values in the fields are typed properly for the type of
+record. Numeric field values are instances of type :py:class:`int` and
+:py:class:`float`, for example, not strings.
The record data has no references to the
:py:class:`~fiona.collection.Collection` from which it originates or to any
other external resource. It's entirely independent and safe to use in any way.
Closing the collection does not affect the record at all.
-.. code-block:: pycon
+.. admonition:: Features are mappings, not dicts
+
+ In Fiona versions before 1.9.0 features were Python dicts, mutable and JSON
+ serializable. Since 1.9.0 features are mappings and not immediately JSON
+ serializable.
- >>> c.close()
- >>> rec['id']
- '1'
+ Instances of Feature can be converted to dicts with
+ :py:func:`fiona.model.to_dict` or serialized using the json module and
+ :py:class:`fiona.model.ObjectEncoder`.
-Record Id
----------
+Feature Id
+----------
-A record has an ``id`` key. As in the GeoJSON specification, its corresponding
-value is a string unique within the data file.
+A feature has an ``id`` attribute. As in the GeoJSON specification, its
+corresponding value is a string unique within the data file.
.. code-block:: pycon
- >>> c = fiona.open('docs/data/test_uk.shp')
- >>> rec = next(c)
- >>> rec['id']
- '0'
+ >>> colxn = fiona.open("zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip")
+ >>> feat = next(iter(colxn))
+ >>> feat.id
+ '0'
.. admonition:: OGR Details
In the :program:`OGR` model, feature ids are long integers. Fiona record ids
are therefore usually string representations of integer record indexes.
-Record Properties
------------------
+Feature Properties
+------------------
-A record has a ``properties`` key. Its corresponding value is a mapping: an
-ordered dict to be precise. The keys of the properties mapping are the same as
-the keys of the properties mapping in the schema of the collection the record
-comes from (see above).
+A feature has a ``properties`` attribute. Its value is a mapping. The keys of
+the properties mapping are the same as the keys of the properties mapping in
+the schema of the collection the record comes from (see above).
.. code-block:: pycon
- >>> pprint.pprint(rec['properties'])
- {'CAT': 232.0,
- 'FIPS_CNTRY': 'UK',
- 'CNTRY_NAME': 'United Kingdom',
- 'AREA': 244820.0,
- 'POP_CNTRY': 60270708.0}
-
-Record Geometry
----------------
-
-A record has a ``geometry`` key. Its corresponding value is a mapping with
-``type`` and ``coordinates`` keys.
+ >>> for k, v in feat.properties.items():
+ ... print(k, v)
+ ...
+ PERIMETER 1.22107
+ FEATURE2 None
+ NAME Mount Naomi Wilderness
+ FEATURE1 Wilderness
+ URL http://www.wilderness.net/index.cfm?fuse=NWPS&sec=wildView&wname=Mount%20Naomi
+ AGBUR FS
+ AREA 0.0179264
+ STATE_FIPS 49
+ WILDRNP020 332
+ STATE UT
+
+Feature Geometry
+----------------
+
+A feature has a ``geometry`` attribute. Its value is a mapping with ``type``
+and ``coordinates`` keys.
.. code-block:: pycon
- >>> pprint.pprint(rec['geometry'])
- {'coordinates': [[(0.899167, 51.357216),
- (0.885278, 51.35833),
- (0.7875, 51.369438),
- (0.781111, 51.370552),
- (0.766111, 51.375832),
- (0.759444, 51.380829),
- (0.745278, 51.39444),
- (0.740833, 51.400276),
- (0.735, 51.408333),
- (0.740556, 51.429718),
- (0.748889, 51.443604),
- (0.760278, 51.444717),
- (0.791111, 51.439995),
- (0.892222, 51.421387),
- (0.904167, 51.418884),
- (0.908889, 51.416939),
- (0.930555, 51.398888),
- (0.936667, 51.393608),
- (0.943889, 51.384995),
- (0.9475, 51.378609),
- (0.947778, 51.374718),
- (0.946944, 51.371109),
- (0.9425, 51.369164),
- (0.904722, 51.358055),
- (0.899167, 51.357216)]],
- 'type': 'Polygon'}
+ >>> feat.geometry["type"]
+ 'Polygon'
+ >>> feat.geometry["coordinates"]
+ [[(-111.73527526855469, 41.995094299316406), ..., (-111.73527526855469, 41.995094299316406)]]
Since the coordinates are just tuples, or lists of tuples, or lists of lists of
tuples, the ``type`` tells you how to interpret them.
@@ -735,46 +677,44 @@ Appending Data to Existing Files
--------------------------------
Let's start with the simplest if not most common use case, adding new records
-to an existing file. The file is copied before modification and a suitable
-record extracted in the example below.
+to an existing file.
-.. code-block:: pycon
+.. code-block:: console
- >>> with fiona.open('docs/data/test_uk.shp') as c:
- ... rec = next(c)
- >>> rec['id'] = '-1'
- >>> rec['properties']['CNTRY_NAME'] = 'Gondor'
- >>> import os
- >>> os.system("cp docs/data/test_uk.* /tmp")
- 0
+ $ wget https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip
+ $ unzip coutwildrnp.zip
The coordinate reference system. format, and schema of the file are already
defined, so it's opened with just two arguments as for reading, but in ``'a'``
mode. The new record is written to the end of the file using the
:py:meth:`~fiona.collection.Collection.write` method. Accordingly, the length
-of the file grows from 48 to 49.
+of the file grows from 67 to 68.
.. code-block:: pycon
- >>> with fiona.open('/tmp/test_uk.shp', 'a') as c:
- ... print(len(c))
- ... c.write(rec)
- ... print(len(c))
- ...
- 48
- 49
+ >>> with fiona.open("coutwildrnp.shp", "a") as dst:
+ ... print(len(dst))
+ ... with fiona.open("zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip") as src:
+ ... feat = src[0]
+ ... print(feat.id, feat.properties["NAME"])
+ ... dst.write(feat)
+ ... print(len(c))
+ ...
+ 67
+ ('0', 'Mount Naomi Wilderness')
+ 68
-The record you write must match the file's schema (because a file contains one
+The feature you write must match the file's schema (because a file contains one
type of record, remember). You'll get a :py:class:`ValueError` if it doesn't.
.. code-block:: pycon
- >>> with fiona.open('/tmp/test_uk.shp', 'a') as c:
- ... c.write({'properties': {'foo': 'bar'}})
- ...
- Traceback (most recent call last):
+ >>> with fiona.open("coutwildrnp.shp", "a") as dst:
+ ... dst.write({'properties': {'foo': 'bar'}})
...
- ValueError: Record data not match collection schema
+ Traceback (most recent call last):
+ ...
+ ValueError: Record data not match collection schema
Now, what about record ids? The id of a record written to a file is ignored and
replaced by the next value appropriate for the file. If you read the file just
@@ -782,15 +722,16 @@ appended to above,
.. code-block:: pycon
- >>> with fiona.open('/tmp/test_uk.shp', 'a') as c:
- ... records = list(c)
- >>> records[-1]['id']
- '48'
- >>> records[-1]['properties']['CNTRY_NAME']
- 'Gondor'
+ >>> with fiona.open("coutwildrnp.shp") as colxn:
+ ... feat = colxn[-1]
+ ...
+ >>> feat.id
+ '67'
+ >>> feat.properties["NAME"]
+ 'Mount Naomi Wilderness'
-You'll see that the id of ``'-1'`` which the record had when written is
-replaced by ``'48'``.
+You'll see that the id of ``'0'`` which the record had when written is replaced
+by ``'67'``.
The :py:meth:`~fiona.collection.Collection.write` method writes a single
record to the collection's file. Its sibling
@@ -799,11 +740,11 @@ iterator) of records.
.. code-block:: pycon
- >>> with fiona.open('/tmp/test_uk.shp', 'a') as c:
- ... c.writerecords([rec, rec, rec])
- ... print(len(c))
- ...
- 52
+ >>> with fiona.open("coutwildrnp.shp", "a") as colxn:
+ ... colxn.writerecords([feat, feat, feat])
+ ... print(len(colxn))
+ ...
+ 71
.. admonition:: Duplication
@@ -843,164 +784,139 @@ Creating files of the same structure
Writing a new file is more complex than appending to an existing file because
the file CRS, format, and schema have not yet been defined and must be done so
by the programmer. Still, it's not very complicated. A schema is just
-a mapping, as described above. A CRS is also just a mapping, and the possible
+a mapping, as described above. The possible
formats are enumerated in the :py:attr:`fiona.supported_drivers` dictionary.
Review the parameters of our demo file.
.. code-block:: pycon
- >>> with fiona.open('docs/data/test_uk.shp') as source:
- ... source_driver = source.driver
- ... source_crs = source.crs
- ... source_schema = source.schema
- ...
- >>> source_driver
- 'ESRI Shapefile'
- >>> source_crs
- {'no_defs': True, 'ellps': 'WGS84', 'datum': 'WGS84', 'proj': 'longlat'}
- >>> pprint.pprint(source_schema)
- {'geometry': 'Polygon',
- 'properties': {'CAT': 'float:16',
- 'FIPS_CNTRY': 'str',
- 'CNTRY_NAME': 'str',
- 'AREA': 'float:15.2',
- 'POP_CNTRY': 'float:15.2'}}
+ >>> with fiona.open(
+ ... "zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip"
+ ) as src:
+ ... driver = src.driver
+ ... crs = src.crs
+ ... schema = src.schema
+ ... feat = src[1]
+ ...
+ >>> driver
+ 'ESRI Shapefile'
+ >>> crs
+ CRS.from_epsg(4326)
+ >>> pprint.pprint(schema)
+ {'geometry': 'Polygon',
+ 'properties': {'AGBUR': 'str:80',
+ 'AREA': 'float:24.15',
+ 'FEATURE1': 'str:80',
+ 'FEATURE2': 'str:80',
+ 'NAME': 'str:80',
+ 'PERIMETER': 'float:24.15',
+ 'STATE': 'str:80',
+ 'STATE_FIPS': 'str:80',
+ 'URL': 'str:101',
+ 'WILDRNP020': 'int:10'}}
We can create a new file using them.
.. code-block:: pycon
- >>> with fiona.open(
- ... '/tmp/foo.shp',
- ... 'w',
- ... driver=source_driver,
- ... crs=source_crs,
- ... schema=source_schema) as c:
- ... print(len(c))
- ... c.write(rec)
- ... print(len(c))
- ...
- 0
- 1
- >>> c.closed
- True
- >>> len(c)
- 1
+ >>> with fiona.open("example.shp", "w", driver=driver, crs=crs, schema=schema) as dst:
+ ... print(len(dst))
+ ... dst.write(feat)
+ ... print(len(dst))
+ ...
+ 0
+ 1
+ >>> dst.closed
+ True
+ >>> len(dst)
+ 1
Because the properties of the source schema are ordered and are passed in the
same order to the write-mode collection, the written file's fields have the
same order as those of the source file.
-.. code-block:: console
-
- $ ogrinfo /tmp/foo.shp foo -so
- INFO: Open of `/tmp/foo.shp'
- using driver `ESRI Shapefile' successful.
-
- Layer name: foo
- Geometry: 3D Polygon
- Feature Count: 1
- Extent: (0.735000, 51.357216) - (0.947778, 51.444717)
- Layer SRS WKT:
- GEOGCS["GCS_WGS_1984",
- DATUM["WGS_1984",
- SPHEROID["WGS_84",6378137,298.257223563]],
- PRIMEM["Greenwich",0],
- UNIT["Degree",0.017453292519943295]]
- CAT: Real (16.0)
- FIPS_CNTRY: String (80.0)
- CNTRY_NAME: String (80.0)
- AREA: Real (15.2)
- POP_CNTRY: Real (15.2)
-
-The :py:attr:`~fiona.collection.Collection.meta` attribute makes duplication of
+The :py:attr:`~fiona.collection.Collection.profile` attribute makes duplication of
a file's meta properties even easier.
.. code-block:: pycon
- >>> source = fiona.open('docs/data/test_uk.shp')
- >>> sink = fiona.open('/tmp/foo.shp', 'w', **source.meta)
+ >>> src = fiona.open("zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip")
+ >>> dst = fiona.open("example.shp", "w", **src.profile)
Writing new files from scratch
-------------------------------
-To write a new file from scratch we have to define our own specific driver, crs and schema.
+To write a new file from scratch we have to define our own specific driver, crs
+and schema.
-To ensure the order of the attribute fields is predictable, in both the schema and the actual manifestation as feature attributes, we will use ordered dictionaries.
+To ensure the order of the attribute fields is predictable, in both the schema
+and the actual manifestation as feature attributes, we will use ordered
+dictionaries.
.. code-block:: pycon
- >>> from collections import OrderedDict
-
-Consider the following record, structured in accordance to the `Python geo protocol <https://gist.github.com/sgillies/2217756>`__, representing the Eiffel Tower using a point geometry with UTM coordinates in zone 31N.
+Consider the following record, structured in accordance to the `Python geo
+protocol <https://gist.github.com/sgillies/2217756>`__, representing the Eiffel
+Tower using a point geometry with UTM coordinates in zone 31N.
.. code-block:: pycon
- >>> eiffel_tower = {
- ... 'geometry': {
- ... 'type': 'Point',
- ... 'coordinates': (448252, 5411935)
- ... },
- ... 'properties': OrderedDict([
- ... ('name', 'Eiffel Tower'),
- ... ('height', 300.01),
- ... ('view', 'scenic'),
- ... ('year', 1889)
- ... ])
- ... }
+ >>> eiffel_tower = {
+ ... 'geometry': {
+ ... 'type': 'Point',
+ ... 'coordinates': (448252, 5411935)
+ ... },
+ ... 'properties': dict([
+ ... ('name', 'Eiffel Tower'),
+ ... ('height', 300.01),
+ ... ('view', 'scenic'),
+ ... ('year', 1889)
+ ... ])
+ ... }
A corresponding scheme could be:
.. code-block:: pycon
- >>> landmarks_schema = {
- ... 'geometry': 'Point',
- ... 'properties': OrderedDict([
- ... ('name', 'str'),
- ... ('height', 'float'),
- ... ('view', 'str'),
- ... ('year', 'int')
- ... ])
- ... }
+ >>> landmarks_schema = {
+ ... 'geometry': 'Point',
+ ... 'properties': dict([
+ ... ('name', 'str'),
+ ... ('height', 'float'),
+ ... ('view', 'str'),
+ ... ('year', 'int')
+ ... ])
+ ... }
-The coordinate reference system of these landmark coordinates is ETRS89 / UTM zone 31N which is referenced in the EPSG database as EPSG:25831.
+The coordinate reference system of these landmark coordinates is ETRS89 / UTM
+zone 31N which is referenced in the EPSG database as EPSG:25831.
.. code-block:: pycon
- >>> from fiona.crs import from_epsg
- >>> landmarks_crs = from_epsg(25831)
+ >>> from fiona.crs import CRS
+ >>> landmarks_crs = CRS.from_epsg(25831)
An appropriate driver could be:
.. code-block:: pycon
- >>> output_driver = "GeoJSON"
+ >>> driver = "GeoJSON"
-Having specified schema, crs and driver, we are ready to open a file for writing our record:
+Having specified schema, crs and driver, we are ready to open a file for
+writing our record:
.. code-block:: pycon
- >>> with fiona.open(
- ... '/tmp/foo.geojson',
- ... 'w',
- ... driver=output_driver,
- ... crs=landmarks_crs,
- ... schema=landmarks_schema) as c:
- ... c.write(eiffel_tower)
- ...
-
- >>> import pprint
- >>> with fiona.open('/tmp/foo.geojson') as source:
- ... for record in source:
- ... pprint.pprint(record)
- {'geometry': {'coordinates': (448252.0, 5411935.0), 'type': 'Point'},
- 'id': '0',
- 'properties': OrderedDict([('name', 'Eiffel Tower'),
- ('height', 300.01),
- ('view', 'scenic'),
- ('year', 1889)]),
- 'type': 'Feature'}
+ >>> with fiona.open(
+ ... "landmarks.geojson",
+ ... "w",
+ ... driver="GeoJSON",
+ ... crs=CRS.from_epsg(25831),
+ ... schema=landmarks_schema
+ ... ) as colxn:
+ ... colxn.write(eiffel_tower)
+ ...
Ordering Record Fields
......................
@@ -1011,90 +927,46 @@ pairs, specifying an ordering that carries into written files. If an ordinary
dict is given, the ordering is determined by the output of that dict's
:py:func:`~items` method.
-For example, since
-
-.. code-block:: pycon
-
- >>> {'bar': 'int', 'foo': 'str'}.keys()
- ['foo', 'bar']
-
-a schema of ``{'properties': {'bar': 'int', 'foo': 'str'}}`` will produce
-a shapefile where the first field is 'foo' and the second field is 'bar'. If
-you want 'bar' to be the first field, you must use a list of property items
-
-.. code-block:: python
-
- fiona.open(
- "/tmp/file.shp",
- "w",
- schema={"properties": [("bar", "int"), ("foo", "str")]},
- **kwargs
- )
-
-or an ordered dict.
-
-.. code-block:: python
-
- from collections import OrderedDict
-
- schema_props = OrderedDict([("bar", "int"), ("foo", "str")])
- fiona.open(
- "/tmp/file.shp",
- "w",
- schema={"properties": schema_props},
- **kwargs
- )
-
3D Coordinates and Geometry Types
---------------------------------
If you write 3D coordinates, ones having (x, y, z) tuples, to a 2D file
('Point' schema geometry, for example) the z values will be lost.
-.. sourcecode:: python
-
- schema_props = OrderedDict([("foo", "str")])
-
- feature = {
- "geometry": {"type": "Point", "coordinates": (-1, 1, 5)},
- "properties": OrderedDict([("foo", "bar")]),
- }
-
- with fiona.open(
- "/tmp/file.shp",
- "w",
- driver="ESRI Shapefile",
- schema={"geometry": "Point", "properties": schema_props},
- ) as collection:
- collection.write(feature)
-
- with fiona.open("/tmp/file.shp") as collection:
- print(next(collection)["geometry"])
+.. code-block:: python
- # {"type": "Point", "coordinates": (-1.0, 1.0)}
+ >>> feat = {"geometry": {"type": "Point", "coordinates": (-1, 1, 5)}}
+ >>> with fiona.open(
+ ... "example.shp",
+ ... "w",
+ ... driver="Shapefile",
+ ... schema={"geometry": "Point", "properties": {}}
+ ... ) as dst:
+ ... dst.write(feat)
+ ...
+ >>> with fiona.open("example.shp") as src:
+ ... print(src[0].geometry.coordinates)
+ ...
+ (-1.0, 1.0)
If you write 2D coordinates, ones having only (x, y) tuples, to a 3D file ('3D
Point' schema geometry, for example) a default z value of 0 will be provided.
-.. sourcecode:: python
-
- feature = {
- "geometry": {"type": "Point", "coordinates": (-1, 1)},
- "properties": OrderedDict([("foo", "bar")]),
- }
-
- with fiona.open(
- "/tmp/file.shp",
- "w",
- driver="ESRI Shapefile",
- schema={"geometry": "3D Point", "properties": schema_props},
- ) as collection:
- collection.write(feature)
-
- with fiona.open("/tmp/file.shp") as collection:
- print(next(collection)["geometry"])
+.. code-block:: python
- # {"type": "Point", "coordinates": (-1.0, 1.0, 0.0)}
+ >>> feat = {"geometry": {"type": "Point", "coordinates": (-1, 1)}}
+ >>> with fiona.open(
+ ... "example.shp",
+ ... "w",
+ ... driver="Shapefile",
+ ... schema={"geometry": "3D Point", "properties": {}}
+ ... ) as dst:
+ ... dst.write(feat)
+ ...
+ >>> with fiona.open("example.shp") as src:
+ ... print(src[0].geometry.coordinates)
+ ...
+ (-1.0, 1.0, 0.0)
Advanced Topics
===============
@@ -1111,16 +983,18 @@ To see debugging information from GDAL/OGR, for example, you may do the followin
.. code-block:: python
import logging
-
import fiona
-
logging.basicConfig(level=logging.DEBUG)
with fiona.Env(CPL_DEBUG=True):
- fiona.open("tests/data/coutwildrnp.shp")
+ fiona.open(
+ "zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip"
+ )
-The following extra messages will appear in the Python logger's output.::
+The following extra messages will appear in the Python logger's output.
+
+.. code-block::
DEBUG:fiona._env:CPLE_None in GNM: GNMRegisterAllInternal
DEBUG:fiona._env:CPLE_None in GNM: RegisterGNMFile
@@ -1139,16 +1013,16 @@ to its previous state.
Driver configuration options
----------------------------
-Drivers can have dataset open, dataset creation, respectively layer creation options. These options can be found
-on the drivers page on `GDAL's homepage. <https://gdal.org/drivers/vector/index.html>`_ or using the
-``fiona.meta`` module:
+Drivers can have dataset open, dataset creation, respectively layer creation
+options. These options can be found on the drivers page on `GDAL's homepage.
+<https://gdal.org/drivers/vector/index.html>`_ or using the ``fiona.meta``
+module:
.. code-block:: pycon
>>> import fiona.meta
>>> fiona.meta.print_driver_options("GeoJSON")
-
These options can be passed to ``fiona.open``:
.. code-block:: python
@@ -1156,7 +1030,6 @@ These options can be passed to ``fiona.open``:
import fiona
fiona.open('tests/data/coutwildrnp.json', ARRAY_AS_STRING="YES")
-
Cloud storage credentials
-------------------------
@@ -1165,13 +1038,16 @@ accessing data stored in AWS S3 or another cloud storage system.
.. code-block:: python
- from fiona.session import AWSSession
- import fiona
+ import fiona
+ from fiona.session import AWSSession
- with fiona.Env(
- session=AWSSession(aws_access_key_id="key", aws_secret_access_key="secret")
- ):
- fiona.open("zip+s3://example-bucket/example.zip")
+ with fiona.Env(
+ session=AWSSession(
+ aws_access_key_id="key",
+ aws_secret_access_key="secret"
+ )
+ ):
+ fiona.open("zip+s3://example-bucket/example.zip")
The AWSSession class is currently the only credential session manager in Fiona.
The source code has an example of how classes for other cloud storage providers
@@ -1199,18 +1075,20 @@ With some vector data formats a spatial index accompanies the data file,
allowing efficient bounding box searches. A collection's
:py:meth:`~fiona.collection.Collection.items` method returns an iterator over
pairs of FIDs and records that intersect a given ``(minx, miny, maxx, maxy)``
-bounding box or geometry object. Spatial filtering may be inaccurate and returning
-all features overlapping the envelope of the geometry. The
+bounding box or geometry object. Spatial filtering may be inaccurate and
+returning all features overlapping the envelope of the geometry. The
collection's own coordinate reference system (see below) is used to interpret
-the box's values. If you want a list of the iterator's items, pass it to Python's
-builtin :py:func:`list` as shown below.
+the box's values. If you want a list of the iterator's items, pass it to
+Python's builtin :py:func:`list` as shown below.
.. code-block:: pycon
- >>> c = fiona.open('docs/data/test_uk.shp')
- >>> hits = list(c.items(bbox=(-5.0, 55.0, 0.0, 60.0)))
- >>> len(hits)
- 7
+ >>> colxn = fiona.open(
+ ... "zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip"
+ ... )
+ >>> hits = list(colxn.items(bbox=(-110.0, 36.0, -108.0, 38.0)))
+ >>> len(hits)
+ 5
The iterator method takes the same ``stop`` or ``start, stop[, step]``
slicing arguments as :py:func:`itertools.islice`.
@@ -1218,7 +1096,7 @@ To get just the first two items from that iterator, pass a stop index.
.. code-block:: pycon
- >>> hits = c.items(2, bbox=(-5.0, 55.0, 0.0, 60.0))
+ >>> hits = colxn.items(2, bbox=(-110.0, 36.0, -108.0, 38.0)))
>>> len(list(hits))
2
@@ -1227,7 +1105,7 @@ indexes.
.. code-block:: pycon
- >>> hits = c.items(2, 5, bbox=(-5.0, 55.0, 0.0, 60.0))
+ >>> hits = colxn.items(2, 5, bbox=(-110.0, 36.0, -108.0, 38.0)))
>>> len(list(hits))
3
@@ -1237,13 +1115,15 @@ record and returns ``True`` or ``False``.
.. code-block:: pycon
- >>> def pass_positive_area(rec):
- ... return rec['properties'].get('AREA', 0.0) > 0.0
- ...
- >>> c = fiona.open('docs/data/test_uk.shp')
- >>> hits = filter(pass_positive_area, c)
- >>> len(list(hits))
- 48
+ >>> def pass_positive_area(rec):
+ ... return rec['properties'].get('AREA', 0.0) > 0.0
+ ...
+ >>> colxn = fiona.open(
+ ... "zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip"
+ ... )
+ >>> hits = filter(pass_positive_area, colxn)
+ >>> len(list(hits))
+ 67
Reading Multilayer data
-----------------------
@@ -1251,25 +1131,20 @@ Reading Multilayer data
Up to this point, only simple datasets with one thematic layer or feature type
per file have been shown and the venerable Esri Shapefile has been the primary
example. Other GIS data formats can encode multiple layers or feature types
-within a single file or directory. Esri's `File Geodatabase
-<https://gdal.org/drivers/vector/filegdb.html#vector-filegdb>`__ is one example of such a format.
-A more useful example, for the purpose of this manual, is a directory
-comprising multiple shapefiles. The following three shell commands will create
-just such a two layered data source from the test data distributed with Fiona.
-
-.. code-block:: console
-
- $ mkdir /tmp/data
- $ ogr2ogr /tmp/data/ docs/data/test_uk.shp test_uk -nln foo
- $ ogr2ogr /tmp/data/ docs/data/test_uk.shp test_uk -nln bar
+within a single file or directory. GeoPackage is one example of such a format.
+A more useful example, for the purpose of this manual, is a directory or
+zipfile comprising multiple shapefiles. The GitHub-hosted zipfile we've been
+using in these examples is, in fact, such a multilayer dataset.
-The layers of a data source can be listed using :py:func:`fiona.listlayers`. In
+The layers of a dataset can be listed using :py:func:`fiona.listlayers`. In
the shapefile format case, layer names match base names of the files.
.. code-block:: pycon
- >>> fiona.listlayers('/tmp/data')
- ['bar', 'foo']
+ >>> fiona.listlayers(
+ ... "zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip"
+ ... )
+ ['coutwildrnp']
Unlike OGR, Fiona has no classes representing layers or data sources. To access
the features of a layer, open a collection using the path to the data source
@@ -1277,45 +1152,44 @@ and specify the layer by name using the `layer` keyword.
.. code-block:: pycon
- >>> import pprint
- >>> datasrc_path = '/tmp/data'
- >>> for name in fiona.listlayers(datasrc_path):
- ... with fiona.open(datasrc_path, layer=name) as c:
- ... pprint.pprint(c.schema)
- ...
- {'geometry': 'Polygon',
- 'properties': {'CAT': 'float:16',
- 'FIPS_CNTRY': 'str',
- 'CNTRY_NAME': 'str',
- 'AREA': 'float:15.2',
- 'POP_CNTRY': 'float:15.2'}}
- {'geometry': 'Polygon',
- 'properties': {'CAT': 'float:16',
- 'FIPS_CNTRY': 'str',
- 'CNTRY_NAME': 'str',
- 'AREA': 'float:15.2',
- 'POP_CNTRY': 'float:15.2'}}
-
-Layers may also be specified by their index.
+ dataset_path = "zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip"
+ >>> for name in fiona.listlayers(dataset_path):
+ ... with fiona.open(dataset_path, layer=name) as colxn:
+ ... pprint.pprint(colxn.schema)
+ ...
+ {'geometry': 'Polygon',
+ 'properties': {'AGBUR': 'str:80',
+ 'AREA': 'float:24.15',
+ 'FEATURE1': 'str:80',
+ 'FEATURE2': 'str:80',
+ 'NAME': 'str:80',
+ 'PERIMETER': 'float:24.15',
+ 'STATE': 'str:80',
+ 'STATE_FIPS': 'str:80',
+ 'URL': 'str:101',
+ 'WILDRNP020': 'int:10'}}
+
+Layers may also be specified by their numerical index.
.. code-block:: pycon
- >>> for i, name in enumerate(fiona.listlayers(datasrc_path)):
- ... with fiona.open(datasrc_path, layer=i) as c:
- ... print(len(c))
- ...
- 48
- 48
+ >>> for index, name in enumerate(fiona.listlayers(dataset_path)):
+ ... with fiona.open(dataset_path, layer=index) as colxn:
+ ... print(len(colxn))
+ ...
+ 67
If no layer is specified, :py:func:`fiona.open` returns an open collection
using the first layer.
.. code-block:: pycon
- >>> with fiona.open(datasrc_path) as c:
- ... c.name == fiona.listlayers(datasrc_path)[0]
- ...
- True
+ >>> with fiona.open(dataset_path) as colxn:
+ ... colxn.name == fiona.listlayers(datasset_path)[0]
+ ...
+ True
+
+We've been relying on this implicit behavior throughout the manual.
The most general way to open a shapefile for reading, using all of the
parameters of :py:func:`fiona.open`, is to treat it as a data source with
@@ -1323,14 +1197,20 @@ a named layer.
.. code-block:: pycon
- >>> fiona.open('docs/data/test_uk.shp', 'r', layer='test_uk')
+ >>> fiona.open(
+ ... "zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip",
+ ... mode="r",
+ ... layer="coutwildrnp"
+ ... )
In practice, it is fine to rely on the implicit first layer and default ``'r'``
mode and open a shapefile like this:
.. code-block:: pycon
- >>> fiona.open('docs/data/test_uk.shp')
+ >>> fiona.open(
+ ... "zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip",
+ ... )
Writing Multilayer data
-----------------------
@@ -1340,65 +1220,18 @@ a unique name to the `layer` keyword argument.
.. code-block:: pycon
- >>> 'wah' not in fiona.listlayers(datasrc_path)
- True
- >>> with fiona.open(datasrc_path, layer='bar') as c:
- ... with fiona.open(datasrc_path, 'w', layer='wah', **c.meta) as d:
- ... d.write(next(c))
- ...
- >>> fiona.listlayers(datasrc_path)
- ['bar', 'foo', 'wah']
+ >>> with fiona.open(
+ ... "zip+https://github.com/Toblerity/Fiona/files/11151652/coutwildrnp.zip",
+ ... ) as src:
+ ... with fiona.open("example.gpkg", "w", layer="example one", **src.profile) as dst:
+ ... dst.writerecords(src)
+ ...
+ >>> fiona.listlayers("example.gpkg")
+ ['example one']
In ``'w'`` mode, existing layers will be overwritten if specified, just as normal
files are overwritten by Python's :py:func:`open` function.
-.. code-block:: pycon
-
- >>> 'wah' in fiona.listlayers(datasrc_path)
- True
- >>> with fiona.open(datasrc_path, layer='bar') as c:
- ... with fiona.open(datasrc_path, 'w', layer='wah', **c.meta) as d:
- ... # Overwrites the existing layer named 'wah'!
-
-Virtual filesystems
--------------------
-
-Zip and Tar archives can be treated as virtual filesystems and collections can
-be made from paths and layers within them. In other words, Fiona lets you read
-zipped shapefiles. For example, make a Zip archive from the shapefile
-distributed with Fiona.
-
-.. code-block:: console
-
- $ zip /tmp/zed.zip docs/data/test_uk.*
- adding: docs/data/test_uk.shp (deflated 48%)
- adding: docs/data/test_uk.shx (deflated 37%)
- adding: docs/data/test_uk.dbf (deflated 98%)
- adding: docs/data/test_uk.prj (deflated 15%)
-
-The `vfs` keyword parameter for :py:func:`fiona.listlayers` and
-:py:func:`fiona.open` may be an Apache Commons VFS style string beginning with
-"zip://" or "tar://" and followed by an absolute or relative path to the
-archive file. When this parameter is used, the first argument to must be an
-absolute path within that archive. The layers in that Zip archive are:
-
-.. code-block:: pycon
-
- >>> import fiona
- >>> fiona.listlayers('/docs/data', vfs='zip:///tmp/zed.zip')
- ['test_uk']
-
-The single shapefile may also be accessed like so:
-
-.. code-block:: pycon
-
- >>> with fiona.open(
- ... '/docs/data/test_uk.shp',
- ... vfs='zip:///tmp/zed.zip') as c:
- ... print(len(c))
- ...
- 48
-
Unsupported drivers
-------------------
=====================================
fiona/__init__.py
=====================================
@@ -126,7 +126,7 @@ __all__ = [
"prop_width",
]
-__version__ = "1.9.2"
+__version__ = "1.9.3"
__gdal_version__ = get_gdal_release_name()
gdal_version = get_gdal_version_tuple()
=====================================
fiona/collection.py
=====================================
@@ -94,8 +94,17 @@ class Collection:
raise TypeError("invalid driver: %r" % driver)
if schema and not hasattr(schema, "get"):
raise TypeError("invalid schema: %r" % schema)
- if crs and not isinstance(crs, compat.DICT_TYPES + (str, CRS)):
+
+ # Rasterio's CRS is compatible with Fiona. This class
+ # constructor only requires that the crs value have a to_wkt()
+ # method.
+ if (
+ crs
+ and not isinstance(crs, compat.DICT_TYPES + (str, CRS))
+ and not (hasattr(crs, "to_wkt") and callable(crs.to_wkt))
+ ):
raise TypeError("invalid crs: %r" % crs)
+
if crs_wkt and not isinstance(crs_wkt, str):
raise TypeError("invalid crs_wkt: %r" % crs_wkt)
if encoding and not isinstance(encoding, str):
=====================================
fiona/fio/load.py
=====================================
@@ -30,9 +30,22 @@ from fiona.transform import transform_geom
"zero-based numbering when accessed by index.",
)
@options.creation_opt
+ at options.open_opt
+ at click.option("--append", is_flag=True, help="Open destination layer in append mode.")
@click.pass_context
@with_context_env
-def load(ctx, output, driver, src_crs, dst_crs, features, layer, creation_options):
+def load(
+ ctx,
+ output,
+ driver,
+ src_crs,
+ dst_crs,
+ features,
+ layer,
+ creation_options,
+ open_options,
+ append,
+):
"""Load features from JSON to a file in another format.
The input is a GeoJSON feature collection or optionally a sequence of
@@ -81,14 +94,19 @@ def load(ctx, output, driver, src_crs, dst_crs, features, layer, creation_option
for k, v in first.properties.items()
}
- with fiona.open(
- output,
- "w",
- driver=driver,
- crs=dst_crs,
- schema=schema,
- layer=layer,
- **creation_options
- ) as dst:
+ if append:
+ opener = fiona.open(output, "a", layer=layer, **open_options)
+ else:
+ opener = fiona.open(
+ output,
+ "w",
+ driver=driver,
+ crs=dst_crs,
+ schema=schema,
+ layer=layer,
+ **creation_options
+ )
+
+ with opener as dst:
dst.write(first)
dst.writerecords(source)
=====================================
fiona/ogrext.pyx
=====================================
@@ -368,16 +368,17 @@ cdef class FeatureBuilder:
string_list = OGR_F_GetFieldAsStringList(feature, i)
string_list_index = 0
props[key] = []
- while string_list[string_list_index] != NULL:
- val = string_list[string_list_index]
- try:
- val = val.decode(encoding)
- except UnicodeDecodeError:
- log.warning(
- "Failed to decode %s using %s codec", val, encoding
- )
- props[key].append(val)
- string_list_index += 1
+ if string_list != NULL:
+ while string_list[string_list_index] != NULL:
+ val = string_list[string_list_index]
+ try:
+ val = val.decode(encoding)
+ except UnicodeDecodeError:
+ log.warning(
+ "Failed to decode %s using %s codec", val, encoding
+ )
+ props[key].append(val)
+ string_list_index += 1
else:
props[key] = None
=====================================
tests/test_fio_load.py
=====================================
@@ -155,3 +155,57 @@ def test_load__auto_detect_format(tmpdir, runner, feature_seq, extension, driver
assert src.crs == {'init': 'epsg:32617'}
assert len(src) == len(feature_seq.splitlines())
assert src.driver == driver
+
+
+def test_fio_load_layer_append(tmpdir, runner):
+ """Checking append mode."""
+ outdir = str(tmpdir.mkdir("tests").mkdir("test_fio_load_layer"))
+ try:
+ feature = {
+ "type": "Feature",
+ "properties": {"key": "value"},
+ "geometry": {"type": "Point", "coordinates": (5.0, 39.0)},
+ }
+ sequence = os.linesep.join(
+ map(partial(json.dumps, cls=ObjectEncoder), [feature, feature])
+ )
+
+ # Write mode to create layer.
+ result = runner.invoke(
+ main_group,
+ [
+ "load",
+ outdir,
+ "--driver",
+ "ESRI Shapefile",
+ "--src-crs",
+ "EPSG:4236",
+ "--layer",
+ "test_layer",
+ ],
+ input=sequence,
+ )
+ assert result.exit_code == 0
+
+ # Here's the append.
+ result = runner.invoke(
+ main_group,
+ [
+ "load",
+ outdir,
+ "--driver=ESRI Shapefile",
+ "--src-crs=EPSG:4236",
+ "--layer=test_layer",
+ "--append",
+ ],
+ input=sequence,
+ )
+ assert result.exit_code == 0
+
+ with fiona.open(outdir) as src:
+ assert len(src) == 4
+ assert src.name == "test_layer"
+ assert src.schema["geometry"] == "Point"
+
+ finally:
+ shutil.rmtree(outdir)
=====================================
tests/test_geojson.py
=====================================
@@ -1,3 +1,7 @@
+"""Tests of behavior specific to GeoJSON"""
+
+import json
+
import pytest
import fiona
@@ -136,3 +140,27 @@ def test_write_json_invalid_directory(tmpdir):
schema = {"geometry": "Unknown", "properties": [("title", "str")]}
with pytest.raises(DriverError):
fiona.open(path, "w", driver="GeoJSON", schema=schema)
+
+
+def test_empty_array_property(tmp_path):
+ """Confirm fix for bug reported in gh-1227."""
+ tmp_path.joinpath("test.geojson").write_text(
+ json.dumps(
+ {
+ "type": "FeatureCollection",
+ "features": [
+ {
+ "type": "Feature",
+ "geometry": {"type": "Point", "coordinates": [12, 24]},
+ "properties": {"array_prop": ["some_value"]},
+ },
+ {
+ "type": "Feature",
+ "geometry": {"type": "Point", "coordinates": [12, 24]},
+ "properties": {"array_prop": []},
+ },
+ ],
+ }
+ )
+ )
+ list(fiona.open(tmp_path.joinpath("test.geojson")))
=====================================
tests/test_schema.py
=====================================
@@ -432,7 +432,7 @@ def test_schema_string_list(tmp_path):
"properties": {
"time_range": '["2020-01-01", "2020-01-02"]',
},
- }
+ },
]
)
View it on GitLab: https://salsa.debian.org/debian-gis-team/fiona/-/commit/6f8de821752dfc74de3d67b8dbf33853f17ae894
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/fiona/-/commit/6f8de821752dfc74de3d67b8dbf33853f17ae894
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20230411/765f63bc/attachment-0001.htm>
More information about the Pkg-grass-devel
mailing list