Bug#802321: gdal-bin: NetCDF-4 / HDF5 support broken in recent gdal

Sebastiaan Couwenberg sebastic at xs4all.nl
Tue Oct 20 07:06:02 UTC 2015 

Control: tags -1 unreproducible
Control: severity -1 important

Hi Otakar,

Thanks for the nc file, can you also send the files used by gmt?

With JEJK_topoC9k_ZABA10mHbpv.nc I can now confirm that the issue is not
present in gdal (1.11.2+dfsg-3+b2) on unstable:

$ ncdump -h JEJK_topoC9k_ZABA10mHbpv.nc
netcdf JEJK_topoC9k_ZABA10mHbpv {
dimensions:
        x = 501 ;
        y = 601 ;
variables:
        double x(x) ;
                x:long_name = "x" ;
                x:actual_range = -546000., -541000. ;
        double y(y) ;
                y:long_name = "y" ;
                y:actual_range = -1073000., -1067000. ;
        float z(y, x) ;
                z:long_name = "z" ;
                z:_FillValue = NaNf ;
                z:actual_range = 2.38614749908447, 11.5756139755249 ;

// global attributes:
                :Conventions = "COARDS, CF-1.5" ;
                :title = "Produced by grdreformat" ;
                :history = "grdreformat CRJK_topoC9c_ZABAGED10m.grd
Jesenik_topoC9_ZABAGED10mBpv.nc -R-546000/-541000/-1073000/-1067000" ;
                :GMT_version = "5.1.2 (r14256) [64-bit]" ;
}

$ ncdump -k JEJK_topoC9k_ZABA10mHbpv.nc
netCDF-4

$ h5dump -n JEJK_topoC9k_ZABA10mHbpv.nc
HDF5 "JEJK_topoC9k_ZABA10mHbpv.nc" {
FILE_CONTENTS {
 group      /
 dataset    /x
 dataset    /y
 dataset    /z
 }
}

$ gdalinfo JEJK_topoC9k_ZABA10mHbpv.nc
Driver: netCDF/Network Common Data Format
Files: JEJK_topoC9k_ZABA10mHbpv.nc
Size is 501, 601
Coordinate System is `'
Origin = (-546005.000000000000000,-1066995.000000000000000)
Pixel Size = (10.000000000000000,-10.000000000000000)
Metadata:
  NC_GLOBAL#Conventions=COARDS, CF-1.5
  NC_GLOBAL#GMT_version=5.1.2 (r14256) [64-bit]
  NC_GLOBAL#history=grdreformat CRJK_topoC9c_ZABAGED10m.grd
Jesenik_topoC9_ZABAGED10mBpv.nc -R-546000/-541000/-1073000/-1067000
  NC_GLOBAL#title=Produced by grdreformat
  x#actual_range={-546000,-541000}
  x#long_name=x
  y#actual_range={-1073000,-1067000}
  y#long_name=y
  z#_FillValue=nan
  z#actual_range={2.386147499084473,11.5756139755249}
  z#long_name=z
Corner Coordinates:
Upper Left  ( -546005.000,-1066995.000)
Lower Left  ( -546005.000,-1073005.000)
Upper Right ( -540995.000,-1066995.000)
Lower Right ( -540995.000,-1073005.000)
Center      ( -543500.000,-1070000.000)
Band 1 Block=167x151 Type=Float32, ColorInterp=Undefined
  NoData Value=nan
  Metadata:
    _FillValue=nan
    actual_range={2.386147499084473,11.5756139755249}
    long_name=z
    NETCDF_VARNAME=z

$ gmt grdreformat JEJK_topoC9k_ZABA10mHbpv.grd
JEJK_topoC9k_ZABA10mHbpv.xnc=nf
grdreformat: Error for input file: No such file
(JEJK_topoC9k_ZABA10mHbpv.grd)
grdreformat: Syntax error: Specify only one input and one output file

$ gmt grdinfo JEJK_topoC9k_ZABA10mHbpv.nc
JEJK_topoC9k_ZABA10mHbpv.nc: Title: Produced by grdreformat
JEJK_topoC9k_ZABA10mHbpv.nc: Command: grdreformat
CRJK_topoC9c_ZABAGED10m.grd Jesenik_topoC9_ZABAGED10mBpv.nc
-R-546000/-541000/-1073000/-1067000
JEJK_topoC9k_ZABA10mHbpv.nc: Remark:
JEJK_topoC9k_ZABA10mHbpv.nc: Gridline node registration used [Cartesian
grid]
JEJK_topoC9k_ZABA10mHbpv.nc: Grid file format: nf = GMT netCDF format
(32-bit float), COARDS, CF-1.5
JEJK_topoC9k_ZABA10mHbpv.nc: x_min: -546000 x_max: -541000 x_inc: 10
name: x nx: 501
JEJK_topoC9k_ZABA10mHbpv.nc: y_min: -1073000 y_max: -1067000 y_inc: 10
name: y ny: 601
JEJK_topoC9k_ZABA10mHbpv.nc: z_min: 2.38614749908 z_max: 11.5756139755
name: z
JEJK_topoC9k_ZABA10mHbpv.nc: scale_factor: 1 add_offset: 0
JEJK_topoC9k_ZABA10mHbpv.nc: format: netCDF-4 chunk_size: 167,151
shuffle: on deflation_level: 3

$ gmt grdinfo JEJK_topoC9k_ZABA10mHbpv.nc=gd
ERROR 1: nBlockYSize = 151, only 1 supported when reading bottom-up dataset
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: IReadBlock failed at X
offset 0, Y offset 0
ERROR 1: nBlockYSize = 151, only 1 supported when reading bottom-up dataset
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: IReadBlock failed at X
offset 1, Y offset 0
ERROR 1: nBlockYSize = 151, only 1 supported when reading bottom-up dataset
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: IReadBlock failed at X
offset 2, Y offset 0
ERROR 1: nBlockYSize = 151, only 1 supported when reading bottom-up dataset
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: IReadBlock failed at X
offset 0, Y offset 1
ERROR 1: nBlockYSize = 151, only 1 supported when reading bottom-up dataset
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: IReadBlock failed at X
offset 1, Y offset 1
ERROR 1: nBlockYSize = 151, only 1 supported when reading bottom-up dataset
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: IReadBlock failed at X
offset 2, Y offset 1
ERROR 1: nBlockYSize = 151, only 1 supported when reading bottom-up dataset
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: IReadBlock failed at X
offset 0, Y offset 2
ERROR 1: nBlockYSize = 151, only 1 supported when reading bottom-up dataset
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: IReadBlock failed at X
offset 1, Y offset 2
ERROR 1: nBlockYSize = 151, only 1 supported when reading bottom-up dataset
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: IReadBlock failed at X
offset 2, Y offset 2
ERROR 1: nBlockYSize = 151, only 1 supported when reading bottom-up dataset
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: IReadBlock failed at X
offset 0, Y offset 3
ERROR 1: nBlockYSize = 151, only 1 supported when reading bottom-up dataset
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: IReadBlock failed at X
offset 1, Y offset 3
ERROR 1: nBlockYSize = 151, only 1 supported when reading bottom-up dataset
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: IReadBlock failed at X
offset 2, Y offset 3
ERROR 1: JEJK_topoC9k_ZABA10mHbpv.nc, band 1: Failed to compute min/max,
no valid pixels found in sampling.
JEJK_topoC9k_ZABA10mHbpv.nc: Title: Grid imported via GDAL
JEJK_topoC9k_ZABA10mHbpv.nc: Command:
JEJK_topoC9k_ZABA10mHbpv.nc: Remark:
JEJK_topoC9k_ZABA10mHbpv.nc: Gridline node registration used [Cartesian
grid]
JEJK_topoC9k_ZABA10mHbpv.nc: Grid file format: gd = Import/export
through GDAL
JEJK_topoC9k_ZABA10mHbpv.nc: x_min: -546000 x_max: -541000 x_inc: 10
name: x nx: 501
JEJK_topoC9k_ZABA10mHbpv.nc: y_min: -1073000 y_max: -1067000 y_inc: 10
name: y ny: 601
JEJK_topoC9k_ZABA10mHbpv.nc: z_min: 0 z_max: 0 name: z
JEJK_topoC9k_ZABA10mHbpv.nc: scale_factor: 1 add_offset: 0


On 20-10-15 07:39, otas wrote:
> On Mon, 19 Oct 2015 20:00:06 +0200, Sebastiaan Couwenberg wrote:
>>
>> I suspect this is not actually an issue in gdal, but caused by the
>> recent HDF5 transition to 1.8.15-patch1+docs-4.
> 
> I had the same suspicion on HDF5 libraries (yes, they were upgraded too).
> However, after downgrade to libhdf5-8_1.8.13+docs-15 and
> libhdf5-cpp-8_1.8.13+docs-15
> on my system nothing changed with gdal, i.e. the same error of gdalinfo.

You'll also need to downgrade gdal to the version before the hdf5
rebuild (1.11.2+dfsg-3+b1), and the other packages in the chain too:
netcdf (1:4.4.0~rc2-1) & gmt (5.1.2+dfsg1-2).

> Besides, NetCDF and HDF5 utils, at least, keep the file format correct
> recognition,
> see e.g. ncdump -h JEJK_topoC9k_ZABA10mHbpv.nc  or h5topng -d z
> JEJK_topoC9k_ZABA10mHbpv.nc
> 
> On the other hand, I can just recognize that something in the chain gmt
> - libgdal - libnetcdf - libhdf5 went wrong recently (Stretch upgrade at
> the begin of October), it may be somehow related to HDF5 transition.

hdf5 (1.8.15-patch1+docs-4) migrated to testing on October 5th.

If the package downgrades don't resolve the issue, can you report the
installed versions of gmt, gdal, netcdf & hdf5?

Kind Regards,

Bas

-- 
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1

More information about the Pkg-grass-devel mailing list