Processed: HDF 4.3.1 uploaded to unstable

[Antonio Valentino]

Dear Bas,

Il 20/10/25 08:20, Sebastiaan Couwenberg ha scritto:
> On 10/20/25 8:03 AM, Antonio Valentino wrote:
>> Finally I manage to spare some time to investigate the issue and I 
>> realized that there is a very similar problem on rioxarray.
>> For rioxarray I was able to identify a minimum set of tests (two) 
>> that, if run together, can trigger the issue. I reported it in [1].
>>
>> The problem, both for satpy and rioxarray, seems to happen during the 
>> garbage collection of objects representing an HDF4 file.
>> In satpy it libhdf4 is used through pyhfd, while in rioxarray through 
>> rasterio and gdal.
>>
>> Are you aware of any change in the libhdf4 API of v4.3.x that could 
>> lead to this kind of issues?
> 
> I don't know more than what upstream documented:
> 
>   https://github.com/HDFGroup/hdf4/blob/hdf4.3.1/release_notes/ 
> RELEASE.txt#L47
> 
> 4.3.1 did have focus on memory leak fixes.
> 
>> The satpy error apparently happens in a portion of the code that 
>> exploits treading.
>> Apart for making the debugging harder, I'm not sure this detail is 
>> relevant for the specific problem.
>>
>> If my understanding is correct, the relevant portion of the satpy 
>> implementation is being rewritten in the upstream main branch for 
>> unrelated reasons. I will give it a try ASAP to understand if the new 
>> implementation has the same issues on debian.
>>
>> Unfortunately, I have very few time to dedicate to it in this period.
>> satpy has been removed form testing and rioxarray will be removed very 
>> soon.
>> I'm sorry for that and I really hope that it does not creates too many 
>> issues.
> 
> There is a new satpy release we can see if that resolves anything.
> 
> I guess your lack of time is also the reason that gmtsar hasn't been 
> updated to 6.6 yet.
> 
> If you want I can update these packages.

I finally managed to come up with a kind of solution that, apparently, 
fixes the segfaults both in rioxarray and satpy.
The build of those packages still fails due to unrelated issues, and 
both of them have new upstream releases.

The attached patch basically reverts the content of hdf4 
mfhdf/libsrc/file.c to the version 4.3.0 of HDF4 itself.

I still would need to do some tests to doublecheck that it does not 
introduces new errors in dependent packages.
According to the HDF4 changelog it could introduce some memory leak.

Apart for the additional checks I'm not sure how to proceed.
I would appreciate if you could give a look to the patch.

Then, maybe, it would be good to have an upload to experimental.
What do you think? Would you be available to do it?

Do you want me to file bug report or prepare a merge request in salsa?

Once we are sure that the patch is really working I will file bug report 
/ PR to https://github.com/HDFGroup/hdf4/


kind regards
-- 
Antonio Valentino
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix-rioxarray.patch
Type: text/x-patch
Size: 2149 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20251122/20000ac6/attachment.bin>