[DRE-maint] Bug#741576: ruby-bdb: FTBFS (on AMD64): Test failures

[David Suárez]

Hi !

Yes, this is 64 bits specific issue.

> According to the documentation, set_h_hash is meant to «specify a
> hashing function». ./examples/func.rb is the only other file to make
> use of this parameter, in exactly the same way as this test (and thus
> will likely fail).

> The most I can dig into the issue right now (as I'm tired and my brain
> does not want to parse C↔Ruby interaction) is that behaviour related
> to set_h_hash is found in lines 666-677 of src/common.h.

The problem is in the hashing function callback;

> static u_int32_t
> #if BDB_OLD_FUNCTION_PROTO
> bdb_h_hash(const void *bytes, u_int32_t length)
> #else
> bdb_h_hash(DB *dbbd, const void *bytes, u_int32_t length)

BDB states that the callback should return a u_int32_t. C code tries to 
convert the ruby number that the returns the (ruby level) hashing function 
into an unsigned int (using the NUM2INT helper), on 64 bits systems this could 
overflow.

> res = rb_funcall(dbst->h_hash, bdb_id_call, 1, st);
> 
> return NUM2INT(res);

I dunno how to procede here. Could simply switch to use NUM2ULONG function to 
evade the overflow but I need to return an u_int32_t due to the callback 
function constrainsts (this could leads to strange behaviour on 64 bits 
system).