[Pkg-crosswire-devel] ICU and stability issues

[DM Smith]

On Jan 26, 2009, at 7:37 PM, Matthew Talbert wrote:

> On Mon, Jan 26, 2009 at 7:24 PM, DM Smith <dmsmith555 at yahoo.com>  
> wrote:
>>
>> On Jan 26, 2009, at 5:58 PM, Matthew Talbert wrote:
>>
>>>> Matthew Talbert wrote:
>>>>
>>>>> However, GnomeSword and BibleTime both provide their own string  
>>>>> casing
>>>>> function for this purpose, both using standard unicode  
>>>>> definitions to
>>>>> provide casing. So, as I've said, even if SWORD is built with  
>>>>> ICU, it
>>>>> will provide no benefit to the end user, and may lead to stability
>>>>> issues.
>>>>
>>>> I think it would be a real help in making this choice  
>>>> appropriately if we
>>>> could get someone (you?) to create a replicable test case, where  
>>>> a libsword
>>>> 1.5.11-based setup compiled with ICU reliably causes stability  
>>>> issues, but
>>>> one without ICU does not.  Ideally the app used in the test would  
>>>> be a
>>>> non-interactive command line tool, so we're not dependent on big  
>>>> GUI apps
>>>> and users clicking on a bunch of things in order to test.
>>>>
>>>> Likewise, I'd be happy to see someone (else) create a test case  
>>>> where a
>>>> search fails when using the 1.5.11 library with ICU disabled, but  
>>>> works with
>>>> it enabled.  Maybe this is as simple as
>>>>
>>>> sudo installmgr -ri X Y
>>>> diatheke <some options here>
>>>>
>>>> (But right now, I don't have a known set of X, Y and <some  
>>>> options here>
>>>> that I know will act as such a test case!).
>>>>
>>>> Once we have repeatable test cases, we can then either fix the  
>>>> stability
>>>> issue with a patch at build time, or we can make an informed  
>>>> decision decide
>>>> to build without ICU.  If there is an existing (replicable) bug  
>>>> report that
>>>> has good info on this, do please point me at it.
>>>>
>>>> If libsword packagers are going to get squeezed between frontend
>>>> developers who say ICU causes stability issues and should be  
>>>> disabled, and
>>>> others in the sword-devel community who say it is really does  
>>>> cause loss of
>>>> functionality to disable it, so it should be enabled... then I'd  
>>>> like to see
>>>> some clear test evidence, from both sets of people, to help  us  
>>>> packagers
>>>> make a sane choice!  Is that realistic?
>>>>
>>>
>>> I have asked for those saying GnomeSword will not search correctly
>>> without ICU to submit just such a test case, but have not seen any.
>>>
>>> There is a thread here,
>>> http://www.crosswire.org/pipermail/sword-devel/2009-January/030641.html 
>>> ,
>>> that while not an official bug report, does indicate the problems,  
>>> and
>>> upstream's action of immediately disabling many of the filters
>>> constitutes to me an admission that they do cause problems.
>>
>> Matthew,
>> I saw the checkin disabling many of the transliteration filters. I  
>> think it
>> was stated here that none of the Linux front-ends expose the  
>> transliteration
>> filters. None of the modules mention them. So I can't see how they  
>> would
>> cause a problem. I didn't see any other changes to the SWORD engine
>> regarding ICU or any other admission to ICU causing a problem.
>
> You may well be right. I am still learning about this. I was under the
> impression that the transliteration filters were used somehow even
> without the front-ends exposing them.
>
>> The only filters that use ICU are utf8arshaping, utf8bidireorder, and
>> uft8nfc, utf8nfkd. The first two are used to display right-to-left  
>> texts.
>> The other two are used by module builders, but I couldn't find  
>> anywhere else
>> that they were used by the engine.
>>
>> I found that diatheke uses ICU extensively.
>>
>> Maybe I don't understand SWORD well enough, but I can't see where  
>> including
>> ICU and not using transliterators causes a problem.
>>
>> I'd suggest, that sword-tools are compiled with ICU and, perhaps,  
>> that
>> libsword7 is not.
>
> I think this is a good idea. However, wouldn't this require that the
> tools be statically linked? Or do they not depend on libsword at all?

They could be statically linked. That's what I do. But I always  
compile them for my own use.

Otherwise, I'd like to see proof that ICU is actually unstable.

>
>
>> While uconv (not iconv) can eliminate the need for ICU in osis2mod  
>> and
>> tei2mod, I added it precisely because people were creating modules  
>> that were
>> not NFC. I think it is a necessary convenience.
>>
>> In Him,
>>       DM
>>
>>