[Pkg-crosswire-devel] ICU and stability issues

[Matthew Talbert]

>
> On Jan 26, 2009, at 7:37 PM, Matthew Talbert wrote:
>
>> On Mon, Jan 26, 2009 at 7:24 PM, DM Smith <dmsmith555 at yahoo.com> wrote:
>>>
>>> On Jan 26, 2009, at 5:58 PM, Matthew Talbert wrote:
>>>
>>>>> Matthew Talbert wrote:
>>>>>
>>>>>> However, GnomeSword and BibleTime both provide their own string casing
>>>>>> function for this purpose, both using standard unicode definitions to
>>>>>> provide casing. So, as I've said, even if SWORD is built with ICU, it
>>>>>> will provide no benefit to the end user, and may lead to stability
>>>>>> issues.
>>>>>
>>>>> I think it would be a real help in making this choice appropriately if
>>>>> we
>>>>> could get someone (you?) to create a replicable test case, where a
>>>>> libsword
>>>>> 1.5.11-based setup compiled with ICU reliably causes stability issues,
>>>>> but
>>>>> one without ICU does not.  Ideally the app used in the test would be a
>>>>> non-interactive command line tool, so we're not dependent on big GUI
>>>>> apps
>>>>> and users clicking on a bunch of things in order to test.
>>>>>
>>>>> Likewise, I'd be happy to see someone (else) create a test case where a
>>>>> search fails when using the 1.5.11 library with ICU disabled, but works
>>>>> with
>>>>> it enabled.  Maybe this is as simple as
>>>>>
>>>>> sudo installmgr -ri X Y
>>>>> diatheke <some options here>
>>>>>
>>>>> (But right now, I don't have a known set of X, Y and <some options
>>>>> here>
>>>>> that I know will act as such a test case!).
>>>>>
>>>>> Once we have repeatable test cases, we can then either fix the
>>>>> stability
>>>>> issue with a patch at build time, or we can make an informed decision
>>>>> decide
>>>>> to build without ICU.  If there is an existing (replicable) bug report
>>>>> that
>>>>> has good info on this, do please point me at it.
>>>>>
>>>>> If libsword packagers are going to get squeezed between frontend
>>>>> developers who say ICU causes stability issues and should be disabled,
>>>>> and
>>>>> others in the sword-devel community who say it is really does cause
>>>>> loss of
>>>>> functionality to disable it, so it should be enabled... then I'd like
>>>>> to see
>>>>> some clear test evidence, from both sets of people, to help  us
>>>>> packagers
>>>>> make a sane choice!  Is that realistic?
>>>>>
>>>>
>>>> I have asked for those saying GnomeSword will not search correctly
>>>> without ICU to submit just such a test case, but have not seen any.
>>>>
>>>> There is a thread here,
>>>> http://www.crosswire.org/pipermail/sword-devel/2009-January/030641.html,
>>>> that while not an official bug report, does indicate the problems, and
>>>> upstream's action of immediately disabling many of the filters
>>>> constitutes to me an admission that they do cause problems.
>>>
>>> Matthew,
>>> I saw the checkin disabling many of the transliteration filters. I think
>>> it
>>> was stated here that none of the Linux front-ends expose the
>>> transliteration
>>> filters. None of the modules mention them. So I can't see how they would
>>> cause a problem. I didn't see any other changes to the SWORD engine
>>> regarding ICU or any other admission to ICU causing a problem.
>>
>> You may well be right. I am still learning about this. I was under the
>> impression that the transliteration filters were used somehow even
>> without the front-ends exposing them.
>>
>>> The only filters that use ICU are utf8arshaping, utf8bidireorder, and
>>> uft8nfc, utf8nfkd. The first two are used to display right-to-left texts.
>>> The other two are used by module builders, but I couldn't find anywhere
>>> else
>>> that they were used by the engine.
>>>
>>> I found that diatheke uses ICU extensively.
>>>
>>> Maybe I don't understand SWORD well enough, but I can't see where
>>> including
>>> ICU and not using transliterators causes a problem.
>>>
>>> I'd suggest, that sword-tools are compiled with ICU and, perhaps, that
>>> libsword7 is not.
>>
>> I think this is a good idea. However, wouldn't this require that the
>> tools be statically linked? Or do they not depend on libsword at all?
>
> They could be statically linked. That's what I do. But I always compile them
> for my own use.
>
> Otherwise, I'd like to see proof that ICU is actually unstable.

My biggest concern was instability causing problems with GnomeSword.
If, as you say, the transliteration filters are never activated by the
library, then there shouldn't be any issues. In this case, I'm fine
with leaving ICU enabled.

Jonathan, note that with ICU, mkfastmod should create good module
indices, and could be used to create the indices at installation time.

Matthew