[Pkg-crosswire-devel] Testing Lucene search

[Jonathan Marsden]

I asked:

>> Anyone who knows how to test the lucene search stuff, please talk to me!

Matthew Talbert wrote:

> The utility mkfastmod builds the lucene indexes. To use, just do
> "mkfastmod KJV". Afterwards, you can build the example utility
> search.cpp (in examples/cmdline) and perform a search with this. It
> should provide *significantly* faster results than diatheke.

Thanks; I'm trying it out now.  I wonder if we should consider building
with --enable-examples=yes, and then include the example programs in the
libsword-dev binary package, or something like that?

Because I installed the KJV module into /usr/share/sword, a simple

  mkfastmod KJV

(as a normal user) runs for a while, and when the progress bar reaches
100% it then errors out (with a very unhelpful generic error message),
because it can't write where it wants to write its index files!  Doing

  sudo mkfastmod KJV

worked rather better.  However, I must be doing something wrong, because
 the search utility seems to always rebuild a set of indexes first, or
something, so I get:



$ time -p diatheke -b KJV -s multiword -k swift hear slow speak
Verses containing "swift hear slow speak"-- James 1:19 -- 1 matches
total (KJV)
real 4.11
user 3.86
sys 0.25
$ SWORD_PATH=/usr/share/sword time -p ./search KJV "swift hear slow speak"
UTF8Transliterator: ICU: no resource index to load
UTF8Transliterator: ICU: status U_MISSING_RESOURCE_ERROR
[0=================================50===============================100]
 ======================================================================
James 1:19
real 4.91
user 4.68
sys 0.22

When I edit search.cpp so that we really use Lucene search, it is much
quicker but gives me an "or" search listing all verses with any one of
the searched-for words, not the single verse containing all of them.  A
bit weird, but at least that seems to suggest that Lucene search is
working as (probably) intended.

It also suggests that we do indeed need to provide those *.res files if
we build with ICU... but that's a different thread :)

Jonathan