[Pkg-crosswire-devel] Module packaging, distribution, and management

[Jonathan Marsden]

Peter von Kaehne wrote:

> And while I am impressed by your dedication to package our 300 + (and
> rising) modules I simply do not see the point.

Exactly 304 .zip files were downloadable from
http://crosswire.org/ftpmirror/pub/sword/packages/rawzip/ so I just
grabbed them, and I am going through their .conf files now... 172 have a
DistributionLicense line, of which 128 are labelled as being Public Domain.

I may have first-pass packages for these 128 ready by end of day, stay
tuned :)  Half a day to package a few hundred modules is not all that
impressive dedication, IMO... just a bit of mild scripting work!

(1) Your stated inability to see the utility of a single standard
packaging system suggests to me that you are more a part of a
Windows-oriented community that a Linux-oriented one.  Community norms
matter, and should be respected.

Of course, packaging these modules does not do any harm to the existing
user community.  If the current Sword user community wants to ignore
these packages, for any reason whatsoever, and wishes to just manually
download modules into ~/.sword per user, well, OK -- no amount of
packaging of modules prevents or in any way restricts anyone's ability
to continue to do exactly that.  Even if our front end packages Depends:
sword-text one can always install them

(2) Also, and perhaps more importantly, I suspect that you do not see
the point because you apparently assume that every single user of Sword
frontends has a fast and reliable Internet connection at low cost and an
individual PC for their own use; this is a totally untenable assumption!
 Everyone is not in that priviledged position.

Consider missionaries using VHF packet radio for data, and some with on
HF radio for email (I have set up and maintained such systems, BTW --
this is not just theoretical!); think of small bible schools or groups
of pastors in "Third World" locations with dialup only Internet access;
think folks on the OM ships and Mercy Ships (yes, I've worked with some
of those kinds of folks, including the IT folks responsible for the
servers and datacomms on those ships, too)... think of children using
OLPC laptops with a slow mesh network connection back to a local school
with an over-subscribed satellite Internet uplink (I've worked on
satellite Internet uplink stuff, too, in the past, though not for OLPC
users!).

Please, do not assume that everyone has the kind of Internet access that
you have, or the same access to "their own" PC.

> A module is a text. An e-book of sorts.

Then surely Sword should switch to an existing standard data format, why
reinvent the wheel? :)  The fact that it does not do this strongly
suggests to me that, in fact, these modules are rather different from a
conventional text file or ebook.

> It is not data like a game episode (a la Wesnoth).

Actually, I think this one is fairly close -- they contain information
in a non-standard data format, data without at least one set of which
the application is useless... this is a pretty good comparison, IMO!  In
what sense are Sword modules "not data"?

> Both BT and GS open up on first time with a dialogue - "you got no
> modules, shall I fetch a couple" so there is no risk that anyone is
> module less for long.

Indeed there is such risk!  Again your assumptions show here.

(a) What if they have no fast cheap reliable Internet access, how will
it "fetch a couple"?

(b) Or if you have a classroom of 30 machines and every student
immediately downloads the same module, is this good stewardship of your
(possibly limited, or capped at so many MBytes/day) bandwidth?  Then the
next week, they come back to class but sit a different workstations, so
now (in a naive setup with no shared /home) they then re-download them
all over again...!

(c) What if there is only one available (old) PC, used by 50 students
one at a time for 30 minutes per week... must each student download
(possibly over a slow dialup Internet link) and then install the same
module(s) to their own ~/.sword/ ?  Is this wise use of limited disk space?

Good use of shared module install locations just makes sense in many
many situations.  Good use of data distribution on CD and DVD also makes
sense sometimes.  Debian/Ubuntu style package management handles both of
these, as well as the more common (in the West) case where individual
users have fast cheap Internet access available to them, and have at
least one PC per user.  Does your suggested approach do as well in each
of these situations?

> We have an elaborate module manager which can deal with a whole range of
> install options, can update modules from a variety of sources and can
> deal with multiple, dynamically added and removed repositories. Apt-get
> gets into the way and for no good reason.

If it deals just fine with data in all of the standard locations
recommended by Sword documentation, including /usr/share/sword, and in
fact handles all locations potentially pointed to by the Sword library
config file(s), some of which may be on read-only filesystems, then you
might have a good point here.  But, as far as I know, it does not, per
our earlier discussion.

It apparently lacks fundamental functionality for handling per-system
(or per set of systems on a LAN) module installations, as far as I can
tell.  It just seems to naively assume that it can write to every
location on every filesystem that it can see is a module location, and
then gets upset or does not work as expected whenever that assumption is
invalid.  This is a pretty serious bug, IMO, especially since one such
system-shared location is clearly recommended by the documentation in
the Sword tarball.

I would like to read the full design documentation for the "elaborate"
module manager, including the set of use cases for it that were
considered before it was implemented.  Does such documentation exist,
and if so, is it publically available?  If so, please provide a URL to it.

Jonathan