[sane-devel] sane_get_devices() is very slow

[Henning Meier-Geinitz]

Hi,

On Wed, Feb 19, 2003 at 01:27:31AM -0800, Derry Bryson wrote:
> But doesn't Linux USB support know when new devices are installed?  PCMCIA
> works pretty well detecting insert/removeal and installing/removing modules 
> as necessary.

Yes, USB hotplugging. Theoretically, you can write an application that
sets a flag when a device is (un)plugged and a frontend/backend would
then know that the device list must be reloaded. Bu that's Linux (and
USB) only.

> > Maybe that's a difference in philosopy. I want to have software that
> > works with as little user intervention as possible. So I don't like
> > installing drivers for every single piece of software and I don't even
> > like pressing buttons when I plug in a USB device.
> > 
> 
> I am certainly not suggesting one would want to install drivers for
> each piece of software (where does that come from)

Windows :-) That was a rather general comment, not targetted to you.

> nor was I saying
> you would have to press a button when you plug in a USB device, although
> this would be a simple solution.  Moving the caching of detected devices
> into SANE allows the backend to manage the cache and update whenever 
> necessary (such as when a new USB device is installed).  Much like 
> current PCMCIA device support.

The kernel already does this "caching", the backends just use the
kernel and ask about it's device list. Problems occur when that
"caching" is not very effective. For SCSI, /proc/scsi/scsi is used to
get the right SCSi device. So the backends just read this file (over
sanei_usb) to find out if there is a matching device. That's quite
fast. With libusb it's fast, too: just scan the list of USB devices
and check for matching IDs. The problem occurs with the USB scanner
driver. It doesn't provide a list of all scanners (and their device
files). So the approach is to check the availability of scanners by
opening all the device files (/dev/usbscanner* and /dev/usb/scanner*).
That's not extremely fast but shouldn't be that slow, either.

There have been two problems that made the detection slower than
necessary:

a) The kernel scanner driver prints a message everytime a device file
   is opened that isn't connected to an actual scanner. This slows
   down the detection if done often.
   Solution: The message isn't printed anymore in recent kernels
   Workaround: Reduced the number of tests (one per backend, not one
   per scanner) in sanei_usb
   Workaround: Reduce the number of tested device files (currently 34).
   Not implemented.
   Workaround: Remove unused device files in /dev/usb. Can be done by
   the user.
b) If devfs (device filesystem) is enabled, opening non-existant files
   in /dev is very slow because devfs tries to be clever and searches for 
   the correct driver for this device file. 
   Solution: Scan the directory before blindly opening files. Done in
   sanei_usb now.

> I am testing on a 400mhz Celeron with 96MB RAM running RH7.2 (which had
> SANE 1.0.5 installed by default), not using devfs, kernel version 2.4.7,
> no actual scanners installed (default dll.conf with pnm backend added).  
> My timing is by stopwatch starting when I select 'Select Source' in my 
> app and stopping when the dialog appears with a list of devices.  Since
> dialogs generally appear nearly instantaneously, most of the time is being
> spent in sane_get_devices().  I can record similar times using Xsane so
> I do not think my code is at fault.  What I am measuring is the actual
> user experience.

I did the same with xsane and the times match my test program (+ a
little overhead for opening the windows, but that's about 0.7 seconds).
So I think my test program measures correctly.

I did some more checks with other kernels, e.g. 2.2.19, 2.4.7,
2.4.21-pre3 + devfs. None take longer than 0.6 seconds. The first test
takes longer usually, (1.5 - 2 seconds), but that's because of loading
the shared libraries.

> I am looking at this from the point of view of distributing software that
> could be installed on systems by various non-guru users.  By default (on RH7.2), 
> I would be looking at a 36 seconds wait before the user would be presented
> with list of devices.  If I request they update and install version 1.0.11,
> they would still have to wait 7 seconds.  This will be faster or slower depending
> on their hardware, of course.

There must be something else that slows down the test. How long does
it take if you don't load the scanner module? If it's faster without
the kernel scanner driver, it's because of the messages printed to
syslog.

Are syslog messages printed to e.g. xconsole? That may slow down
things a bit.

Bye,
  Henning