[sane-devel] Insights from a newbie backend writer...

Bertrik Sikken bertrik@zonnet.nl
Mon, 11 Apr 2005 20:53:50 +0200

Fred Odendaal wrote:
> So, I'm thinking "I've got a degree in comp. sci. How difficult can it
> be to write a scanner driver?" I leaf through the sane documentation -
> looks simple enough! "Okay", I decide, "I'll do it!" Then I start
> looking for information about my scanner - a Lexmark X1185. I start
> with  Lexmark. They've got a Linux developer's kit. "Oh wow! This is
> going to be easier that I thought...".
> Nope.
> The developer's kit only works with specific models of Lexmark scanners
> - not mine, and is only compliant with really old versions of RH Linux.
> I'm using FC3. The kit includes libraries for the low level drivers
> (they won't supply the source code) and C++ code to interface between
> the low level drivers and the sane backend code. "Yuck!"
> Okay, back to the drawing board. I go back to Lexmark and read their
> spec. sheet. Later, I find you really can't trust the information in the
> spec. sheet, which I find out after 4 rounds of emails with their help
> desk - I don't think they really understood  the difference between
> resolution and colour depth. The spec. sheet says 48-bit colour and
> 18-bit gray scale. Well, the scanner never returns enough data to
> support this. Lexmark's claim is that the scanner does this bitdepth
> internally and picks the best results from the sample, but only sends
> 24-bit colour or  8-bit gray scale back to the computer.

Sounds indeed like inflated marketing claims.

Part of the claim is probably true. A scanner I worked on has a
table that converts from raw 12-bit data to processed 8-bit data.
It is simply a lookup table that takes a 12-bit input data as an
index and outputs 8-bit data. Such a table is most useful for
applying gamma correction.
Gamma correction boosts the dark areas of an image. This operation
needs to be done while lots of bits are still available, if it is
done on 8-bit data it results in banding in the dark areas.
Besides gamma correction, the table can also be used to apply
contrast or brightness correction (basically any intensity transfer
function, for example even image inversion is possible).
There are probably three such tables, for the R G an B components.

Xsane can take care of supplying a proper intensity transfer curve
given a certain contrast, brightness and gamma setting. You just
need to tell it the dimensions of your table.

> Oh, as another interesting side bar - black and white scanning is not
> really black and white. It's a gray scale scan that gets converted to
> black and white. Also, the Windows driver lets you select a myriad of
> resolutions from 72dpi up to 9600 dpi. The scanner really only does
> 75x75, 150x150, 300x300, 600x600, 600x1200 dpi scans.

Sounds familiar too :)

> So, eventually I find enough information from various sources. I found
> the part number (rts8852C) of the scanner chip from the sane pages and
> the best usb sniffer to use from this mailing list. As I dug a little
> further I found there has been a lot of work already done on the
> predecessors to the rts8852, the rts8801 and rts8891. I'm not sure why
> this isn't part of the sane backends yet, but there is a sane web page
> at http://hp44x0backend.sourceforge.net/ for several HP scanners that
> use these chips.
> Finally after many Windows scans and log files at various different
> sizes, resolutions, and colour or gray  scale mode I start writing the
> backend. It only took me about 2 weeks before I could scan on Linux, but
> this is the easy part! What I soon discovered is something called
> CALIBRATION. At 600x1200dpi colour resolution the Windows driver
> actually does ~20 scans off to the side of the glass flat bed. I have no
> idea what its doing. By looking at the usb logs it appears to be setting
> the gain and the dc offset in a gradual fashion over several calibration
> scans.

Can you get a scan of what this part off to the side looks like?

The scanner I worked on had a black spot there and a white strip
over the width of the scan bed.
I used the black spot as a reference for a black pixel (this RGB value
needs to be subtracted from every scanned pixel). I used the white
stripe as a reference for white pixels. Every pixel along the width
is multiplied by a value that would make the white pixels from the
white strip exactly RGB=(255,255,255), thereby correcting for uneven
lighting or variations between pixels in the CCD.
These subtract and multiply operations are more accurate if
they are done before conversion to 8-bit. So in my scanner they are
done inside the scanner on the raw data. The subtract and multiply
factors are stored per-pixel in a table that is sent to the scanner
before the scan.
(again actually three tables, one table per R, G or B component)

> Now I'm faced with a dilema. I ask myself - "Do I continue and try to
> reverse engineer the Windows driver?" "Is this even legal?" I really
> can't see progressing any further without taking a peak, but I'd really
> rather not get sued. I've already looked at the .ini file used to
> configure the driver. I wish I'd found it earlier. It's got a lot of
> valuable information in it, which confirms what I've said above...

I've followed the sane devel mailing list for a couple of years now
and never read about anyone getting into legal trouble.

On one hand, it seems that the scanner makers have little to lose:
they provide the specs, we write the driver, for free!
On the other hand I think that some scanner makers may be under NDA
with the scanner chip makers, which makes it hard to open up the

> I'd really like to continue. It's actually become a bit of an obsession
> with me, but I'm also a realist.

I think that without detailed specifications or help from the
scanner chip manufacturer that a reverse-engineered driver
will always perform worse than the windows driver....

Kind regards,