[sane-devel] Passing all scanner usage through local saned

[Till Kamppeter]

Hi,

this sounds interesting. I am running into a similar problem.

Generally we have immutable distributions. The core system and the applications 
are in their own, isolated sandboxes, which could be Snaps, Docker containers, 
Flatpaks, ... With this, each application brings its own libraries and they 
often are of mismatching versions, and also any system-provided, centrally 
stored scanner drivers (SANE backends) cannot be dlopened as the core system's 
or a driver container's file system is not visible inside the application's sandbox.

IP connections can usually be easily defined between sandboxes/containers though.

So my idea, mainly with the all-Snap Ubuntu Core Desktop in mind, but applying 
to any immutable distro, is to use the same IP protocol as driverless network 
scanners and printer/scanner multi-function devices already use. These protocols 
are well-defined, established in the industry, and (therefore) stable across 
different software versions and implementations.

In my design (derived from driverless IPP printing) the applications come with 
only the sane-airscan (written by Alexander Pevzner, CCed) backend, the backend 
which supports driverless scanning, currently eSCL and WSD, later perhaps also 
IPP Scan. All modern driverless printer/scanner multi-function devices 
understand either eSCL or WSD (and IPP for printing). There are also some 
driverless standalone scanners around which support these standard protocols. 
This way the applications talk directly to the scanners, without need of any 
centralized scanner support in the operating system.

Also USB devices can be driverless. Here the IPP-over-USB standard comes to play 
which maps the USB device onto localhost, port 60000 and following (should be 
actually HTTP-over-USB, as it supports all HTTP protocols: IPP, eSCL, WSD, web 
admin interface).

Using industry-standard protocols also allows connections across operating 
systems and does not limit to only SANE (we did it more than a decade ago with 
CUPS, replacing CUPS' own broadcasting/browsing protocol by DNS-SD).

For scanners which need a driver, as they have a non-standard communication 
protocol, we introduce emulations of eSCL scanners, the so-called Scanner 
Applications. At the front they appear as eSCL (driverless) scanners and at the 
back they are talking with the actual scanner device, converting the protocols 
internally. Like a physical network device they advertise the presence of the 
scanner via DNS-SD (Avahi under Linux) and the eSCL protocol allows clients to 
poll the full set of capabilities (resolutions, scan sizes, ADF, ...) from the 
Scanner Application and also to conduct the actual scan.

One Scanner Application can contain any amount of scanner drivers and there can 
be any amount of Scanner Applications installed on the system. So we can have 
one Scanner Application retro-fitting the whole sane-backends package and other 
Scanner Applications issued by scanner manufacturers to support their devices. 
Internally Scanner Applications do not need to use SANE, they can accomplish 
their job with whatever the developer considers best. So we will have native 
Scanner Applications not using SANE internally and retro-fitting Scanner 
Applications using SANE.

Applications which scan and scanner drivers (Scanner Applications) can be in 
different sandboxes and so be installed independently on immutable (and also 
classic) operating system distributions, so especially one can add scanner 
drivers to an immutable distribution this way, and as each sandboxed package 
brings their own libraries, they can work with different libraries. So 
especially also the situation of packages using different libraries in Nix OS 
gets solved this way.

For the implementation, on the client side there is not much to do. One only 
needs to have a SANE-supporting application and ship it together with the 
sane-airscan backend.

On the server/driver/device side is more to do but we are well into it. As we 
come from the situation of printer/scanner multi-function devices, out emulation 
is actually also an emulation of such a multi-function device if we want to 
support a non-driverless multi-function device.

Several years back we (at OpenPrinting) have already started with printing. All 
modern printers are driverless. they do driverless IPP printing. AirPrint, 
Mopria, IPP Everywhere, and Wi-Fi Direct Print are all just flavors of 
driverless IPP printing. Here the printer always advertises itself by DNS-SD. 
Then clients can find the printer and poll the printer's capabilities from the 
printer via an IPP request and then print the job in a standard data format 
(PDF, but also some raster formats for cheaper printers), so there is no need 
for a driver (aka device-model-specific software or data, like CUPS filters 
and/or PPD files) for printing.

For non-driverless legacy or specialty printers we have introduced Printer 
Applications, emulations of driverless IPP printers. To create such Printer 
Applications there is a standard library, PAPPL (Printer APPlication Library) 
written by Michael Sweet, author of CUPS. It contains everything which every 
Printer Application needs: Daemon, handlers for DNS-SD and IPP, web admin 
interface, ... and Michael also ha written two native (no CUPS filters and PPD 
files hidden inside) Printer Applications, hp-printer-app (for legacy PCL 
lasers) and LPrint (for label printers).

I have written pappl-retrofit, an additional library for PAPPL with which one 
can easily wrap classic CUPS printer drivers and PostScript PPD files into 
Printer Applications (retro-fitting).

Currently Akarshan Kapoor (CCed), GSoC contributor for OpenPrinting last year 
and this year, is adding scanner support to PAPPL, so that with PAPPL one can 
make both Printer and Scanner Applications, even support multi-function printers 
with one single application. He is adding eSCL support to PAPPL (which before 
only supported IPP printing) and SANE support to pappl-retrofit.

This way we get a universal Printer/Scanner driver format which allows for 
distribution-independent binary package and for adding drivers not only to 
classically installed but also to immutable operating systems.

We are snapping these Printer and Scanner Applications and putting them into the 
Snap Store. Currently you find all free software printer drivers which usually 
come with Linux distributions in 4 retro-fitting Printer Applications and the 2 
native Printer Applications from Michael Sweet. Scanner Applications will 
hopefully come soon.

Printer Applications and Scanner Applications are not required to get 
distributed as Snaps, one can also just install them classically or put them 
into other containers.

At OpenPrinting we have this year also a GSoC project on OCI (Docker, ROCKs, 
podman, ...) containerization to cover most immutable distros which are around.

For the issues you mentioned, network scanners are available, those on modern 
network multi-function devices usually directly, without need of a Scanner 
Application. SANE-shared scanners we can support by the planned 
sane-backends-retro-fitting Scanner Applications.

Scanner Applications can be configured to share scanners in the local network or 
to restrict them for only local use.

The eSCL being an industry standard protocol it will be supported by any version 
of sane-airscan and PAPPL once it has scanning support. No changes between 
software versions are expected.

Links:

Printer Applications: 
https://openprinting.github.io/achievements/#all-free-drivers-in-a-ppd-less-world---or---all-free-drivers-in-snaps
Scanner Applications: https://openprinting.github.io/current/#scanner-applications
PAPPL: https://github.com/michaelrsweet/pappl/
pappl-retrofit: https://github.com/OpenPrinting/pappl-retrofit/
LPrint: https://github.com/michaelrsweet/lprint/
OpenPrinting in the Snap Store: https://snapcraft.io/publisher/openprinting
LPrint in the Snap Store: https://snapcraft.io/lprint
Scanning support for PAPPL: 
https://dev.to/kappuccino111/sandboxing-scanners-a-leap-into-the-driverless-realm-gsoc-23-report-3eci 
  https://www.youtube.com/watch?v=AAeUseU35Cc

    Till


On 09/05/2024 17:39, Guillaume Girol via sane-devel wrote:
> Hello,
> 
> I'm a downstream maintainer of SANE on NixOS, a linux distribution. I
> consider modifying NixOS to make it so all application using libsane
> would actually only use saned, and never load the backends to directly
> access scanners themselves. I send this message to the mailing list to
> know if I missed reasons it could be a bad idea.
> 
> Rationale
> =========
> 
> NixOS is quite different from typical linux distributions in that there
> is no globally installed library. /usr/lib does not exist and without
> special packaging care, dlopen() never finds anything. Each library is
> installed in a unique prefix /nix/store/unique-id/lib/foo.so so several
> versions of the same library can coexist. Exeecutable find the
> libraries they depend on via explicit RPATH. As a result I can have an
> old version of python in /nix/store/some-id/bin/python using an old
> version of glibc in /nix/store/other-id/lib/libc.so via RPATH and the
> rest of the system using a newer version of glibc in /nix/store/yet-
> another-id/lib/libc.so also via RPATH.
> 
> This does not work for SANE as it uses dlopen() to find backends
> depending on a single, global configuration file. For now we have an
> exception to the "no lib installed globally" design for SANE. However
> this is not perfect: if I use an old version of, say, simple-scan, it
> may dlopen() more recent version of some SANE backend, which depends on
> a more recent glibc than the version that old simple-scan brings with
> its RPATH. Thus dlopen() fails. In practice this means that software
> installed from the past or the future often fails to scan, whereas this
> usually works quite well for other functionnality.
> 
> Solution considered
> ===================
> 
> To solve this, I want applications using SANE to stop using dlopen() to
> access the scanner, but to use a network protocol instead. This is a
> bit like glibc can talk to nscd over a socket instead of dlopening nss
> modules.
> 
> The solution I'm considering consists in linking all normal
> applications to a stripped-down variant of libsane which only has the
> network backend and is configured to connect to a saned instance on
> localhost (instead of obeying what you would find in /etc/sane.d on
> typical distros). Each application comes with its own copy of this
> libsane, it's not global anymore. (Don't mind the disk space cost, all
> of NixOS works this way, this is part of the tradeoff). When the user
> configures their system for scanner support, a global instance of saned
> is setup which is linked against a full-fledged copy of libsane obeying
> the full configuration files you would typically find in /etc/sane.d.
> 
> When an old application attempts to scan, it uses its own, old version
> of libsane to talk to a recent saned which dlopen()s sane backends of
> matching version, so all works fine.
> 
> Possible issues?
> ================
> 
> Any issues with this design I missed? In my light testing, it appears
> to work quite fine. The problems I found:
> - all users can use the scanners instead of just users in the lp or
> scanner group previously. I suppose this can be solved with a firewall
> (firewalls can filter the uid of local sockets).
> - network scanners are not available. I propose a fix in
> https://gitlab.com/sane-project/backends/-/merge_requests/834
> - is the network protocol that saned uses stable across versions? In my
> testing, it is, but maybe it's by sheer luck.
> - maybe stuff I have not considered.
> 
> 
> Cheers,
> Guillaume Girol
> 
>