[sane-devel] Backend activation and configuration (was Re: sane-backends release 1.0.26 schedule)

[Olaf Meeuwissen]

Hi Louis,

Sorry for the delay in replying.
I managed to fat-finger your mail into the trash :-(

Louis Lagendijk writes:

> On Sun, 2017-05-07 at 07:34 +0900, Olaf Meeuwissen wrote:
>> Hi all,
>>
>> Louis Lagendijk writes:
>>
>> > On Sat, 2017-05-06 at 15:55 +0200, Wilhelm wrote:
>> > > Am 06.05.2017 um 14:27 schrieb m. allan noah:
>> > > > On Sat, May 6, 2017 at 5:52 AM, Louis Lagendijk <louis at fazant.net> wrote:
>> > >
>> > > From a scanbd point of view it would be suffcient to have either:
>> > >
>> > > 1) an env-var e.g. SANE_CONFIG_FILE which is normally unset and all
>> > > sane-applications use the compile-time setting. scanbd can set this
>> > > to an alternative (e.g. scanbd-dll.conf) file before starting saned.
>> > >
>> > > 2) give saned a -c <file> option
>> > >
>> > > I would prefer 1)
>> >
>> > Well 2 is not an option in my opinion as there is no way for saned to
>> > pass an argument to the rest of saned. That is why I implemented 1).
>>
>> I don't understand why saned would have to pass arguments to the rest of
>> saned.  Can you explain?
>
> A terribly confusing typo: I meant sane (or libsane that by default
> runs dll.c. Libsane does not have the possibility to pass any arguments
> to it.  So saned could not pass the argment to libsane (or dll.c for
> that matter).Or it would have to use a environment variable anyway.
> that is what I tried to convey.

Okay, now I understand.  You're right, frontends have no way to pass any
arguments to the *backend* other than via environment variables.  I have
run into this myself (at the office) when writing the (third-party, C++)
utsushi backend.  While you can configure a SANE_Handle anyway you like,
using sane_control_option(), there is no SANE API to dynamically
configure a SANE *backend*.

# Hmm, maybe something for a next version of the SANE API?  Talking
# about which, scan button support should be included too!

>> > The change to an alternative dll.file was driven by the fact that when
>> > packaging scanbd you want to minimize the amount of configuration to be
>> > done by the user.
>>
> That would not solve the dll.d issue though, sigh (see below for more
> details).
>
>> There's post-installation scripts that can take care of most of that and
>> pre-removal scripts can restore things back to how they were.  As far as
>> I understand, the only thing the user needs to configure are the desired
>> actions when a button gets pushed.
>
> This would not work for dll.c[onf] updates on rpm based systems at least:
> new versions of config files will be stored as dll.conf.rpmnew.  So at
> least on rpm based systems not touching dll.conf is desirable. A
> restore of the dll.conf from a scanbd removal can not restore the
> dll.conf to the latest dll.conf.

Hmm, that sounds more like a problem with rpm to me.  Everything under
/etc is sysadmin territory.  If a sysadmin makes changes, these should
not be clobbered.  Putting the new file in *.rpmnew takes care of that
all right but is there no way to notify the person doing the upgrade?

Debian based systems ask what you want to do when it detects a changed
conffile.  Of course, this kind of interaction may not be desirable when
things have to be done non-interactively but I believe that's supported
too (by doing things the RPM way).

# It's not supported yet(?), but it'd be great if this supported merging
# of changes by the chunk!

> Hence my preference to make an dll-override (see below for an
> alternative solution) and leave dll.conf alone (and only load the
> backends mentioned in that alternative dll.conf file and ignore dll.d
> if an override is specified. I am still unhappy about the inconsistent
> behaviour in that case though).

I don't really understand why you need to ignore dll.d when using this
alternative dll.conf.  What's wrong with enabling third party backends?

> I only wish that dll.d was only loaded from a directive in dll.conf.
> Dll.d is a nasty problem. Any better suggestions for that?

The dll.d support is a very convenient mechanism for third-party SANE
backends.  Think about how you'd handle the dll.conf editing for those.
Imagine if every single backend in sane-backends came in it own binary
package, as if it were third-party.  How would you manage dll.conf?

If a sysadmin installs a backend, I'd assume the purpose is that it'd
get used by SANE frontends.  Dropping a one-liner file in dll.d is all
that is needed to activate the backend.

So even if you have scanbd installed, you'd want to be able to use any
third-party backends, no?

>> - in /etc/scanbd/sane.d/
>> - add symlinks to all files (and directories) in /etc/sane.d
>> - replace the /etc/scanbd/sane.d/dll.conf link with a copy
>> - disable the net backend in /etc/scanbd/sane.d/dll.conf
>> - in /etc/sane.d/dll.conf disable all backends
>>   You could prefix all lines with '#scanbd ' for example.
>> - in /etc/sane.d/dll.conf enable the net backend
>>   I would also add a note that admins should make changes in
>>   /etc/scanbd/sane.d/dll.conf instead.
>> - in /etc/sane.d/net.conf make sure localhost is enabled
>>
> Copying dll.conf + symlinking the rest should be ok. I am happy with
> this part of the solution.

Good.

> Modifying sane config files I am not sure about. I seem to recall that
> modifying other packages config files is not allowed or at least
> frowned upon for fedora. So this would still be a manual action for
> the user.

While I agree that modifying conffiles automatically is to be frowned
upon in the general case, the modifying that I suggested has no effect
on the *observable* behaviour of SANE frontends (as long as scanbd does
not crash and burn).

> Postinstall for scanbd could do the copying/linking as you propose.
> This solves the scanbd side of things. The problem is on the sane
> (default) side as we have dll.d to deal with.
>
>> After that scanbd can just set SANE_CONFIG_DIR=/etc/scandb/sane.d and
>> do its thing.
>>
>> On the off chance that sane-backends package upgrades add/remove conf
>> files, you should probably also add a trigger to update the symlinks.
>>
> Again, I need to check the rules.
>
>> Did I miss something?  If not, why would we need to complicate the dll
>> backend's configuration file handling if this can be handled fairly
>> easily by package post-installation and pre-removal scripts?
>>
>> The only "wart" in the above is the pile of symlinks.
>
> I am not too worried about that part.

Me neither (just have a look at /etc/rc?.d ;-) but I thought I'd point
it out for the aesthetically inclined among us.

>> But please read on as I am open to modifying the dll backend's conffile
>> handling.  Just not in the upcoming 1.0.26.I think it's all too last
>> minute and could use some more thought and discussion.
>
> I fully agree on not including this now. I started my initial mail
> stating that I realized that I was very late taking this on. I said in
> the report on Alioth that it could take a long time and most likely
> would miss 1.0.26.
>
> If the scanbd package drops a dll2.conf file in
>> > SANE_CONFIG_DIR there is no need for manual configuration.
>> >
>> > I am now implenmenting a #pidfile directive for in dll2.conf that will
>> > check for the pidfile and fail processing of dll2.conf and fallback to
>> > dll.conf.
>>
>> Hmm, #pidfile looks like a comment to me (and the few backends whose
>> code I looked at).  You probably should use something without a #.
>
> pidfile does not work anyhow, when scanbd is started from systemd it
> will not have a pid-file. I should have known as I wrote the systemd
> units for scanbd.

# Glad I'm on Devuan these days, where this is not an issue (together
# with another pile of, eh, "pötterware" issues).

> I need to find a better start for directives, but I
> selected # as it is now that start of a comment and could not interfere
>  with any backend names (although the all start with [a-z] for now.
> Using # as start for any directive made sure that I would not create
> new problems. But I am open for any suggestion. I consider what I have
> now more as a proof of concept, so we can change whatever we need.

Quite a few, if not all, SANE backends treat # in their config as a
comment.  Our .desc files treat ; as a comment too (as does Lisp).  So
those are out and we want something that's not valid in a C identifier.
At the same time we want to convey that the presence of said PID file
indicates that the dll-override.conf file takes effect.  It also has to
be simple to implement, right?  Also assuming that *only* scanbd needs
special casing for now, something like

  %if pidfile=/var/run/scanbd.pid
    net
  %else
    # the regular dll.conf content
  %endif

where the conditional would be equivalent to

 sh test -e /var/run/scanbd.pid

could be included in sane-backends (with suitable changes to dll.c *and*
scanbd signalling when to skip this test, of course).

# The % is a LaTeX comment, btw.  Feel free to use something else.
# LaTeX uses \ to start commands but seeing that's an escape in C and
# most other places ... maybe not a good idea.

>> > The way I envisage this working is:
>> > The package drops a dll2.conf file containing:
>> >
>> > #pidfile=/var/run/scanbd.pid
>> > net
>> >
>> > so sane users see only the net backend if scanbd is active.
>>
>> Or there is a stale pidfile lingering around.
>>
>> I doubt that there is much you could do about that.It would be up to
>> systemd (cough), the init script or scanbd to make sure the pidfile is
>> removed, even in the case scanbd crashes.
>
> Yeah systemd or an init script could create another file and remove it.
> But that still leaves the possibility that scanbd would crash or hang
> without the init system knowing about it.
>
>> BTW, this could be simplified if scanbd creates that dll2.conf file as
>> /var/run/sane/scanbd.conf at startup (and removes it on exit) and the
>> dll backend modified to prefer that file unless explicitly told to
>> ignore it (by scanbd via an environment variable).  When ignoring, the
>
> Why not create a /var/lib/sane.d where scanbd or any other program can
> place a dll.conf at start and remove it at exit?

What if two (or more such) programs need files with different content?
You don't want all of them to create their own dll.conf and clobber
what's there already, right?  Or remove the file created by a process
that started *after* the process that's exiting now.

# Are there any other programs right now that need this?  If not,
# dll.conf or any other name is fine, for now.

BTW, /var/run/ is a better location if those files are only supposed to
be around when a program is running.  It's for stuff that isn't supposed
or expected to survive a reboot.  Things in /var/lib/ are supposed and
expected survive a reboot.

Actually, this /var/whatever/ approach only works for programs of which
there will only be a single instance (think daemons) at any time.  If
there is a remote possibility of running the same program twice or more
at the same time, you need a different solution.  If these programs can
use the same file, maybe a static config file in /etc/$program/ would be
more appropriate.

>> regular conffile handling kicks in and dll.conf is searched for in all
>> the directories given in SANE_CONFIG_DIR if defined.  If SANE_CONFIG_DIR
>> ends with a path separator, : on Unix, $PWD and $sysconfdir are searched
>> as well.If SANE_CONFIG_DIR is not defined, the search looks in $PWD
>> first and $sysconfdir next.
>
> Let's just add /var/lib/sane.d in the search path?

Eh, /var/run/sane.d please.

# Coming back to other programs also using this, how would the dll
# backend know which of the files in this directory to prefer?

>> # BTW, I just noticed that the use of $PWD is open to abuse by frontends
>> # that call chdir() before sane_init() and backends that chdir() before
>> # searching for their conffile.
>>
> Should we not completely remove $PWD on Unix then?

+1

Fortunately this is *not* in the SANE API spec (so could be changed
rather easily) but it will be *breaking* behaviour that people may have
been relying on for just a bit short of about two decades :-(

> It seems to be a security liability anyhow (just imagine what happens
> if something just places a config file in the directory where the user
> running a program?

Calling chdir() to a location with a known "bad" file is easier ;-)

>  Adding ~/.config/sane.d to the search path might be a better solution
> for the user that wants an own configuration.

Something below $HOME would seem right.  IIRC, XSane puts stuff under
~/.sane.  The XDG guidelines[1] (hmm, more pötter stuff) would rather
have this some place under $XDG_CONFIG_HOME, with a default of
$HOME/.config/.  I'd vote for $HOME/.sane/ because of XSane precedent,
but we'd have to work out some naming conventions to accommodate the
needs of both frontends and backends (irrespective of the directory
location).

 [1] https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

>> > The default dll.conf remains unchanged, but net should be commented out
>
> But we then should ignore dll.d

Why do you want to ignore third-party backends when using scanbd?

>> > Scanbd uses dll.conf (sets SCAN_CONFIG_FILE=dll.conf)
>>
>> In my /var/run/sane/scanbd.conf scenario, this is where scanbd needs to
>> signal the dll backend to ignore that file.
>>
> The only configuration required from the user is:
>> > - comment net from dll.conf if not already done
>> > - populate net.conf with localhost
>>
>> That's still food for post-installation and pre-removal scripts.The
>> idea being that you want to use scanbd if you install it.
>>
>> BTW, you want to add localhost to net.conf.  There may be other hosts
>> configured already.
>>
>> > I changed the name of the alternative config file to dll-override.conf
>> > (but remain open for better alternatives).
>>
>> I think that /var/run/sane/scanbd.conf is pretty self-explanatory.  You
>> could also go for /etc/sane.d/scanbd.conf but that might be just a tad
>> confusing as there is a /etc/scanbd/scanbd.conf as well.
>
> See above: /var/lib/sane.d /dll.conf for overrides may be a small
> burden for scanbd and is open for other users. And it removes a lot of
> the burden for sane. when scanbd starts it just creates that dll.conf
> file and we are done.
>
> Thanks for the fruitdull discussion guys!

I'd like to return the thanks.  Not because I don't want them but
because you deserve some too!  ;-)

The discussion has become a bit fragmented and hard to follow perhaps
but overall I feel there is something "deep" that is not addressed by
the SANE API standard's backend mechanism.  Maybe we've come to a point
where we need to review whether the SANE API standard still covers the
needs of today's scanners and our user's use cases in a suitable way.
After all, your cell/smart phone of today has many times more computing
resources than a high-end desktop at the time the SANE API standard was
formulated.

# Heck, today's fridge beats a 20-year old desktop hands down now!  And
# it *scans* its contents to let you know you're out of milk (or beer!)
# to boot!  Here's wondering if it's using SANE under the covers ... ;-)

On the other hand, I do realise that many of you (and me) also want a
solution (or kludge) now rather than sometime in the future.  Let's take
a step back and focus on something that works with the inventive kludge
that scanbd provides in the face of the shortcomings of the SANE API
standard.

Perhaps Wilhelm's SANE_CONFIG_FILE approach is the simplest solution for
the time being after all and perhaps we should simply leave the more
"elegant" solution(s) for an updated SANE API standard.

Hope this helps,
--
Olaf Meeuwissen, LPIC-2            FSF Associate Member since 2004-01-27
 GnuPG key: F84A2DD9/B3C0 2F47 EA19 64F4 9F13  F43E B8A4 A88A F84A 2DD9
 Support Free Software                        https://my.fsf.org/donate
 Join the Free Software Foundation              https://my.fsf.org/join