[Surfraw-devel] custom searches

[Ian Beckwith]

Hi,

So, instead of finishing off the release, I went and had an Idea.

Elvi that search by doing "google site:something.org search terms"
can now be configured to choose which search engine to use.

By setting SURFRAW_customsearch_provider to either google, duckduckgo or
ixquick, they can use that search engine for those searches.

ixquick doesn't seem to work very well, not sure what's happening but I
think it only supports one site: term and/or one inurl: term, but we
need one site: and multiple inurl: terms. I might remove it.

Can anyone suggest more search engines that support site:/inurl:
semantics?

I've defaulted custom search to using duckduckgo

All three search engines appear to support the same site: and inurl:
arguments, but there are subtle differences in semantics.

You can't do inurl:refman/en/5.1, you have to do
inurl:refman inurl:en inurl:5.1 and for google the latter
has to be inurl:5\.1. ixquick doesn't completely work
no matter what I try, see above.

Good luck finding docs for the above.

Awk was whinging about escape characters when I tried passing
'inurl:5\.1' so I changed that bit of code to use sed instead.
As much as possible the code isolates per-search-engine-quirks
in the code.

Oh, the url-encoding char for '\' was wrong, fixed.

To use w3_custom_search in an elvi, instead of w3_browse_url, use:

w3_custom_search -s=site.com -u=urlparam -u=urlparam "$w3_shquoted_args"

see mysqldoc for an example.

I accidentally entered POSIX shell-quoting hell while doing this.

The only was I could get it to work while still preserving
"strings with whitespace in" was by manually adding 'quoting'
then eval'ing commands. I think the only other way is to
switch to #!/bin/bash and use arrays.

It's tempting, bash would make several things easier and reduce the
amount of shelling out to awk and sed, but I hate to reduce portability.

new elvi:

* mdn - simple mozilla dev network elvi
* phpdoc - uses w3_custom_search
* mysqldoc, pgdoc - use w3_custom_search, take arguments for db version

I also converted netbsd, openbsd and slinuxdoc to
use w3_custom_search

I wrote pgdoc as penance for writing mysqldoc :)

The above could all really use more testing, especially of the quoting
stuff. I'm out of time for now but when I get a chance I'll add some
tests and docs for the above, finish off preparing for the release
and create a release candidate tarball.

cheers,

Ian.

-- 
Ian Beckwith - ianb at erislabs.net - http://erislabs.net/ianb/
GPG fingerprint: AF6C C0F1 1E74 424B BCD5  4814 40EC C154 A8BA C1EA
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 237 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/surfraw-devel/attachments/20130930/56573c06/attachment.sig>