Bug#732037: epiphany-browser: Appears to trigger arxiv.org's malignant robot detection

[Laurent Bigonville]

Le Fri, 13 Dec 2013 07:23:50 +0100,
Matijs van Zuijlen <matijs at matijs.net> a écrit :

> On 12/12/13 23:12, Laurent Bigonville wrote:
> > Le Thu, 12 Dec 2013 22:22:08 +0100,
> > Matijs van Zuijlen <matijs at matijs.net> a écrit :
> > 
> >> Package: epiphany-browser
> >> Version: 3.10.1-1
> >> Severity: normal
> >>
> >> If I access any of arxiv.org's pages using Epiphany, access is
> >> denied with the following text:
> >>
> >>   Access Denied
> >>
> >>   Sadly, your client "Mozilla/5.0 (Unknown; Linux x86_64)
> >> AppleWebKit/538.7 (KHTML, like Gecko) Chromium/25.0.1349.2
> >> Chrome/25.0.1349.2 Safari/538.7 Debian/unstable (3.10.1-1)
> >> Epiphany/3.10.1" violates the automated access guidelines posted at
> >> arxiv.org, and is consequently excluded.
> >>
> >>   If you believe this determination to be in error, see
> >>   http://arxiv.org/denied.html for additional information.
> >>
> >> I have asked the arxiv.org admin to unblock access, and this was
> >> apparrently done, but upon attempting access again, it was
> >> immediately denied.
> >>
> >> I have no problem accessing arxiv.org using Chromium or Iceweasel.
> >>
> >> The page mentioned in the error text gives the presence of
> >> 'accelerator' or pre-fetching software as a possible cause, but I'm
> >> not aware of the presence of such functionality in Epiphany.
> > 
> > I have a similar error when accessing this site with iceweasel 25.
> > I'm not sure we can do anything for this.
> 
> Since I have no problem accessing the site using Iceweasel (granted,
> it seems to be version 24), I find that rather strange. If indeed
> Iceweasel 25 would also be blocked, it is in fact rather worrisome.
> arxiv.org is a very valuable source of information, and should be
> accessible from Debian.

I did it wrong, I actually clicked on your link /denied.html instead of
going directly to http://arxiv.org/.

I can confirm that it's actually working for me with both Iceweasel,
epiphany 3.8 and 3.10, I tried from 2 different ISP's (.be).

> > I'm closing this bugreport.
> 
> Please consider investigating a bit more first.

Anyway, are connected behind some kind of proxy?

I've a different User-agent here:
User-Agent: Mozilla/5.0 (X11; Intel Mac OS X) AppleWebKit/538.1 (KHTML,
like Gecko) Safari/538.1 Debian/unstable (3.10.1-1) Epiphany/3.10.1

Could you please try to downgrade libwebkit2gtk-3.0-25 to the version
from unstable?

I'm not even sure why they are putting such restrictions, the user
agent is so easy to fake...

Cheers

Laurent Bigonville