[Qa-jenkins-scm] [Git][qa/jenkins.debian.net][master] another stab at blocking some bad bots
Holger Levsen
gitlab at salsa.debian.org
Tue Oct 30 18:28:27 GMT 2018
Holger Levsen pushed to branch master at Debian QA / jenkins.debian.net
Commits:
b0fe38dd by Holger Levsen at 2018-10-30T18:27:55Z
another stab at blocking some bad bots
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
1 changed file:
- hosts/jenkins/etc/apache2/sites-available/jenkins.debian.net.conf
Changes:
=====================================
hosts/jenkins/etc/apache2/sites-available/jenkins.debian.net.conf
=====================================
@@ -19,6 +19,17 @@
ServerName $name
ServerAdmin holger at layer-acht.org
+ #block bad bots with a 403
+ SetEnvIfNoCase User-Agent "facebookexternalhit" bad_bot
+ SetEnvIfNoCase User-Agent "Twitterbot" bad_bot
+ SetEnvIfNoCase User-Agent "Baiduspider" bad_bot
+ SetEnvIfNoCase User-Agent "MetaURI" bad_bot
+ SetEnvIfNoCase User-Agent "mediawords" bad_bot
+ SetEnvIfNoCase User-Agent "FlipboardProxy" bad_bot
+ SetEnvIfNoCase User-Agent "SemrushBot" bad_bot
+ SetEnvIfNoCase User-Agent "Sogou web spider" bad_bot
+ #end block bad bots
+
<Directory />
Options FollowSymLinks
AllowOverride None
@@ -69,6 +80,11 @@ Use https-redirect www.diffoscope.org
Use common-directives tests.reproducible-builds.org
RedirectMatch permanent "^/((?!debian/repository/debian/).*)" https://tests.reproducible-builds.org/$1
DocumentRoot /var/lib/jenkins/userContent/reproducible
+ <Limit GET POST HEAD>
+ Order Allow,Deny
+ Allow from all
+ Deny from env=bad_bot
+ </Limit>
</VirtualHost>
<VirtualHost *:443>
@@ -91,6 +107,12 @@ Use https-redirect www.diffoscope.org
DocumentRoot /var/www
AddDefaultCharset utf-8
+ <Limit GET POST HEAD>
+ Order Allow,Deny
+ Allow from all
+ Deny from env=bad_bot
+ </Limit>
+
Alias /userContent /var/lib/jenkins/userContent
Alias /robots.txt /var/lib/jenkins/userContent/robots.txt
@@ -150,6 +172,12 @@ Use https-redirect www.diffoscope.org
DocumentRoot /var/lib/jenkins/userContent/reproducible
AddDefaultCharset utf-8
+ <Limit GET POST HEAD>
+ Order Allow,Deny
+ Allow from all
+ Deny from env=bad_bot
+ </Limit>
+
Include reproduciblemap.conf
<Directory /var/lib/jenkins/userContent/reproducible/debian/artifacts>
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/commit/b0fe38dd6f103f29c46006bdb98768a5eccefbab
--
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/commit/b0fe38dd6f103f29c46006bdb98768a5eccefbab
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20181030/c69789d2/attachment-0001.html>
More information about the Qa-jenkins-scm
mailing list