www.eamoncaddigan.net

commit 7a7c31cebfe82934cf144f978a8afbf3ffd7b9df
parent c7c6ce7dbb37d2e776df139eb494cc34c0d7cf0b
Author: Eamon Caddigan <eamon.caddigan@gmail.com>
Date:   Sat,  4 May 2024 14:57:54 -0700

Exclude TurnitinBot from crawing the site

I was never a fan of their service, but I was inspired by this older
blog post:
<https://blog.asgaard.co.uk/2012/10/13/turnitinbot-and-why-you-should-block-it>

Diffstat:
M
static/robots.txt
 | 
3
+++

1 file changed, 3 insertions(+), 0 deletions(-)
diff --git a/static/robots.txt b/static/robots.txt
@@ -1,2 +1,5 @@
 User-agent: GPTBot
 Disallow: /
+
+User-agent: TurnitinBot
+Disallow: /