www robots.txt

[Karl Berry]

CVSROOT:        /web/www
Module name:    www
Changes by:     Karl Berry <karl>       08/06/16 23:02:28

Modified files:
        .              : robots.txt 

Log message:
        Crawl-delay: 4 to try to calm down MSN, #364760

CVSWeb URLs:
http://web.cvs.savannah.gnu.org/viewcvs/www/robots.txt?cvsroot=www&r1=1.1&r2=1.2

Patches:
Index: robots.txt
===================================================================
RCS file: /web/www/www/robots.txt,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -b -r1.1 -r1.2
--- robots.txt  13 Feb 2001 01:32:57 -0000      1.1
+++ robots.txt  16 Jun 2008 23:00:32 -0000      1.2
@@ -1,4 +1,5 @@
 # robots.txt for http://www.gnu.org/
 
 User-agent: *
+Crawl-delay: 4
 Disallow: /private/