This Is The Goo I've Got

Colliding hadrons in my underpants since 2002

Daily

Dailies for Friday 30 May 2008

Post author By
Post date 2008-05-30
No Comments on Dailies for Friday 30 May 2008

How can I reduce the number of requests you make on my web site? – Yahoo! Search Help

I noticed that the Yahoo search engine spider was using up disproportionate quantities of bandwidth. In a nutshell, put the following two lines into your robots.txt:
User-agent: Slurp
Crawl-delay: 5

Leave a Reply Cancel reply