Robots.txt, .htaccess | featured news

Relieving High Server Load by Blocking Search Bots

Coding

Over the years, whenever the site was slow to load up, I always went through the same routine. I checked traffic logs to see if it was due to a sudden increase in traffic. If not, I would look at the mysql slow query log to see if there were slow queries bogging up the database. If that wasn’t the culprit, then I would take a look at the access logs to see if there were any irregular activities from search engine bots.

 

Subscribe to this RSS topic: Syndicate content