10 Ways to Increase Your Site Crawl Rate

1.) Update your content often and regularly (and ping Google once you do)

2.) Make sure your server works correctly: mind the uptime and Google Webmaster tools reports of the unreached pages. Two tools I can recommend here are Pingdom (pingdom.com/) and Mon.itor.us.(http://mon.itor.us/)

3.) Mind your page load time: Note that the crawl works on a budget - if it spends too much time crawling your huge images or PDFs, there will be no time left to visit your other pages.

4.) Check the site internal link structure: make sure there is no duplicate content returned via different URLs

5.) Get more back links from regularly crawled sites.

6.) Adjust the crawl speed via Google Webmaster tools.

7.) Add a sitemap (though it’s up for a debate whether the sitemap can help with crawling and indexing issues, many webmasters report they have seen increased crawl rate after adding it).

8.) Make sure your server returns the correct header response. Does it handle your error pages properly?

9.) Make sure you have unique title and meta tags for each of your pages.

10.) Monitor Google crawl rate for your site and see what works and what not:

access crawl stats via Google Webmaster tool.

Take advantage of this Great Wordpress Plugin that tracks crawl rate for Google, Yahoo and MSN. (Blogstorm.co.uk/wordpress-crawl-rate-tracker/)

We can’t force Googlebot to visit you more often - what you can do is to invite it to come.


You can Read the complete Guides of this theory at
http://www.searchenginejournal.com/10-w ... rate/7159/

Thank you