Web Development Insight

Don’t get confused: a regular sitemap (links on a webpage which summarize your website) is not a Google sitemap.

A Google Sitemap file will inform Google of the URLs on your site, including the dates when you last changed them. If and when Google is ready to crawl your site, it will take this information into account and use it to optimize it’s visit. If it already knows your site a bit and you signal that you have changed one of those pages (added a new link to it or just fixed a misspelling), then it will go have a look as soon as your site is up again.

Microsoft and Yahoo have joined Google to adopt the sitemap as a standard. sitemaps.org has an FAQ on sitemaps along with the XML schema and a few other tidbits.

Publishing a sitemap will not get your site crawled more often; it will just optimize the search bot visit when it does crawl your site. Likewise, it will *not* get more of your site crawled, but it might concentrate on the more important parts. When it does crawl your pages, it will process them regularly, meaning that any content you have on it will usually get used for web search.

How to Create a Sitemap

Google Webmaster Tools offers a python script that generates a sitemap.

Another popular tool is the GSiteCrawler: it will crawl your site, take a look at all of your pages (and yes, it will make counters count, if your gallery program counts all visits) and use that information to make a Google Sitemap file. In a sense, you are looking at your site with the GSiteCrawler and taking that information so that Google does not have to do as much work (and can concentrate on the important parts).

One advantage of running a sitemap crawler is that if it gets stuck on your site, so will Google and other search indexers.

There are plug-ins for blogs, such as WordPress, that will generate a new sitemap every time a blog is created or modified and notify Google of the change.

Google sitemaps are there to help the search bot; not improve your website’s search result placement.

Doug

Yahoo and MSN are jumping on board Google’s sitemap idea. At sitemaps.org the first paragraph describes the use of sitemaps:

Sitemaps are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site.

The sitemap protocol is described on the site along with a comprehensive FAQ.
The upshot of the project is only one sitemap needs to be published for all search engines (if the others gravitate to its use).

Remember that using the sitemap does not guarantee web pages will be included in an index.

Doug