Keep Google, Yahoo, MSN, and other Search Engines informed with an XML Sitemap
If you are like me, you wonder what's with Google. Google seems to find blog posts with out much trouble, but when it comes to other pages, it seems to just ignore them and let them languish. If Wordpress is only a small section of your website, then a sitemap becomes an essential tool for informing Google and other search engines that your pages exist and should be indexed.
Google will index your content sometime before the cows come home
While Google's bots will eventually find any content on your website linked by other pages, you'll find that Google will not quickly index non-blog content especially without a Sitemap.
You can find information on the structure and coding for an XML Sitemap which works for Google at http://www.sitemaps.org/
Consider an XML Sitemap written in PHP using your database
You can write your sitemap file with .php extension as long as the Sitemap content uses XML as shown. Making a script to create your Sitemap is a good idea because as your website grows, it will become harder and harder to keep a manually updated sitemap current. Basically you query your database's posts and articles and then echo that information in XML. What? Your website is just plain HTML pages? You could probably use a script that spiders your site and makes an XML file, but you'd have to run it frequently.
A Sitemap script that is the same file that Google fetches is the best choice because it would be current everytime the search engine requests it.
Consider an online XML Sitemap creator
A third option would be to use a free Sitemap spider service online. They will spider up to 500 pages as they are found from the homepage of your website. This, like other "spider" script sitemap creators, has the weakness of just adding what they find which could easily be duplicate content due to variables and search options and other URL issues that will also give you sitemap errors and warnings.
Avoid duplicate content
When creating an XML Sitemap remember that you don't want to include duplicate content, so if there is more than one path to a specific post, article, or product on your website, just include the best path where the content is going to be found more long term.
Consider making a Sitemap Index
you'll also find on the above mentioned site, information regarding the creation of a Sitemap Index. The Sitemap Index is also XML (but you can have a php script make it). You can submit the Sitemap Index file to Google Webmasters Tools as your Sitemap and Google will then check every sitemap listed inside that file.
There are a couple great benefits to using a Sitemap Index. First, you can make modular simpler Sitemap files and list them or not as needed. Second, you only have to submit one Sitemap file per site.
There is a limit to the size of a sitemap index as well as the number of links, so even if you have a small website now, it wouldn't hurt to start off with two files, your Sitemap Index which you'll submit to the search engines and your actual sitemap file. It will only take a couple minutes more to start off right, and you'll save plenty of time you'd have lost later submitting separate files to each search engine.
Ready for the next step, add your Sitemap to your Robots.txt file
While you should submit your Sitemap or Sitemap Index to Google, Yahoo, and other search engines, consider also including it in your robots.txt file. It is very simple. Just add:
Sitemap: http://yoursite.tld/sitemapfilename.xml
If your website is mywebsite.com and your sitemap index is sitemap.xml then you'd add:
Sitemap: http://mywebsite.com/sitemap.xml
Unfortunately you cannot use relative URLs, so you can't just say /sitemap.xml
I really wish they'd allow relative URLs for those of us with multiple websites, but I guess they just didn't think that far ahead.
No comments:
Post a Comment