Internet and Technology

What is a Google Sitemap?

Google Sitemaps help Google’s search engine spider discover and index pages on your website. In its basic form, a Google Sitemap is a list of all the webpages on your website. When Google’s search engine spider reads this list, it then knows about all those webpages that are specified in the sitemap. Google Sitemaps come in two formats: xml sitemaps and text sitemaps. Both formats contain the addresses of all the webpages on your website. The XML version contains additional information about each webpage such as its last modification date and roughly how often it is updated.

How Does a Google Sitemap Help Me?

If your website does not have a Google Sitemap, Google’s search engine spider downloads a webpage from your website and scans through it looking for any links that it contains to other webpages in your website. Google’s spider then downloads all those newly discovered pages and repeats the process of scanning for links. Such download and scanning takes time. If you have a Google Sitemap, Google’s spider immediately knows about all the webpages on your website. Reading the Google Sitemap is considerably faster than having to download and scan each page. A Google Sitemap also helps if your webpages are not well linked together or not at all. In that case, without a Google Sitemap, it may take a while for some webpages to be discovered or discovered at all. Google Sitemaps eliminate that problem.

Does Google.com Index Everything?

The answer to that question is no. Google.com states, “we can’t guarantee that URLs from your Sitemap will be added to the Google index.” Even though Google.com does not guarantee that it will index everything that you specify in your Google Sitemap, a Google Sitemap should increase the opportunity that your webpages will be indexed sooner since Google will know about them sooner. If Google does not know about your webpages, they definitely will not be indexed.

If I Create a Sitemap Will It Hurt Me?

Google.com states, “In most cases, webmasters will benefit from Sitemap submission, and in no case will you be penalized for it.” Google.com uses the information contained in your Google Sitemap to learn about the structure of your website and to better schedule its search engine spider in the scanning (a.k.a. crawling or spidering) of your website.

How Do I Generate a Google Sitemap?

There are several tools available that you use to create Google Sitemap. Google.com itself even provides a sitemap generator written in the Python programming language. There are also websites where you type in your website address and its spider goes and scans your website to determine all your webpages; however, such scanning is time consuming since every page on your website must be scanned, and the process must be initiated by you. A faster way of generating a Google Sitemap is to use Google Sitemap generator software that runs locally on your website.

How Can I Automate Google Sitemap Generation?

Creating sitemaps can be an automated process. The simplest way is to install and use the sitemap.pl Google Sitemap generator software. Once you install this software in your cgi-bin directory, the software will automatically generate the Google Sitemap each time it is accessed. This software is of the type whereby you can “set it and forget it”. You can go about adding to your website and you do not have to worry about updating your Google Sitemap. The software works by scanning your website’s hard drive looking for files to include in your Google Sitemap. By directly accessing the hard drive rather than downloading webpages, the software very quickly generates the Google Sitemap. On a typical server, the sitemap.pl sitemap generator software finds about 500 webpages per second (that’s 2 ms/page).

How Do I Tell Google.com About My Google Sitemap?

There are two ways that you can use to tell Google.com about your Google Sitemap. The first method is the simplest and the quickest to do. In your robots.txt file, include a line that says “sitemap:” followed by the website address of your Google Sitemap. For example, the Google Sitemap of the bime.com website is located at http://www.bime.com/sitemap.xml thus its robots.txt file contains a line that states, sitemap: http://www.bime.com/sitemap.xml The second method involves logging into Google.com Webmaster Tools at http://www.google.com/sitemaps and adding your site to the Sites Dashboard, and then submitting your Google Sitemap. Once you add your site, click the “Verify” link and follow the instructions and you will gain access to additional statistics about your website and status information about the processing of your Google Sitemap.

In Summary, What Do I Need to Do?

1. Use a Google Sitemap Generator such as sitemap.pl2. Add your Google Sitemap to your robots.txt file.3. Add your site to Google Webmaster Tools4. Submit your Google Sitemap.

Having an Google Sitemap is a good first step to get your webpages indexed. And with an automated sitemaps generator, improve the possibility of your webpages being indexed and showing up in search engine results.

Three Ways To Index Your Site With Google Sitemaps [Difficult,
Hard, And Easy]

Google has recently implemented a program where any webmaster
can create a Sitemap of their site and submit it for indexing by
Google. It is a quick and easy way for you to keep your site
constantly indexed and updated in Google.

The program is appropriately called Google Sitemaps.

In order for you to best use Sitemaps, you must have an XML
generated file on your site that will transmit or send any
updates, changes, and data to Google. XML (Extensible Markup
Language)is everywhere these days, you have probably seen the
orange XML logo on many web sites and its often associated with
Blogging because Blogs use XML/RSS feeds to syndicate their
content.

Today RSS is known mostly as ‘Really Simple Syndication’ but its
original acronym stood for ‘Rich Site Summary’. XML is only
simple code like HTML and it is used to syndicate your content
to all interested parties.

And the interested party in this case is Google. By creating
Sitemaps, Google is really asking webmasters to take charge of
the indexing and updating of their sites. Basically, doing the
Googlebot’s job!

This is a ‘Good’ thing! With the steady influx of new web sites
growing rapidly, indexing all this material will become a
challenge, even with the resources of Google. With Sitemaps,
websmasters can now take charge and make sure their site is
crawled and indexed.

Please note, indexing your site with Sitemaps WON’T improve your
rankings in Google. You will still be competing with the other
sites in Google for top positions. But with Sitemaps you can
make sure all your pages are crawled and indexed quickly by
Google.

There are some other big advantages of using Google’s Sitemaps -
mainly you have control over a few key variables, attributes or
tags. To explain this as simply as possible, your XML powered
sitemap file will have this simple code for each page of your
site:

http://www.yoursite.com/ 1.0 2005-07-03T16:18:09+00:00 daily

Along with ‘urlset’ tags at the beginning and end of your code,
and an XML version indication – that’s basically your XML file!
File size will depend on the number of webpages you have.

Taking a closer look at this XML file:

location – http://www.yoursite.com – name of your webpage

priority – you set the priority you want Google to place on that
page in your site. You can prioritize your pages: 0.0 being the
least, 1.0 being the highest, 0.5 is in the middle. This is ONLY
relative to your site. It will not affect your rankings. Why is
this important? You have certain pages on your site that are
more important than others, (home page, high profit page, opt-in
page, etc.) by placing high priority on these pages, you will
increase their importance in Google.

last modified – when you last modified that page, this timestamp
allows crawlers to avoid recrawling pages that haven’t changed.

change frequency – you can tell Google how often you change that
particular page. Never, weekly, daily, hourly, and so on – if
you frequently update your page this could be extremely
important.

Why do I need a XML Generator?

In order for this XML sitemap file on your site to be constantly
updated, you need a Generator that will spider your site, list
all the urls and automatically feed them to Google. Thus
constantly updating your site in Google’s massive index or
database. Keep in mind, Google also gives you the option of
submitting a simple text file with all your URLs.

Now there is already a flood of these generators popping up!
Different ways of generating your XML powered sitemap file. More
are probably appearing as you read this. For your convenience,
three ways to generate your XML Sitemaps file are listed below:

Difficult – Google’s Python Generator

That’s a relative term, if you know your server like the back of
your hand and installing scripts doesn’t scare the bejesus out
of you, you’re probably smiling at the word difficult. Google
supplies a link to a generator which you can download and set up
on your server. It will cough up your sitemap XML file and
automatically feed it to Google. Google XML Generator

https://www.google.com/webmasters/sitemaps/docs/en/sitemap-

generator.html

In order for this Generator to work, Python version 2.2 must be
installed on your web server – many servers don’t have this. If
you know what you’re doing, this will probably be a good choice.

You don’t need a Google Account to use Sitemaps but it’s
encouraged because you can track your sitemap’s progress and
view diagnostic information. If you already have another Google
Account gmail, Google Alerts, etc. just use that one to sign in
and follow directions from there.

To submit your Sitemap using an HTTP request, issue your request
to the following URL:

www.google.com/webmasters/sitemaps/ping?sitemap=sitemap_url

Hard – A PHP Code Generator

This is a php generator that you can place on your server. This
generator will spider your site, and produce your XML sitemap
file. Download the phpSitemapNG and upload it to your server.
Run the generator to get your XML sitemap file and send it to
Google. PHP Generator http://enarion.net/google/

Again, this is only hard to do if you don’t know your way around
PHP files or scripts.

Easy – Free Online Generator

These Generators are popping up everywhere, and Google now keeps
a list of these ‘third party suppliers’ of generators on their
site. Find them here: http://code.google.com/sm_thirdparty.html

One of the easiest to use is http://www.xml-sitemaps.com/ and
you can index up to 500 pages with this online Generator very
quickly and it will give you the sitemap XML file Google needs
to index your site. It will go into your site, spider it and
index all your pages into an XML sitemap of your site. You can
download this file, Compressed or Non- compressed and make minor
changes such as setting the priority, changing frequency, etc.

Then upload this file to your site as sitemap.xml to the root
directory of your server i.e. where you have your homepage. Then
notify Google Sitemaps of your XML file and you’re in business.

Of course, the only drawback, if you constantly add pages to
your site you will need to also add these pages to your XML
sitemap file. This won’t be much of a problem unless you’re
daily adding pages to your site – then you will need something
like the PHP or Python generator to do all this for you
automatically.

Google is still the major search engine on the web so getting
your pages indexed and updated quickly is the major reason to
use Google Sitemaps. If you want your site to remain competitive
it’s probably the wisest route to take.

What is a Google Sitemap?

A Google Sitemap is a very simple XML document that lists all the pages in your website, but the Google Sitemaps program is actually much more important than that. In fact, the Sitemaps program provides a little peek inside Google’s mind – and it can tell you a lot about what Google thinks of your website!

Why Should You Use Google Sitemaps?

Until Google Sitemaps was released in the summer of 2005, optimizing a site for Google was a guessing game at best. A website’s page might be deleted from the index, and the Webmaster had no idea why. Alternatively, a site’s content could be scanned, but because of the peculiarities of the algorithm, the only pages that would rank well might be the “About Us” page, or the company’s press releases.

As webmasters we were at the whim of Googlebot, the seemingly arbitrary algorithmic kingmaker that could make or break a website overnight through shifts in search engine positioning. There was no way to communicate with Google about a website – either to understand what was wrong with it, or to tell Google when something had been updated.

That all changed about a year ago when Google released Sitemaps, but the program really became useful in February of 2006 when Google updated it with a couple new tools.

So, what exactly is the Google Sitemaps program, and how can you use it to improve the position of your website? Well, there are essentially two reasons to use Google Sitemaps:

Sitemaps provide you with a way to tell Google valuable information about your website

You can use Sitemaps to learn what Google thinks about your website

What You Can Tell Google About Your Site

Believe it or not, Google is concerned about making sure webmasters have a way of communicating information that is important about their sites. Although Googlebot does a pretty decent job of finding and cataloging web pages, it has very little ability to rate the relative importance of one page versus another. After all, many important pages on the Internet are not properly “optimized”, and many of the people who couldn’t care less about spending their time on linking campaigns create some of the best content.

Therefore, Google gives you the ability to tell them on a scale of 0.0 to 1.0 how important a given page is relative to all the others. Using this system, you might tell Google that your home page is a 1.0, each of your product sections is a 0.8, and each of your individual product pages is a 0.5. Pages like your company’s address and contact information might only rate a 0.2.

You can also tell Google how often your pages are updated and the date that each page was last modified. For example your home page might be updated every day, while a particular product page might only be updated on an annual basis.

What Google Can Tell You About Your Site

Having the ability to tell Google all this information is important, but you don’t even need to create a sitemap file in order to enjoy some of the perks of having a Google Sitemaps account.

That’s because even without a Sitemap file, you can still learn about any errors that Googlebot has found on your website. As you probably know, your site doesn’t have to be “broken” for a robot to have trouble crawling it’s pages. Google Sitemaps will tell you about pages it was unable to crawl and links it was unable to follow. Therefore, you can see where these problems are and fix them before your pages get deleted from the index.

You can also get information on the types of searches people are using to find your website. Of course, most website analytics tools will give this information to you anyway, but if the tool you use doesn’t have this feature, then it’s always nice to get it for free from Google.

But the best part of the Sitemaps program is the Page analysis section that was added in February of 2006. This page gives you two lists of words. The first list contains the words that Googlebot associates with your website based on content on your site. The second list contains words that Googlebot has found linking to your site!

Unfortunately, Google limits the number of words in each list to 20. As a consequence, the inbound links column is partly wasted by words such as “http”, “www”, and “com” – terms that apply equally to all websites (hey Google, how about suppressing those terms from the report?). That said, this list does provide you with a way to judge the effectiveness of your offsite optimization efforts.

When you compare these two lists, you can get an understanding of what Google thinks your website is about. If the words on your Site Content column are not really what you want Googlebot to think about your site, then you know you need to tweak your website’s copy to make it more focused on your core competency.

If, on the other hand your inbound links don’t contain any keywords that you want to rank well for, then perhaps you should focus your efforts in that direction.

Above all else, you really want these two lists to agree. You want your inbound linked words to match up to the site content words. This means that Google has a clear understanding of the focus of your website.

Additional Benefits of the Sitemaps Program

Google has even started notifying Sitemaps-participating Webmasters if they are breaking any of Google’s Webmaster Guidelines. This can be very valuable information if your site suddenly becomes de-listed on Google and you don’t know why.

Only Sitemaps participants can get this information, and it is only provided at Google’s discretion. In fact, Google will NOT notify you if you are creating worthless websites that offer no original content, or if you are creating thousands of doorway pages that are redirecting to other web sites. Google doesn’t want to give the sp@ammers any clues as to how to improve their techniques.

How Do You Get Started with Google Site Maps?

The first thing you must do is obtain a Google Account. If you already have a Gmail, Adsense, or Adwords account, then you are all set. If not, you can register an account by visiting the Google Accounts page (https://www.google.com/accounts/NewAccount).

Building your sitemap file is pretty easy to do if you are familiar with XML, and if you aren’t you can always use a third-party tool such as the ones that are listed on Google’s website (http://code.google.com/sm_thirdparty.html). Google also has a “Sitemap Generator” that you can download and install on your server, but unless you are fairly adept at managing Python scripts, you should probably stick to the third-party tools.

At any rate, once you have your Google Account and your Sitemap file built, the rest is very easy. All you have to do is:

Log into your account (http://google.com/webmasters/sitemaps/siteoverview)

Type your website’s URL into the “Add Site” box and click on “OK”

Click on the Manage Sites link for the website you are adding, and add your sitemap file to your account.

Google Sitemaps – An Excellent SEO Tool

Google Sitemaps help Googlebot quickly find new content on your website. They allow you to tell Google what’s important, what’s new, and what changes often. The tools provided to webmasters through the program can play a vital role in helping you understand how the search engines (especially Google) view your website.

Using this information you can dramatically improve the position of your website and quickly clear up any issues Google finds. You can also use the tools provided by Google to gauge the effectiveness of your off-site optimization efforts so you can better focus your time and energy on activities that bring you the most success.