We all know that search engines can identify pages on a website via its internal (and external) linking structure. However, the most comprehensive and accessible method of providing our website structure to search engines by far is via a valid XML sitemap that is uploaded to a search engine's Webmaster Tools (for example, Google Webmaster Tools).
Naturally, the development team behind Magento realized that manually creating an XML sitemap from all the ever-changing pages, products, and category URLs would be an impossible task. Therefore, they developed their very own XML Sitemap Generator.
In order to generate our XML sitemap, we must first configure its contents. Go to System | Configuration | Google Sitemap and configure Frequency and Priority for our main page types. We can also configure how often we want to generate our sitemap.
Depending on our Magento store, we may decide that our categories are the most important pages. They're our most optimized pages and we want search engines to index them first. Our next most important pages would be our individual product pages; we want those to appear in search engines for customers searching specifically for our product names. The page type with the least priority would normally be our CMS pages.
Note
As mentioned previously, the home page in Magento is classified as a CMS page; therefore, based on our specifications, it will receive a lower priority. In addition, the home page URL or <loc> will be set as http://www.mydomain.com/home, which is not how we want our home page to appear.
These are both problems that can be overcome via Magento extensions mentioned in Chapter 8, Purpose-built Magento Extensions for SEO/CRO.
The priority is simply a value that is passed to Google in order for it to prioritize the list of pages it will index; it will then (supposedly) do so programmatically.
Based upon our chosen SEO campaign, we would set the priority higher for those pages we are optimizing. Therefore, if we are optimizing our categories and products more than CMS pages (recommended) we would set their priorities to match the following:
Within Categories Options, set Frequency to Daily and Priority to
1.Within Products Options, set Frequency to Daily and Priority to
0.8(or anything less than 1 and more than we are about to set the CMS pages to).Within CMS Pages Options, set Frequency to Weekly and Priority to
0.25.Within Generation Settings, set Enabled to Yes, Start Time to 01 00 00 (01:00 a.m.), and Frequency to Daily, and enter your e-mail address into the Error Email Recipient field.
Click on Save Config.
In the end, we should have something that looks like this:
Note
In order for our generation settings to automatically generate our sitemap, the Magento CRON must be enabled. A quick tutorial on how to do this can be found here: goo.gl/q3ngaJ.
Our next step is to make sure that we have an XML sitemap that will be updated based on these settings. To do this, we need to first create one as follows:
Navigate to Catalog | Google Sitemap and click on Add Sitemap.
For Filename, enter
sitemap.xml.For Path, we can specify a path, but we would usually place an XML sitemap on the root of our website (enter
/).If we have multiple store views, we can enter a specific sitemap for each Store View (in which case we would change our filename to suit the convention, for example,
sitemap_en.xmlfor English).Click on Save & Generate.
This should generate an XML sitemap in our chosen path with our chosen filename. We can test this by visiting our path/filename in the URL, for example, http://www.mydomain.com/sitemap.xml.
Tip
If an error message appears informing you that the specified directory is not writable, please make sure that the folder specified under Path has sufficient privileges to allow the system to write a file—usually 775, or failing that, 777.
Once we have confirmed that our XML sitemap is set up and working correctly, we now need to make sure that it has been submitted to our chosen search engine—Google.
There are two ways to do this, but for safety's sake, we would usually perform both:
Open the
robots.txtfile and add inSitemap: http://www.mydomain.com/sitemap.xml.Log in to Google Webmaster Tools (www.google.com/webmasters/tools/), click on our website (or add our site if we need to create one), and then, within Crawl, click on Sitemaps and Add/Test Sitemap.