Chapter 8: Tools for Search Engine Optimisation

Tools for Search Engine Optimisation

Sitemaps

A sitemap is simply a list of the pages on your site, which can make it easier for users and search engine crawlers to navigate your site. A sitemap will help a search engine to catalogue content on your site that it may not have found.

There are three different types of sitemaps – XML, RSS and TXT.

With XML (Extensible Markup Language) you are getting the most widely used format for a sitemap. Basically, what XML does is to define rules for encoding documents into a format that can be read by both humans and machines. Because XML requires you to have an open and close tag around your elements the files can be very large, but with XML you can share information in a consistent way.

Moving on to RSS or Really Simple Syndication (Rich Site Summary) you will see that this is a web feed format that provide information on constantly updating content in a standardised way. RSS, however, can be harder to manage.

TXT or text files, on the other hand, are extremely easy to use and simply contain plain, human-readable text. This option, however, does not allow you to add metadata for your pages so it not as powerful as using XML.

robots.txt

robots.txt is a file on your website that gives directions to the web crawlers that might visit your site. Using robots.txt allows you to tell the search engine which parts of the site you want to keep bots from crawling and specify the location of a sitemap file.

With robots.txt there are three commands that are available. Disallow prevents compliant robots from going into certain pages or folders. Sitemap will show the location of your website’s sitemap(s). Lastly, crawl delay specifies the speed that a robot can crawl through pages on a particular server.

On-Page Search Engine tags

A Meta Robots tag offers a page-level instruction to bots for search engines. This tag should be included in the header section of an HTML document.

The attribute rel=“nofollow” will let you link to a resource but takes out your “vote” for the purposes of the search engine. This literally tells the search engine not to follow the link and can be useful when linking to an untrusted source.

When you have two or more copies of the exact same content on your page but are under different URLs this can cause a search engine to devalue the content and its possible ranking. By using rel=“canonical” you can essentially tell the search engine which page is the authoritative version and that only it should be counted in the results.

Google Search Console

Google Search Console is an SEO research tool provided by Google.

Key features include:

  • Labs – This is useful for webmasters because it contains the Site Performance report that indicates how slow or fast your site is loading for traffic.
  • +1 Metrics – One way to get your content annotated in search results is when users share your content with the +1 button on Google+. This can be an important factor.
  • Site Configuration – This allows you to adjust site links. When you move from one domain to another it allows you to make address requests submissions, and you can test robots.txt files and submit sitemaps.
  • Your Site on the Web – This search engine tool offers key metrics like keyword impressions, top pages delivered in search results, linking statistics, click-through rates, and unique insights to your SEO through statistics.
  • HTML Suggestions – Title tags and issues with meta descriptions can be found here by looking for search-engine-unfriendly HTML elements.
  • Crawl Errors – These are reports by Googlebot when significant errors arise while crawling sites, such as 404s.
  • Malware – If any malware is found, Google will indicate it here. Malware can significantly hurt rankings and create a bad user experience.
  • Crawl Rate – During Googlebot’s requests during the crawl process, the speed is affected by the crawl rate (non-frequency).
  • URL Parameters – You can make your site more easy to crawl by Google by indicating information about URL parameters to Google, such as “sort=price”.
  • Preferred Domain – This is what the webmaster wants to use as their site’s domain (e.g. www.example.com or simply example.com)
  • Geographic Target – Here you can specify the regional target for your site. This is useful to help Google improve its search results for geographical queries by targeting users in a particular location.

Bing Webmaster Tools

This tool from Bing will help you to get the big picture view of how your site is performing in Bing search results. The items included in this overview are clicks, impressions, pages that are indexed and the number of pages that are crawled for your site.

  • Crawl Stats – Using this tool will help you determine how many pages on your site contain errors and how many times Bing has crawled your site.
  • Index – With this tool, you can look at how Bing indexes your pages on your site and have some control over this process. The settings are similar to Google Search Console and allow you to explore how your content on your site is organised on Bing. You can also submit and remove URLs from different search results, explore your inbound links and adjust different parameter settings.
  • Traffic – This tool in Bing Webmaster Tools shows you click-through data from not only Bing but Yahoo! search results as well. The report in this tool will show you your average position as well as how much it would cost you if you were to buy ads that target each word.

Keep in mind that learning about SEO for yourself is extremely important. While search engines are trying to provide better tools for webmasters so that they can improve their search results they can only do so much. The main responsibility still lies with marketers and webmasters.

Need Some Advice?