April 13th, 2010 by Susie

Search engine optimisation services generally try to get search engines to index as much of their clients’ sites as possible, to facilitate SEO link building and thus increase revenue. However, there are times when they have to do the opposite. A search engine optimisation company also doubles as an internet reputation management firm, after all.

If, for example, your site has attracted spammers or other malicious links, the URL needs to be taken down as soon as possible, while repairs are done and the links reconfigured. Another common reason is that a site or page has simply become outdated, and people linking to it are finding it detrimental to their searches. This can do untold damage to your online reputation management strategy, especially if you have clean links to another site bearing your name.

Google Webmaster Tools is one of those SEO tools that many SEO companies can’t do without. However, it can only be used for Google. It can be used to block an entire website, a single directory or even a single page. So long as the URL is indexed, you can use Google Webmaster Tools to remove it, and it has the benefit of being reversible. Robots.txt is another tool which tells Google not to crawl certain sites or pages. However, even if you block parts of your site with this command, they can still be referenced in the index.

Alternatively, a URL can be manipulated so it drops off the search engine radar until the problem is fixed. There are several ways an SEO company can do this. For example, they can use .htaccess. This is a way to password-protect content that you don’t want the search engines to index. This is a powerful tool, as it indicates to the search engines exactly who is allowed to access the content on specific directory files, thus protecting the content inside.

Another method is not allowing site linking from the page that’s opened. However, it’s not totally foolproof, as someone who opens that page link may then go to another page which lists the referral page. This will create a link that the search engines could follow.

A preferable option would be to use ‘No Index’, a piece of code that supposedly blocks all references to the URL in the search index. However, it is only foolproof on Google; Yahoo! and Bing will still show the reference. Nofollow is an alternative, but time intensive method of removing a page reference from an index. It’s ineffectual unless you are prepared to put a nofollow insertion on every path leading to that page.

Most of us have come across 301 redirects, even if we haven’t implemented them ourselves. A 301 redirect command is a permanent way of removing a defunct URL from the index, replacing it with a new one. It is doubly useful as Google PageRank is automatically transferred from the old URL to the new one, meaning your search engine optimization program doesn’t have to be restarted from scratch.

Link to us

If you want to link to this blog, copy and paste the following HTML code to your website.