SEO The Game

Normally, as SEO’s we try to concentrate on getting the search engines crawl and index as much of the site as possible. However, occasionally there might be reasons why you would want to remove content from search engines. For example, if spammers have infiltrated your site and put links to their sites.
There are a number of ways in which you can tell the search engine what to do.

.htaccess – password protect the content you really don’t want search engines to index. (you have to do this before they index the content.) This is a strong method of protecting the content inside

Not linking to the site from a page. This could work as there are no links from any other page for the search engines to follow, however it is not completely secure because there is a still chance that someone might visit the page and then visit another page which lists the referring page and thereby creates a link for the search engine to follow.

Robots.txt – a fairly powerful way of telling Google to not crawl pages. In addition to blocking sites there are also commands that can be used on individual pages but they should be used cautiously. However, for Google, even if parts of the site are blocked through the robots.txt file, a reference to them can still appear in the index. This can also be accompanied by a snippet taken from DMoz, so it could look like the page had been crawled.

No index – no index blocks any reference to the page in the search index. However their is inconsistency between search engines when interpreting this. Google will drop any reference to the page but Bing (Yahoo) will still show a reference.

Nofollow – can be used to remove a page from the index but it is a weak method as every path to the page must be no-followed in order to work.

Google Webmaster Tools – if URLs have been indexed, then you can remove them using this tool. It can be found in a tab under crawler access. It can be used to block a whole website, individual directory or just a single page. It is also reversible.

301 Redirects – redirects can be used to remove the content from Google. As the 301 redirect is made, the old URL should be eventually be removed from the search engine index. This is particularly useful as PageRank is also made from the old URL to the new URL.

The British people produce more web pages per head of its population than any other country in Europe. In the UK we write around 17 pages per person. At least this is the result of a survey by Greenlight. Although this does not say anything about the quality of those pages.
Perhaps this is not all that surprising as English is still the dominant language of the Internet (and spam) and the UK has a fairly developed infrastructure compared to many other countries in Europe.
The next most productive country is Germany with 10 pages per person while Greece produces the fewest number of pages at a mere, 4 pages per person.
This survey highlights the competitive nature of SEO in the UK compared to other countries in Europe, where the markets are still underdeveloped.

AJAX (Asynchronous JavaScript And XML)  is a combination of  Javascript and XML to process information without having to refresh the page. It is used increasingly in web applications and may websites and  search engines have great difficulty crawling  websites that use AJAX because they do not execute  Javascript.

With this being a modern and useful feature for visitors on websites, there is a considerable need to be able to crawl this information.

Sites that use AJAX produces URLs that look like, http://www.example.com/#state1 everything after the octothorpe is meaningless to a crawler.

Google have been researching methods to crawl AJAX sites and have developed a way  that uses the server to run its own JavaScript at the time the crawler visits, via a headless browser. (This is a browser that executes and processes the code on the site in the same way as a visitor might including all the javascript content but does not show the output on screen.)  To finally get the information the crawler can take a snapshot of the processed page from which it can extract the URLs required to make the site work  for visitors.

The good thing about this is that the crawler will see the same thing as a visitor and so everyone is happy.

Because the site needs to run its own code for the crawler, this would be an opt-in method to say which states the crawler is allowed to index but unless it is going to tax your server too much, who is going to mind? Especially if it gets more information for your site indexed.

For more information visit the Official Google Webmaster Central Blog.

Google labs have created another project which looks to be very useful. Google Fast Flip is a news reader that presents recent articles from well respected or well read news sources. Most striking thing about it is that the articles are presented on the same page allowing you to scan them very quickly. Clicking on one of the brings takes you to that story.

You can also search for the latest stories about a partiular topic and this will return stories that are particularly topical. How useful it becomes depends on the quality of the content from sites that are allowed to show news. It will be extremely annoying to see made for Adsense sites or the same story over and over again but think that it would be good for finding new sites to add to Google reader.

Getupdated

Just when it looked like Bing was going to take off, they have got the marketing so wrong. While people speak of Googling for something, it is far to soon for people to speak in a meaningful way about Binging. The truth is that most people still don’t realise that there is a Google alternative.
To try an promote the site, a viral video has been launched with the intention of being so bad that it is good. But, as with so much corporate advertising that tries to be with it and cool, it falls flat on its face. The video is shown below:

Be warned it is so bad that you will just think it is awful. The search engine is pretty good and this video will ensure that Bing will be used as an as a transitive verb as in, “The’ve binged it up now.”

Getupdated