Keyword Research

What is the point of search engine optimisation (SEO) if you have a web site and you are optimising for terms that no one searches for. At the same time, SEO on terms for generic terms is also futile. There must be a happy medium which if you do your homework, you can find search terms that will give you a good amount of traffic and conversions.

When you are starting your website consider what it is that you want people to search for to reach your web site. As an example, consider the fictional company Gibson’s Hydrochloric Acid. They make chemicals. What sort of terms should they be considering?

Generic versus Specific Keywords

At the most generic level they would like to be found for: chemicals, acids, alkalis. These may have lots of searches but return  unrelated categories including: Wikipedia, university chemistry departments and ‘The Chemical Brothers’, etc.

More specific searches such as chemical companies, chemical manufacturers, chemical suppliers, chemical supplies, chemical products are in the right area and could be good to optimise for to bring in business.

They might want to search for their specific products such as Oluem, Hydrochloric acids, Sulphuric acid, Prussic acid, etc..

The more specific the search in the fewer search results returned and it is more difficult to predict what users will search for. In practise, these so called, ‘long-tail searches tend to appear without prompting if there is a lot of good and useful content on your site.

Finding Keywords

The obvious way is to guess at what keywords you would like your page to be returned for. Try to think like one of your customers who may not have inside knowledge of your industry or field of work. They may use more simplistic language that you may not think of.

The second way is ask other people to think of keywords they would search for to find your products or website. There are also programs which will generate related words to a keyword you type in. Finally, you can use Google to search for the keywords that your competitors might be using. A little known modifier is the tilde ‘~’ which will also return search results for similes and words that closely match your search query.

For example: ~physics will return results on physics and also science. The related words are displayed in bold within the search results.

Keyword Value

In an ideal world, a keyword would describe your product exactly, have lots of searches and none of your competitors would be trying to get visitors using that that term. It is very unlikely you will find something like this but it gives us a clue as to what to look for when choosing a keyword:

In broad terms, the keyword value = number of searches/the competitiveness of the terms.

Estimating Competition

So now we need to find ways of estimating the number of searches and competitiveness. To find the number of searches made we can get use tools such as WordTracker, Adwords etc. which give statistics for the number of search made for particular terms taken from data from the search engines. WordTracker uses the DogPile search engine as a source of its data. This has less than 1% of the search engine market. From this information we can estimate the numbers of searches for the other search engines using their market share.

In addition, we can look at the search results in Google. When the search results for a particular term are returned we notice that there are so many thousand, million results return. However, for any search the first thousand results are visible. This will be our first estimate of competition. This pretty basic. We can do better.

I would say that if a website had the search term in its title it would more likely to be competing for a term than if it just was returned in the search terms. Fortunately, using the allintitle: modifier in Google we can find out how many search results have the search term in the title.

Another more specific way of searching is the allinanchor: modifier. When used with the search term it will return results which contain the search text as the anchor text. This is also a good indicator of the competition.

Putting it all together we can create a weighted average which takes into account the search term results, the all in title results and the all in anchor results using a weight term. The allintitle having the greatest weight, then the allinanchor: and finally the ordinary search results.

Weighted Average <x>  =  c₁N_in-title +c₂N_in-anchor+c₃N_unmodified

and

Σ_i c_i=1 where i =1, 2 or 3.

The finial estimate gives a good indication of the competition side of the search results, especially for more specific keywords in our middle range. The final values of c_1,2,3 are the result of educated guesswork. One might call this the ‘Art of SEO’.

We define a keyword index (KWI) which measures the ratio of the search volume to competition.

KWI = Log(Search Volume/Competition)

The log is used because the search volume can vary overs such a large range of values. Negative numbers mean that the competition is greater than the search volume.

The smaller the competition compared to the search volume or the greater the search volume the larger the keyword index value.  It is a simple matter to use a spreadsheet with your guesses at keywords. Ordering the values of search volume in descending order along with the KWI we can see the best keywords to choose by selecting the largest search volume with the highest KWI.

In theory this should be an indicator of which keywords to select and provides a semi-scientific way of choosing your keywords.