If there's anything I particularly hate when it comes to SEO, it's the meta keywords tag. I so wish it had never been invented. It's practically useless, yet people still obsess over it. In this article, I'll explain more about why you shouldn't worry about it except perhaps for misspellings, as well as which search engines support it.
The meta keywords tag is one of several of meta tags that you can insert into your web pages to provide search engines with information about your pages that isn't visible on the page itself. For example, my Meta Robots Tag 101: Blocking Spiders, Cached Pages & More article covers how you can use a different meta tag -- the meta robots tag -- to block pages from being indexed. Users don't see this information (unless they look at your source code), but search engines do.
Meta Tags & Your Header
Meta tags go within the header area of your web pages. A typical head might look like this:
Welcome To Shoe Central!The header is the section that begins and ends . Between those elements, in our example, you have these tags:
Title: The text here becomes the title that is shown in search engine listings, in most cases.
Description: The text here is text that search engines sometimes use as a description for your web page when listing it (a meta tag lesson for another time).
Robots: This particular tag is configured to ensure that the page isn't described using the a description that the Open Directory might have for it (Meta Robots Tag 101 explains this more).
Keywords: This tag is the topic of this article, so read on!
History Of Meta Keywords
I've long written about search engines and meta tags, but I have never been able to pin down exactly who created the meta keywords tag. There's a December 1995 internet draft memo that's the earliest and most authoritative mention of the tag I know of. It says:
The spaces between a comma and a word or vice versa are ignored....
These 'keywords' were specifically conceived for exhaustively and completely catalogue the HTML document. This allows the software agents to index at best your own document. To do a preliminary indexing, it's important to use at least the http-equiv meta-tag "keywords".
Sounds good, right? Like this is designed for the search engines to use? The issue is that HTML specs like these (especially drafts) are not necessarily used by the search engines. They can use them, ignore them or build upon them as they see fit.
As it turns out, several of the major search engines got together in May 1996 to talk about meta data. That meeting gave birth to a common standard for the meta robots and the meta description tags. As for the meta keywords tag, it was discussed, but no specification emerged.
Despite no specification, both Infoseek (later Go.com, these days no longer crawling the web) and AltaVista (now owned and powered by Yahoo) offered support for the meta keywords tag in 1996. If you looked at their help files at the time, they encouraged site owners to use the tag. Inktomi (now owned by Yahoo) also provided support when it began operations later in 1996, and Lycos (no longer crawling the web) added support in 1997.
That year -- 1997 -- was the last year that the meta keywords tag enjoyed support among the majority of major crawlers out there (4 out of 7 - Excite, WebCrawler and Northern Light, also crawling the web that year, did not support it).
Support Dies Off
When new search engines emerged in 1998, such as Google and FAST, they didn't support the tag. The reason was simple. By that time, search engines had learned that some webmasters would "stuff" the same word over and over into the meta keywords tag, as a way of trying to rank better. At the time, search engines didn't rely so heavily on link analysis, so page stuffing like this was more effective. Alternatively, some site owners would insert words that they weren't relevant for.
In July 2002, AltaVista dropped its support of the tag. That left Inktomi as the only major crawler still supporting it, causing me to somewhat famously in the SEO world to declare the tag dead, since it was no longer a major ranking factor for even Inktomi:
Traffick.com's Andrew Goodman wrote recently in an essay about meta tags, "If somebody would just declare the end of the metatag era, full stop, it would make it easier on everyone."
I'm happy to oblige, at least in the case of the meta keywords tag. Now supported by only one major crawler-based search engine -- Inktomi -- the value of adding meta keywords tags to pages seems little worth the time. In my opinion, the meta keywords tag is dead, dead, dead. And like Andrew, good riddance, I say!
Since that time, Inktomi was rolled up into Yahoo, which continues to support the meta keywords tag as part of its Yahoo search engine. Or does it?
Search Engine Rep Confusion
Sigh. But if this question still coming up wasn't depressing enough, then the search engine reps starting responding with a load of confusion. To paraphrase:
No, we don't support it. Well, we read it. We read it, but it doesn't matter. Actually, maybe we don't read it.
Even Evan Roseman from Google said at one point that Google reads the meta keywords tag, suggesting no doubt to some that Google uses the tag.
To be clear, Google doesn't. I'll prove it further below, but it doesn't, OK?
I gave Evan (hopefully) some good humored hassle afterward for saying this. He's at least the second Google rep to declare this on panels I've moderated in as many years, and the problem is that the engineers (from any of the search engines) often take the question too literally.
Indexing Versus Retrieval Versus Ranking
To understand, let me talk about three different things a search engine does when it crawls and lists your page:
Indexing: This is where the search engine effectively makes a copy of your page. The search engine is going to read and store the HTML content it finds -- all of it. Evan was right when he said that the meta keyword tag is indexed by Google. Google knows that the tag exists and has recorded what's in it. But that doesn't mean it does anything else with it.
Retrieval: This is where the search engine finds all the matching documents relevant for what you searched for. Most of those documents will actually have the words you searched for on them, in the sections that the search engine searches against (there are some exceptions, such as when anchor text is used to find pages. Google Now Reporting Anchor Text Phrases, Google Kills Bush's Miserable Failure Search & Other Google Bombs and Google Declares Stephen Colbert As Greatest Living American explain more about this). While the search engine has recorded the entire page, it won't search against everything indexed for retrieval. In other words, Google will look to see if words you searched for appear in the body area of a document, but it will NOT look in the meta keywords tag for matching words. The keywords tag, while indexed, is not used for retrieval at Google. At Yahoo, it is.
Ranking: This is where the search engine looks at all those documents retrieved for a search and puts them in order of most importance, according to its algorithm. Retrieval (or what information research professionals call "recall") is about finding everything). Ranking (or what the IR folks call "precision" -- see Tim Bray's excellent On Search: Precision and Recall document) is about getting the best stuff up to the top. Yahoo, while using the tag for retrieval, really doesn't assign much weight to it for ranking.
Testing For Retrieval
Back to my panel experience. Since the reps were unclear, I declared to the audience that I'd just go out and test it again myself. It's literally been about five years since I've last tested the tag, because I (and many others) feel it is so useless. There are better things to do with our time. But since that question needs a big old stake to the heart, I rolled up my sleeves and got cracking.
On the Search Engine Land home page, I inserted this meta keywords tag:
I had searched for all of these words on the four major search engines of Google, Yahoo, Microsoft and Ask and found no pages that matched. If these search engines made use of the meta keywords tag, I'd know in short order, if my page started coming up.
The tag went up on August 28. I then needed to wait until I could see each search engine had the most current version of my page (Squeezing The Search Loaf: Finding Search Engine Freshness & Crawl Dates explains more on how to do this).
Google: No
It took two days, until August 30, for Google to show the latest version of my page in its index. I searched for each of the words, and my home page didn't come up. The meta keyword tag was not used for retrieval and thus not supported.
Microsoft Live: No
It took five days, until September 2, for Microsoft to show a version of my page with the meta keywords tag on it. As an aside, Microsoft is kind of annoying. It will say something like this in the cached copy of the page:
This is a version of http://searchengineland.com/ as it looked when our crawler examined the site on 9/2/2007. The page you see below is the version in our index that was used to rank this page in the results to your recent query. This is not necessarily the most recent version of the page - to see the most recent version of this page, visit the page on the web.
If you glance quickly at the date, you might think the page has been revisited fairly recently. But as the text explains, it might be older. Indeed, when I looked on September 2 (as is the case today), the copy of the page in the index was as of August 30, as I could tell from the stories shown.
As with Google, I searched for each of the words, and my page didn't come up. The meta keyword tag was NOT used for retrieval and thus not supported.
Yahoo: Yes
It took two days, until August 30, for Yahoo to have my latest page. Searches there did bring up the home page for all words. So the meta keywords tag IS used for retrieval.
Ask: Yes
Ask took the longest to show the most current version of my page, not reflecting the changes until today. Actually, when I look at the cached copy even now, it says that the page is from August 13 and uses a redirection URL rather than my http://searchengineland.com address.
Still, I can tell Ask has a version with the meta keywords tag on it since I'm getting back my home page when searching for words in that tag. As with Yahoo, the meta keywords tag IS used for retrieval.
Should You Use It? Sure, For Misspellings
So there you have it -- half of the major crawlers (Yahoo & Ask.com) DO support the tag. Should you begin using it? My advice would be only for misspellings and really unusual words.
As explained, the tag can help with retrieval. A word in the tag is treated as if it were a word visible on the page itself. Now that's handy for misspellings. For example, say you're writing about Basset hounds. You suspect some people might misspell the name as Bassett hounds, adding an extra T. You could misspell the word yourself on the visible page, but that makes you look bad. You could insert the word and then try to hide it using CSS styles or putting it in the same color as the page background. But this type of "hidden" text is generally against search engine guidelines.
Enter the meta keywords tag. Just do this:
Now you've got the misspelling on your page in a "legal" means that will be read by Yahoo and Ask. You're still out of luck for Google and Live.com, but two out four ain't bad.
But I Want To Rank!
What about ranking better with the tag. I mentioned already that many experienced SEOs don't find it useful. Believe me, if just putting a single word into that tag was going to rank your page better, everyone would be doing it. Instead, search for anything on Yahoo or Ask. You'll see plenty of pages ranking well for words without those words appearing in the meta keywords tag. And if you do see the words in the tag, it's more due to coincidence -- the words also appear in the body copy, in the title tag and often in links pointing at the page. The words in the meta keywords tag aren't the primary reason the page is ranking well. Promise.
Back to our Basset Hound example. Sure, you can add the correct spelling to your meta keywords tag. Go ahead, if you want. Just understand that it is not likely to make you rank any better than if you didn't include it at all. Moreover, beginners are especially likely to spend far too long worrying about getting the "right" words in the meta keywords tag rather than just writing good body copy.
Comma Conundrum
One of the most common questions I used to get way back in the old days was over using commas in the meta keywords tag. Consider these options:
Sigh. See why I hate this tag so much, when I've had to deal with people wondering about commas and spaces and variations like this. Let's take it from the top, as to the motivations behind these versions:
This is someone who thinks that each word should be on its own, separated by a comma and with a space in front of the next word.
This is someone who thinks that getting rid of the spaces means they can squeeze in more words.
This is someone who thinks that if there are particular phrases they want to be found for, those phrases should be together and set off by commas.
As with three, but losing the spaces to squeeze in more words.
Similar to three but thinking you don't need commas at all.
This is Mr. or Ms. Paranoid. They're concerned about saying any word too often. So they lose the commas, restrict repetition and hope that proximity will help (IE, put "basset" behind "hound" rather than in front and maybe you'll still show up for "basset hound."
Which way should you go? I'd suggest number three, for these reasons:
Yahoo has long recommended using commas and in particular supported them as a way to separate out distinct terms for those in their paid inclusion programs. I'll update this page with the latest advice, but commas still seem to make sense.
Spaces just make things look nicer, and you shouldn't be shoving a ton of terms in the tag anyway. How long is too long? No idea! In the past, the search engines just wouldn't index content beyond around 250 to 1,000 characters. Maybe I'll test this in the future.
You do want phrases kept together. "bassett, hound" is probably going to be seen as "bassett hound" anyway, but why risk it?
Other Uses
I mentioned that misspellings were a key use for the tag. You could also use it for synonyms. For example, if you have a page all about shoes and you never say "footwear," you could put that word in your tag. However, it's far better if you just find a way to make use of the word in the body copy itself. That text is retrieved by all the major search engines, not just some.
Aside from synonyms, perhaps you have a page that's all Flash or all images. Use the meta keywords tag to describe the page. Just remember that you're still not likely to rank better than other pages that have textual information. Search engines are textual creatures. Give them what they want.
Some Official Guidelines
The W3C has guidelines (and here) in HTML 4.0 about meta data and search engines, while the XHTML specs don't get into it at all. Ignore the specs. YES, IGNORE THE SPECS. Some of them are wrong; some are outdated. The only thing I can see that they explain is the difference between these:
See how the second tag ends /> rather than > in the first? As best I can tell, this is because a meta tag is an "empty element" in XHTML, where there's not a "start" and a "finish" (as with a paragraph element:
is the beginning, with
the end). Empty elements in XHTML need that /> format.
I haven't tested things without the />, but there are so many (so very, very many) pages out there not following that syntax that it is virtually certain Yahoo and Ask will read the tag either way. Doing it fresh? Do it /> style. But don't go back and start changing things.
Aside from that, if you want to know how a search engine deals with meta data officially, you go to the search engine itself. Ask's webmaster guidelines don't mention the meta keywords tag, so that leaves Yahoo:
Yahoo Quality Guidelines: "Metadata (including title and description) that accurately describes the contents of a web page." This is telling you don't lie with your keywords. Don't insert words that aren't somehow related to the topic of your page.
How do I improve the ranking of my web site in the search results?: "Use a 'keyword' meta-tag to list key words for the document. Use a distinct list of keywords that relate to the specific page on your site instead of using one broad set of keywords for every page." Note that it doesn't say you'll automatically rank better by doing this. Also, unique words for each page would be my advice, as well -- but do NOT worry if you decide to use the same set of key terms on each of your pages. It isn't that big of a deal.
Looking for the exact format that you should use for the meta keywords tag from Yahoo? You know, commas, spaces and all that. Sorry -- they don't provide it, which is another sign you're probably worrying too much about it.
Labels: best keywords, design for seo, design search engine, how to leggally hide your text, no keyword for search engine, renta-coder, search engine loves keywords, search engine optimisation, webdesign
