What is wrong with full text searches

What is wrong with full text searches and why do I need an index where full text searches are available?

Where electronic text is available, whether in websites, eBooks or PDFs, providing full text searching is often thought all that is necessary to make the information accessible, but they have distinct shortcomings. Exactly the same problems occur with a poor-quality back-of-book index which has been constructed by identifying every occurrence of specific words throughout the text.

Problems with full text searches are:

  • Full text searches do not cope with homographs (words spelt the same but with different meanings), for example:
    • a book on boat-building might refer to the front of the boat (the bow), instruct that something is tied temporarily (with a bow) and that, as part of the construction, something should be bent into an arch (like a bow as in a bow and arrow).
    • an Internet search for information on the pop performers Madonna, Prince and the group Queen will give you millions of unwanted references to religious art and royalty.
    • an Internet search for “lead” will find “Lead in paint…”, “LEAD International: Leadership for Environment and Development…” and “12-lead ECG library …”
  • Full text searches do not cope with synonyms (words spelt differently but with the same meanings), for example:
    • bruise / contusion,
    • flammable / inflammable,
    • Religious Society of Friends / Quakers,
    • Çanakkale Bogazi / Dardanelles / Hellespont.
  • Full text searches do not distinguish between significant and trivial references to a topic, for example:
    • it would not be helpful to the index user seeking information on ‘children’ to direct them to the texts
      • “teaching adults (as opposed to children) …”
      • “the subject of children is dealt with later …”.
  • Full text searches do not pick up inferences, where a concept is discussed but the actual search term is not used, for example:
    • a biography which says about the subject that he was “a strong advocate of religious tolerance” and, elsewhere in the text, “he was not a member of the Church” should show both these references in the index under ‘religion’.
  • Full text searches do not cater for graphics – they may pick up the caption to a picture, but cannot access or assess the content of a picture, for example:
    • The New Yorker Cartoon “On the Internet, Nobody Knows You’re a Dog” (viewable here) is of interest to searches on security and identity, but probably not for searches about dogs.

After a full text search you have two unanswered questions:

  • Are all these hits really relevant ?
  • Have I missed any significant information ?

A properly constructed back-of-book style index, whether for book or a website, provides the answers. Every piece of information indexed is evaluated by a human indexer to determine to what questions it would provide a relevant answer, and it is indexed under those questions.

Furthermore, a back-of-book style index provides scope for value judgements to be made by the indexer and communicated, for example:

  • In a back-of-book index, page numbers for the main discussions on a topic may be shown in bold typeface
  • On a website index, the index heading may include explanatory text such as “unverified source”


Full text searches are quick and easy to supply, saving time and cost for the information provider. They are suited to rapidly changing material.

Back-of-book style indexes, in books and for websites, take more time and skill to produce but provide significantly improved access, saving time and cost, for the information seeker. They are suited to material which changes slowly, or only by addition.


1 thought on “What is wrong with full text searches”

  1. “Our friends” at Google should read this article.

    There are times that I need to find a specific email message from a client, and Google’s answer is to “just search for it”. The problem that Google does not seem able to comprehend is that users do not always know what keywords to use in their search. This is why I now use a third party email program that lets me sort messages by sender, by date, and by message thread.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s