The problem of search engines and keyword searches

Article from Jason Slater

Introduction

For my ongoing search engine research project I need to understand much more about search mechanics, and to gain a deeper understanding of where it is heading into the future. Search engines have come a long way since their humble beginnings out of directory listings and approachable and accessibly keyword search techniques have driven their popularity for finding information on the Internet (Li et al, 2008). However, perform a few searches and you may discover that using keyword searching for non-trivial information is as much a problem today as it was in the early days of search (Finkelstein et al, 2002).

Searching for non-trivial information can be broadly split into three areas (Torrey, et al 2009).

• Locating and navigating to sources of information
• Making sense of the content presented
• Engaging in the process of social seeking of information

When considering the future impact of search engine mechanics in the context of information retrieval there are three factors which may be useful in measuring success which are coverage of information, unbiased content, and user focus – the information should be presented fairly, be accurate and be accessible and relevant to the searchers needs (Datta, et al 2008).

When looking into search using keyword techniques it may be useful to talk about what we would do if search was not an option – this may offer some insight into how we look for information and the decisions we make in deciding which information is useful to us.

The starting point for the analysis is the question: if the Internet did not exist – what would be the process for finding new information?

For example, if I wanted to know more about black holes – what steps might I take?

I will probably break this down into a few steps and consider what I am looking for, where I might find more information, and why I need this information. Answering these three questions appropriately will offer some useful insight that may be able to apply later.

Step 1: What…?

So we start with the question What…? – What are we looking for? We already know that – we want to know more about black holes. Where next? The next thing might be to decide the format of the information I need. For example:

• Do I have just a passing interest? If so, I could simply ask someone.
• Am I writing an academic paper?  If so, I need researched, peer reviewed material.
• Do I need an image of a black hole? Could I use an image library?
• Is it for a competition? What level of detail do I need?
• Have I seen a black hole and wanted to find out if it was dangerous?
• Do I have concerns about black holes in my immediate vicinity?
• Is my interest similar to black holes but not exactly black holes?

The last three points start to clarify our requirement for information further and have indicated some new areas of information which might lead to further information.

Step 2: Where…?

There are many sources of information, ranging from local gossip or research academic papers. The next step in our process would be to decide where to start looking for this information:

• Ask someone close to me for more information
• Buy a book or magazine related to my interest
• Contact a professional who might have detailed knowledge
• Telephone someone – for example the local observatory
• Do a college course – this may take longer but could give a good grounding into what we are looking for
• Call someone out – a builder or pest control perhaps?
• Borrow a book from someone
• Visit the library
• Watch television

Hopefully, you may have noticed the “call a builder or pest control” point – what sort of black holes do I really need more information about? Now we are starting to explore the context of the question. Context and Clarification are becoming important factors in finding a solution to our problem.

Read More