Better And Faster Search Engines

The goal of all search engines is to attain the most relevant responses as quickly as possible. When search engines calculate their search results, they are steered by an algorithm that assigns higher or lower values to features of Web pages. The most common search engines on the Net, such as Google, generate a gigantic single ranking based on a search of all pages available on the Net.

The algorithm that Ola Ågren has developed ranks pages, instead, on the basis of each relevant starting page, and includes pages that are directly or indirectly linked to by the starting page. Then a normalised mean value of the relevance of the various pages is calculated.

A page that has links to it from several different pages is therefore assigned a higher value than those that are found only once. In this way it is faster to find pages of interest. For ordinary standard algorithms it takes more than seven days to go through and rank Web pages in a certain database. Using his algorithm, Ola Ågren has managed to do this in 158 seconds.

What’s more, his algorithm has proven to yield the most relevant responses. He studied the relevance of hits in the top ten lists for three different algorithms: the one he developed and two variants of PageRank, the algorithm used by Google. He examined a total of 100 different expressions for all Nordic languages and English, including the expression master of engineering science (civlingeniör in Swedish).

The top ten lists always had some form of overlapping between the different algorithms, but they were never completely identical. Users were then asked to judge the relevance of the various hits, without knowing which search engines had generated the alternative responses.

“The users in the study found that the search engine I developed is better than the others in more than 60 percent of cases,” says Ola Ågren.

Besides search engines, the dissertation is also about methods for finding structures in huge masses of information, such as keywords and methods for extracting free text, such as parts of the documentation from the source code.

Thesis abstract.