Internet advertising: Paid search ads don't always pay off, study finds

"We found that when you turn off the paid advertising, almost all of the traffic that came through the paid search is just substituted by the other free channels," says Tadelis, associate professor in the Haas Business and Public Policy Group.

Tadelis conducted the study, "Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment," at eBay. The study was co-authored by Thomas Blake, an economist in the economics research team that Tadelis started at eBay, and former eBay economist Chris Nosko of the University of Chicago.

To measure the effectiveness of paid search, the researchers turned off eBay's paid search in 68 direct marketing areas in the U.S. In other words, if a consumer typed in the search term "white blouse" while online in these markets, he or she would only see the generic search results at the top of the list; not the paid ad that typically appears in a shaded box at the top of the search. She would not see any retail ads by eBay for "white blouse" but only from other advertisers who bid on the "white blouse" keywords.

At the end of 60 days, Tadelis and his colleagues compared sales of two groups: one group that received no paid search results and another group in which paid search remained untouched. Again, consumer sales as a result of the paid search showed no measurable increase off those who made purchases via unpaid channels (such as organic searches, or directly visiting eBay.com).

In order to ensure the robustness of their results, in a second experiment, the researchers also eliminated eBay's paid keyword searches throughout the country and then compared sales for that period to an equivalent period with paid search on.

"If advertising is indeed a strong driver of sales, we should have seen sales plummet," says Tadelis. "But the impact on sales was indistinguishable and not significantly different than zero."

Furthermore, for "brand" keywords such as "eBay" or other company name keywords, paid ads sit just above the generic search results. For example, a search for "Macy's" results in a Macy's free search below the Macys paid ad. Consequently, Tadelis says the paid search result adds no additional benefit to the advertiser. "It's not that clicking on the result caused engagement, it's that the intent to engage caused people to click on it," says Tadelis.

On any given day advertisers, including eBay, bid on millions of keywords. Tadelis hopes their work will encourage other e-commerce businesses to conduct this type of microeconomic research to better measure the impact of paid search traffic on the web.