University Experts Dispute Hawk-eye's Wimbledon Line Call

Hawk-Eye, and similar officiating tools which are used to supplement or replace decision-making by umpires and referees, are becoming an increasingly common part of televised sports coverage. But the new technology has also attracted its critics.

They include Wimbledon champion Roger Federer, who during his Final in 2007 against Nadal, claimed the technology was flawed. Nadal hit a ball which appeared to television viewers, to the umpire, and to Federer as impacting well behind the baseline, but Hawk-Eye called it in. Federer appealed to the umpire but the umpire accepted Hawk-Eye's judgment.

New research from Cardiff School of Social Sciences draws on the claims made by Hawk-Eye Innovations' on its website about the disputed line call. The machine reported that the ball nicked the baseline by 1mm. However, Hawk-Eye Innovations also report that the average error of the machine is 3.6mm. If the Cardiff analysis is correct, the errors can be even larger than 3.6mm on some occasions. The International Tennis Federation, which tests the machines for use, would accept that Hawk-Eye had passed its test if it called the ball in by 1mm while the true position was out by 5mm.

Researchers challenges whether aids such as the Hawk-Eye system can actually always be right in a new article.

Led by Professor Harry Collins and Dr Robert Evans, the team argues that such devices could cause viewers to overestimate the ability of any technological devices to resolve disagreement among humans. It also suggests that a more detailed understanding of how the device works could play a vital role in public education the benefits of which could spread to all technological decision making in the public domain.

Professor Collins said: "We used Hawk-Eye as the principal illustrative example as it is the most well-known of the commercial systems and is currently being used to make decisions in major tennis competitions such as Wimbledon. Technologies such as Hawk-Eye are meant to relegate line call controversies to the past, however, our analysis has shown that Hawk-Eye does not always get it right and should not be relied on as the definitive decision maker."

The paper puts forward the Automated Decision Principle. This states that automated sports decision aids should not correct but should reproduce human systematic errors -- the typical errors made by human judges and viewers such as calling a tennis ball `out' when it looks out to everyone even if the electronics suggest it might just have been in. At the same time, the devices should be used as they are now to correct or reduce human random errors, which come from lapses of concentration, an obscured view or very fast action - but the fact that the machine can also make mistakes should always be clear. If adopted, this Principle, would involve significant changes in the way devices such as Hawk-Eye are currently used in sport.

Professor Collins says: "To avoid any chance of misleading the public we believe that Hawk-Eye's conclusion should be accompanied by statements and/or displays of the size of the possible errors, as is normal in science. This would ensure that the public is much better informed as to the limits and possibilities of technology.

"Although such systems have not have been designed for this purpose, they could actually become a useful way to educate the public of the limits and possibilities of technology and serve to ask questions about the degree of certainty the public can actually attribute to scientific measurement and technological tools.

"What's more, even perhaps create a new role for sports commentators whereby they would interpret and explain levels of uncertainty for the viewer."

The analysis also concludes that Hawk-Eye might be in danger of unnecessarily changing the traditional nature of certain games because it does not take account of traditional areas of human systematic error when making judgements.

These systematic errors are an integral part of the sport. They include giving the benefit of the doubt to batters in the case of lbw or judging tennis balls to be out when skidding gives the appearance of their being out even if they have just clipped the line.