How online video stream quality affects viewer behavior

They conducted the first large-scale study of its kind to quantitatively demonstrate how video stream quality causes changes in viewer behavior. "Video stream quality is a very big topic of interest," says Sitaraman. "Anyone who provides online video content, from the major news channels to sports and movie outlets, is worried about such things as do videos fail, how fast a video starts up, does it freeze, and how such loss of quality affects viewers."

"You want viewers to not abandon your content. You want them to watch longer, and return to your site often to watch more videos, resulting in more opportunities to show ads and to increase your subscriber base. The link between video streaming quality and viewer behavior has long been recognized as hugely important, but we couldn't study it with any scientific precision until now."

"The ability to collect large amounts of relevant data, plus new techniques we developed for the analysis were the game changers," says Sitaraman, who is also an Akamai Fellow. More than a dozen years ago, he helped build the Akamai network and helped pioneer content delivery network technology now used to stream a large fraction of the online videos on the Internet today.

This week in Boston, he presents research done in collaboration with S. Shunmuga Krishnan at the Association for Computing Machinery (ACM) Internet Measurement Conference in a talk titled, "Video Stream Quality Impacts Viewer Behavior: Inferring Causality using Quasi-Experimental Designs."

They report that viewers begin to abandon if the video does not start up within two seconds. Beyond two seconds, every additional one-second delay resulted in roughly a 5.8 percent increase in the abandonment rate. They also found that viewers are less tolerant of startup delay for short videos such as news clips compared to longer ones such as movies. Also of interest is that viewers with better connectivity are less patient and will abandon a slow-starting video sooner, while mobile users have significantly more patience than others.

Sitaraman also says viewers watch fewer minutes of the video if the video "freezes" while playing. A typical viewer whose video froze for 1 percent of its duration watched for 5 percent fewer minutes than a similar viewer whose video played smoothly without freezes. Further, viewers who experienced failures were less likely to return to the same web site to play more videos. A typical viewer who left the site after failing to play a video was 2.3 percent less likely to return to the same site within the week compared to a similar viewer who experienced no failures.

This study analyzes an unprecedented 6.7 million unique viewers from around the world who in aggregate watched 23 million videos for 216 million minutes over a period of 10 days. To conduct this research, Sitaraman and colleagues used a method more familiar to medical and social science research than to computer systems research. Through the collaboration with Akamai that currently delivers 15 to 30 percent of global web traffic, the researchers had access to millions of anonymous traces that contain information of how users watch videos and the quality they experienced.

To assess the impact of video quality on viewer behavior, the researchers assigned viewers to one of two "treatment" groups. One had a good quality video viewing experience while the other group experienced poor quality such as video failure to launch, delayed start or repeated freezing.

A viewer in a good quality group was then randomly matched with a viewer in the poor quality group so viewers were matched as closely as possible on geography, connection type, content and other characteristics of interest. Thus viewers in each pair differ on the quality of experience but are similar in other regards. "By comparing the difference in behavior of the paired viewers for hundreds of thousands of pairs, we are able to better isolate the impact of quality alone and exclude other confounding factors," says Sitaraman.

He adds, "A scientific understanding of the causal impact of streaming quality on viewers is a key piece of the puzzle for the success of online media. It helps computer scientists build better-distributed networks that deliver videos with higher user-perceived quality. It also helps video content providers better understand their users and the opportunities for video monetization. Research in this area is particularly important now as even traditional media like television are migrating quickly to the Internet."