Internet as new frontier in collecting data on the mind

But psychologists are also looking to the Internet as a new source of information about the mind -- and an Indiana University researcher is on the forefront of those developing the tools to make it happen.

Josh de Leeuw, a graduate student in the IU Bloomington College of Arts and Sciences' Department of Psychological and Brain Sciences, is the creator of jsPsych, a free open-source software platform that employs a common Web technology to conduct psychology experiments over the Internet. The program allows psychologists without significant programming skills to deliver tasks common to research on the mind through a Web browser.

"Conducting psychology research online is appealing for a number of reasons: faster data collection, lower costs and improved anonymity of subjects and experimenters," de Leeuw said. "Internet users are more demographically diverse than the population conventionally sampled for behavioral research in psychology -- a fact that could contribute to results that better reflect the population at large."

This means the current pool of psychology research volunteers -- generally, college students -- could expand since participants would no longer need to travel to a researcher's lab to perform tasks common in psychological research. Using jsPsych, a psychologist could present anyone with Internet access with on-screen stimuli and then collect responses. Study instructions and consent forms could also be delivered electronically.

"Online experimentation is the bleeding edge of psychology research, and many believe online studies will play an important role in addressing the so-called 'replication crisis' in psychology," said Ben Motz, senior lecturer and director of undergraduate instruction in the Department of Psychological and Brain Sciences, who has collaborated with de Leeuw. "Josh's work is at the epicenter of this -- and it's all the more impressive that he's developed this platform as a graduate student."

The replication crisis refers to a growing concern in experimental psychology -- and the larger scientific community -- about the drop in studies able to confirm previous work with experiments that achieve the same results using the same methods, as well as the increased risk of data manipulation in studies with small sample sizes.

Reducing costs and other barriers to psychological research could increase the number of replication studies, as well as the overall number of volunteers and data produced, in turn creating stronger results.

A few psychology researchers are already using online services to conduct research. One of those tools is Amazon Mechanical Turk, a service from the retail giant that enables anyone to hire online workers to perform short, simple tasks.

Others remain hesitant to employ these methods due to worries about the quality of data collected outside the highly controlled environment of a research lab. To address this concern, de Leeuw recently published a study in the journal Behavior Research Methods investigating whether one specific issue among brain researchers -- the potential delays in response introduced by the use of online technology -- represents a significant barrier to conducting research online.

That study looked at differences between Matlab's Psychophysics Toolbox -- one of the most popular software systems for psychological experiments -- and jsPsych.

To measure the speed that information is displayed on-screen and its effect on reaction time, de Leeuw asked 30 volunteers to sit in front of a projection screen and hold a thumb-activated switch in each hand. If volunteers saw the letter "N," they clicked a button with their dominant hand. If they did not see the letter, they clicked with their other hand. Volunteers repeated this experiment 400 times, with only a 45-second break in the middle.

To prevent weariness from affecting the results, de Leeuw set up the screen to project two images simultaneously -- one generated by jsPsych, the other by the traditional system -- enabling response times to both systems to be measured simultaneously.

In psychology research, response times may be used to discover which mental operations are more difficult than others, or to measure how uncertain people are about a decision. They can help distinguish between different theories of perception, memory, decision-making and social cognition. Psychologists refer to the study of the relationships between stimulus and response as "psychophysics." Measuring response times is only one of many applications of jsPsych.

The study revealed that jsPsych produced delays of 10 to 40 milliseconds compared to laboratory-based software. Three other statistical parameters measured showed no reliable difference between the systems.

"This is an extremely small lag time -- it's roughly the same as might be caused by other variations in experimental setup, such as a change in keyboard, mouse or monitor -- and it is constant between different experimental conditions," de Leeuw said. "In the context of a real human experiment, both systems are nearly identical."

He added that results are also applicable to other software packages that use JavaScript, the only programming language supported by all major Web browsers that does not require additional software to run.

"Hopefully this study will convince more people that you can, in fact, conduct psychophysics in a Web browser," de Leeuw said. "We're looking at technology with potential to transform how research on the brain gets done."

In addition to Motz, IU researchers contributing to the study were Richard Viken, Chris Eller, Michael Bailey, John Kruschke, Tony Walker and Alex Shroyer. Equipment and technical assistance were provided by the IU Advanced Visualization Laboratory. This research was supported by a National Science Foundation Graduate Research Fellowship Grant (DGE-1342962).

A tutorial on jsPsych is available online. De Leeuw will also deliver a tutorial on the use of the program at the 2015 Cognitive Science Society meeting July 22.