Reading the minds of pilots on the fly

Consider what was racing through her mind: where to land, the speed and altitude to maintain, how to aid an injured passenger and comfort a bewildered crew. It's likely Shults was experiencing "cognitive overload," said Frédéric Dehais, PhD, a professor at ISAE-SUPAERO in Toulouse, France, and an expert in flight safety.

The efficiency and safety of human-machine systems depend on the cognitive workload and situational awareness of human operators, according to Hasan Ayaz, PhD, an associate research professor in the School of Biomedical Engineering, Science and Health Systems at Drexel University.

"Unfortunately, many human-machine interfaces expose users to workload extremes, diminishing the operator's attention and potentially leading to catastrophic consequences," Ayaz said.

An ideal human-machine system would actually be able to read its operator's mind in real-time, to know how well he or she was paying attention or able to process new information. Such a system may sound like the makings of a sci-fi movie. But Ayaz and Dehais, along with a team of researchers at Drexel and ISAE-SUPAERO, have now successfully measured the brain activity of pilots in real-time using functional near-infrared spectroscopy, or fNIRS. Their results were published this week in Frontiers in Human Neuroscience.

Unlike traditional functional magnetic resonance imaging (fMRI), in which a person is resigned to a large machine while lying down, Drexel's portable fNIRS system is worn like a headband, making it ideal for measuring brain activity while study subjects move around freely in their natural environments.

The fNIRS system measures "the brain at work" by monitoring blood oxygenation changes in the prefrontal cortex -- the area underneath the forehead that is involved in cognitive functions such as problem solving, memory, judgement and impulse control. When first learning a new task, for instance, this area of the brain is highly activated. However, as you become more proficient, these tasks move to other brain areas, clearing up important resources in the prefrontal cortex for skills like split-second decision making.

"The exciting thing is we can now quantify this," Ayaz said. In a flying scenario, Ayaz envisions that the airplane itself might one day be able to assess the cognitive and emotional state of the pilot, and then make adjustments accordingly.

To find out whether this could be possible, the researchers split 28 pilots into two categories: The first group flew a real aircraft, while the second group operated a flight simulator. In both cases, the researchers monitored the pilots' brain activity as they completed a series of memorization tasks from pre-recorded air traffic control instructions, which varied in levels of difficulty.

The results showed that the pilots in the real flight condition committed more errors and had higher anterior prefrontal cortex activation than pilots in the simulator when completing cognitively demanding tasks.

The implications are twofold, says Ayaz. First, the researchers successfully demonstrated the feasibility of monitoring cognitive workload in a realistic flight situation. Secondly, the differences between the two groups underscores the need for "ecologically valid" research studies. In other words, solely measuring brain activity in a laboratory likely won't lead to the most accurate results.

In the future, understanding the underlying neurocognitive process of pilot-plane interactions could help to make simulators more realistic, as well as to improve the safety and efficiency of aircraft-pilot interactions.

"We believe that this type of approach will open a whole new direction of research for studying parameters in an aviation setting and eventually designing better machines," Dehais said.