Don't look at me like that!

What happens during eye contact from a psychological point of view? This is what interests Anne Böckler-Raettig, Professor at the Department of Psychology III at the University of Würzburg (JMU). Social cognition is one of the focal points of her research; she has headed the research group "More than meets the eye: Integration, influences and impairments of direct gaze processing" since 2017.

Now, together with her team and scientists from the USA and Canada, Böckler-Raettig has decoded new information about how we process gazes and facial expressions. She presents the results of her studies in the Journal of Experimental Psychology.

Emotional expression shapes attention

"We were able to show that the emotional expressions of faces influence how their gazes shape our attention," explains explains Dr. Christina Breil, the first author of the study. What this means in concrete terms is this: Faces showing joy, an emotion that expresses approach, grab attention when they look directly at the observer, i.e. when there is eye contact (which also signals approach). The same applies to an angry facial expression because, from a psychological point of view, anger is also an approach-oriented emotion.

The situation is different with avoidance-oriented emotions such as disgust or fear. In these cases, it is averted gaze (hence, avoidant gaze) that attracts the observer's attention more..

Eye contact draws attention with neutral faces

In a previous study, Böckler-Raettig and her team had already demonstrated that faces with a neutral expression attract the attention of the observer especially well when those faces look directly at them. The participants were presented with four images of a woman's face on a computer screen. The only difference was that some faces looked at the test subjects, while others did not. In addition, all the faces had a small number 8 positioned on their forehead.

"After exactly 1.5 seconds we replaced the four eights with letters. One of them was either an S or an H," explains Böckler-Raettig. The participants were asked to react as soon as these letters were visible on one of the four faces, by pressing an S on a keyboard when they saw an S, or an H when an H appeared. The reaction time was then the measure as an indication of the level of attention.

The results of this study showed that although faces can be ignored for this task, people recognise the relevant letters more quickly if they are presented on a face that is looking at them. Why? "Because eye contact attracts our attention," says the psychologist.

Anger and joy signal approach

But how do different emotional facial expressions affect attention capture by direct gaze? An interesting hypothesis about facial expressions and gaze direction suggests that gazes and facial expressions have a particularly strong impact if they are congruent in terms of "approach" or "avoidance."

"For example, a delighted face that looks at you is congruent in that sense because joy is an approach-oriented emotion and the direct gaze also expresses approach," says Breil. Accordingly, a disgusted face that looks away is also congruent.

To investigate this effect, the team changed the design of the original study. In the new version, the faces changed from a neutral expression, when the eights were shown, to either an approach- or an avoidance-oriented emotional expression -- in one series of experiments to anger or fear, in the other to joy or disgust.

With disgust, averted gaze attracts more attention

A total of 102 people participated in this study. The results are clear: If happy facial expressions follow neutral ones and the gaze is directed at the observer, the reactions are the fastest. The pattern is reversed if the facial expression changes to disgust. Then the reaction is faster if the gaze is averted.

To better understand these results, the team repeated this experiment and measured participants' eye movements. "We find the same pattern in eye movements, too: Participants look faster and longer at faces that are happy and that are looking directly at them, and participants look faster at disgusted faces that look away" says Böckler-Raettig.

Quick processing of expression and gaze direction

According to the psychologist, this pattern indicates that people can process and integrate the facial expression and gaze direction of faces very quickly and efficiently. In fact, this process of integration begins just 200 milliseconds or so after the stimulus is shown. The results also suggest that it is not the case, as is often thought, that gazes are processed independently of the context and always have the same effect. Even when it comes to processes as basic as attracting attention with a gaze, the context plays a role.