STAY AT HOME!
What is the best way to communicate an important message such as 'Stay at home'? The research team of Mauri compared the emotional reactions to three short videos containing this message using FaceReader.
Posted by
Published on
Mon 11 May. 2020
Topics
| Emotion Recognition | Emotions | FaceReader | Facial Expression Analysis | Measure Emotions | Psychology |
This is a guest blog post by Maurizio Mauri, head of User Experience & Neuro-Marketing Research at SRLABS in Milan, Italy. He is kindly sharing the results of a study with FaceReader during the COVID-19 pandemic. Enjoy reading!
STAY AT HOME!
This is the message that we have all heard in the past few months because of the “Corona virus”. Have you ever thought about the best way to communicate such an important message?
We compared three short videos of influential persons who adopt three completely different communication styles to approach the topic of the pandemic caused by the spreading of the virus called Covid-19, in order to persuade Italians to "STAY AT HOME".
Emotional reactions to “stay-at-home” messages
The comparison of the effects of the three audiovisual messages is based on the emotional reactions of people, detected through the analysis of their facial expressions, while watching the three videos.
Take a guess… which one would be the most effective video message?
- The statement made by Italian first minister Giuseppe Conte
- The message communicated by the well-known virologist Roberto Burioni
- The video made by Italy's most famous scientific popularizer Alberto Angela
How to persuade Italian people to "STAY AT HOME"
Is it better to convey a message in a "strong way", communicated by an expert virologist (like Roberto Burioni), while being interviewed by a famous TV presenter? Does this make the message more effective and vivid?
Or is it better to perform it in a "calm way", by a famous scientific popularizer (such as Alberto Angela), who speaks in an "informal" setting?
Or rather in the "serious and institutional way", expressed by the Italian political leader, responsible for managing the crisis (the first minister Giuseppe Conte)?
Ah, of course, all in a minute only! This was the time available to expose the participants to our little research in which three audiovisual messages were shown (communicated by the three mentioned persons).
Measuring satisfaction and emotional responses
The answer about which audiovisual message was most effective, in terms of satisfaction and emotional responses, is presented here by our company SR LABS, together with our partner JUNGLES, with whom we have designed the research. The project was conducted by means of an online participation and combined the results of the innovative technique based on the automatic analysis of the emotional expressions of the face with those of a short traditional questionnaire about the perceived satisfaction.
In our research, 34 people (mainly university students from the course in "Cognitive Sciences applied to Ergonomics, User and Customer Experience", at the Catholic University of Milan, we thank the students together with all the other volunteers) were invited by e-mail to participate in the study.
After receiving a link, the participants comfortably opened each of the three videos sitting in front of their own PC while staying at home. The three audiovisual messages were presented to each participant in a random order. The random order of the exposure of the three videos allows to avoid any kind of “sequence effects”.
Which messenger is preferred? The results of the analysis of the questionnaire, which explored the level of satisfaction, show that:
- Alberto Angela was the most appreciated (18 people out of 34 reported to prefer his message in comparison to the other two audiovisual communication)
- Roberto Burioni (12 people out of 34)
- Giuseppe Conte (4 people out of 34)
It is clear that a preference emerges for the “calm and informal way” performed by a scientific popularizer who is quite appreciated in Italy. Both in comparison to a virologist who takes advantage of a strong and assertive way to explain why people must stay at home, as in comparison to a political leader who must maintain the greatest seriousness within an institutional communication format.
However, so far we have measured the "WHAT", not the "WHY". To understand the “WHY”, it is possible to take advantage of insights from neuroscience and experimental psychology.
Insights from neuroscience
From neuroscience we know that, thanks to the brilliant scientific research of Stephen Porges and his "polyvagal theory", facial expressions represent direct reactions of the nervous system (such as the psychophysiological reactions reflected by the cardiac system, the respiratory system, etc.). For this reason, facial expressions can be useful for exploring spontaneous "non-verbal" reactions, together with "verbal" and more "rational" ones (as more "reasoned" and less spontaneous) collected in the questionnaires.
Insights from experimental psychology
From experimental psychology we know that for campaigns based on "negative emotions", such as fear or sadness, to be more effective, they must induce a moderately high level of emotion. We’ve learned this from the theory of the so-called "Fear Arousing Appeals", a theoretical framework commonly used in communication campaigns when a message tries to convince members of the public to adopt a more "healthy behavior”.
Ideally, the negative emotion trend should describe a sort of inverted "U", where the negative emotion begins with an increase in intensity, until it reaches an apex or "emotional peak", which then returns to fall, in which there is a sort of final "relief", compared to the emotion of fear (or sadness) elicited in the initial part of the message.
By providing solutions or explanations (also known in psychological terms as "coping instructions") in the final part of the communication, it is possible to help people understand how to reduce or avoid the threat generating negative emotions. This "relief" is elicited thanks to the possibility to face or manage the danger presented in the message, thus also conveying the hope of being able to reduce or avoid the threat itself.
Analyzing emotions
The emotions reflected by faces of participants while watching the three videos were analyzed. Given the difficult general context caused by the emergency of COVID-19 and the “heaviness” raised by the new rule to stay at home for a long period, sadness (after the “neutral expression) was undoubtedly the most predominant emotional reaction, at least in absolute values:
- in the video with Burioni, the average is 10%
- in the video with Conte, the average is 9%
- in the video with Angela, the average is 8%
To better understand these results in percentages, keep in mind that every emotional expression (fear, anger, sadness, joy, etc.) is revealed with a percentage ranging from 0% to 100%: an expression of joy, for example, 100% could be detected when the person finds out he/she has won the lottery!
Obviously, with the exception of special cases, it is rare for people to show an emotional expression in its maximum intensity equal to 100%. In most cases, people’s facial expressions represent an "inkling" of emotional expressions (such as an inkling of a smile), a prodromal of the expressions of joy, which reveals more a slight satisfaction or contentment (what in psychology is also called "engagement"), rather than a real expression of "joy".
For this reason, when we mention emotions like "joy" or "sadness" equal to 10%, we refer to an expression that in that second or in those seconds shows an inkling of the configuration of the muscles of the face that is organized in order to express that emotion in a mild way (up to 10%).
By studying the trend of emotions during the exposure to the various videos, however, you can see different trends (like different "stories", told differently):
- in the case of Burioni, the percentage of sadness is always around or above 10%, from the beginning to the end, it never shows big changes, neither downward nor upward, for all 60 seconds of the audiovisual message:
- in the case of Conte, it remains below 10%, around 8% more or less, with the same trend without big changes, downside nor upside, from the beginning till the end:
- in Angela's case, however, there is a continuously descending trend. It starts with values around 12%, when the message begins, he says: "... we are going through an important moment in our history, it is not easy, I know ..." and then he adds, to the second number 5 of the video: "... we have to stay at home ...", in which one of the highest peaks of percentage of sadness is revealed. Then, at about 20 seconds, the percentage average values decrease to approximately 10%. Especially in the final part of the message, there is a negative peak that falls down to 5%. In short, in the video of Angela the negative emotion diminishes up to the minimum in the final part of the message:
The facial expression “sad”
The graph below shows the trends of the “sad facial expression” during the three videos, with relative "trend lines" (a simple algorithm that calculates the general trend, rising or falling). Looking at the chart, notice how the green signal (Angela) is the only one that has a downward trend: sadness systematically decreases as the video is shown to participants, while the other two signals (Burioni, in red; Conte, in blue) increase progressively as an average percentage. This reveals that there is, on the contrary, an increase in percentage of expressions of sadness. On the x-axis of the graph, time is represented according to the sampling points provided by the technology (equal to 5 times per second: thus 60 seconds become 300 samples).
The emotion “joy”
Another emotion is slightly elicited in all three videos: joy ("happy facial expression"). In absolute terms, we see here the average results in percentage:
- 1% in the video with Burioni
- 2% in the video with Conte (interesting to note that when Conte claims: "... in the first place there is and there will always be the health of Italians...", there is a peak of increase in intensity of "joy" that exceeds 4%)
- 4% in the video with Angela
However, it is interesting to consider once again the trends, related to "happy facial expressions” during the exposure to the three videos:
- Angela's video shows the most important change. It starts very low (similarly to the other two videos, between 1% and 2%), but increases as the video goes on, to the 23rd second (when saying: "... friends can also meet with social networks, books can be rediscovered..."). It exceeds 6-7%, up to a peak, raised in the final part, equal to 10% (when Angela claims "... one person who saves another, is a hero; but if this person saves 100 people, he/she is a nurse, or a physician..."). The relative trend line (in green, in the graph below) is the only one that rises upwards.
- Conte's video shows a general decreasing trend: in the very last final seconds only it is possible to see an increase (when he declares "... all together we will make it..."), but the increase is less accentuated, and in fact the relative trend line decreases always.
- Burioni's video, after a peak that shows an increase in "happy" expressions (at the 5th second, when he declares "... this virus is not a flu...", in which it exceeds 5%), rises a completely flat reaction in terms of “Happy facial expression”, and in fact, the relative trend line is always inclined downwards and the video closes near 0%.
The graph below shows the three trends of "happy" for the three videos, with the related "trend lines". Only the one relating to Angela's video rises upwards with a rather steep "slope", signaling a significant increase in terms of percentage of joy, especially at the end of the message.
Towards the end, Alberto Angela's video message conveys words aimed at trying to show the positive effects of the adoption of the new rule that invites to "stay at home". This corresponds to the "relief" which also leads to the hope of reducing or avoiding the threat, which are reflected by the participants' facial expressions, marked by a pattern of "joy" in their face muscles equal to 10% of “happiness”.
Therefore, a inkling of "joy", however enough, thanks to the high-level sensitive detection of the technology, to reveal a greater satisfaction and participation aroused by the message (the "WHY" it was more appreciated), which then mediated the levels of satisfaction ("WHAT" was more appreciated).
Comparing emotions within different groups
Another interesting result is revealed when we tried to divide the participants into two macro-age groups: young (under 25) and more "aged" (over 25): younger people respond less intensely to messages, while those over 25 years of age show intensities of emotions systematically more accentuated than younger ones. As mentioned, since the participants are mostly university students, we could only compare a dozen young subjects taken at random with a dozen over 25. Despite the small number of participants for each age group, different emotional reactions to the same audiovisual film already emerge according to the age group they belong to.
The usefulness of facial expression analysis
We hope to have shown the contribution of the automatic analysis of facial expressions in deepening why people can appreciate one audiovisual message more than another can. It is not just a matter of prevalence of one emotion over the other, but rather the dynamics of the trends, as learned by studying Angela's video, in which we started with a predominant emotion of sadness equal to or even greater than the other two videos, followed by the insight to see the same emotion of sadness diminishing systematically. At the same time leaving space for a new emotion rising, a growing of "contentment" and "satisfaction" (10% of “Happy facial expressions”).
All this can be applied to any type of audiovisual message; actually, to any type of experience, whose effects can be assessed in terms of emotional reactions mediating the perception of liking and satisfaction.
Related Posts
How to use facial expression analysis in pain research
Comparing machine emotion recognition with a human benchmark
