Multimodal Emotions and Stress Detection in UAV Operators Using Deep Learning

Unmanned aerial vehicles (UAVs) and drone operators perform a demanding job with many challenges.

As UAVs become increasingly autonomous, this allows a reduction in the number of personnel in command during missions. However, this also poses safety challenges, as control and supervision periods are growing longer.

CRIM’s project, carried out through the National Defence’s IDEaS program, aims to assess the psychological state of operators in a non-intrusive manner using AI and computer vision. This new technique uses facial emotion analysis to assess the emotional state of the individual using a deep learning model.

The development of such a reliable non-intrusive technique would make it easy to monitor the psychological state and stress level of each operator and avoid mental overloads which could lead to incidents.

More information : L’IA détecte l’état émotionnel des opérateurs de drones – CScience.ca [Article in French]

Mohamed Dahmane, researcher in computer vision at CRIM, will offer a masterclass on the subject:
Human Factors in Autonomous System Operators: AI in the Loop.

At the CRIAQ RDV Forum, February 19 at 1:30 p.m. Registration here

Cours de maître CRIM

Scientific publication related to the project: 
M. Dahmane, J. Alam, P. St-Charles, M. Lalonde, K. Heffner and S. Foucher, “A Multimodal Non-Intrusive Stress Monitoring from the Pleasure-Arousal Emotional Dimensions” in IEEE Transactions on Affective Computing, April 20, 2020.