For autonomous vehicles to gain the trust of their drivers and passengers, the vehicles need to understand or trust human emotions, according to Mehrdad Nojoumian, Ph.D., an associate professor in the College of Engineering and Computer Science, and director of FAU’s Privacy, Security and Trust in Autonomy Lab.
Building that trust is behind Nojoumian’s new patented technology, which allows an autonomous vehicle to perceive a driver’s or passengers’ emotions and react accordingly.
The patent, titled Adaptive Mood Control in Semi- or Fully-Autonomous Vehicles, utilizes non-intrusive sensors in autonomous vehicles to perceive the mood of the drivers and passengers. Information is collected based on facial expressions, sensors within the handles/seats and thermal cameras among other monitoring devices. Additionally, the adaptive mood control system contains real-time machine-learning mechanisms which will continue to learn the driver’s and passengers’ moods over time. The results are then sent to the autonomous vehicle’s software system allowing the vehicle to be responsive to perceived emotions by choosing an appropriate mode of operations, like normal, cautious or alert driving mode.
“The uniqueness of this invention is that the operational modes and parameters related to perceived emotion are exchanged with adjacent vehicles for achieving objectives of the adaptive mood control module,” he said.
To develop the technology, Nojoumian and his team studied trust between humans and autonomous vehicles using a self-driving car simulator with 360-degee realistic videos from roads and highways in South Florida. The videos were incorporated into a virtual reality simulator with a motion chair to replicate the car movements. The simulator exposed 100 human subjects to a sequence of normal, trust-building, or trust-damaging scenarios in two rounds of data collections, such as being cut off by another car or having to abruptly stop due to an upcoming accident. This allowed the simulator to perceive the emotional state of the human during data collection, and responsively adjust actions.
Before coming to FAU, Nojoumian was an assistant professor at Southern Illinois University, researching trust and security in robotics. He earned his doctoral degree in computer science from the University of Waterloo in Ontario, Canada, in 2012, and his master’s degree in computer science from the University of Ottawa in 2007.
After spending time working on theoretical models of trust, he said, he was ready to dive into his first research questions, like can you make a controller for a self driving car, how can you ensure that a passenger’s trust of a car would be sustainable over time.
“Human-AI/autonomy interaction is in the center of attention by academia and industries, and more specifically, trust between humans and AI/autonomous technologies plays a critical role in this domain because it will directly affect the social acceptability of these modern technologies,” Nojoumian said. “I know this is something that will revolutionize the AI and autonomous industries, and I am truly proud that Florida Atlantic University is behind this cutting-edge technology.” ♦
Photography by Aldo Frias