People recognize gestures and interpret glances very quickly and almost automatically. Computers and robots cannot do this. That is why scientists around the world are working on ways to make human-computer collaboration more social, efficient and flexible. Computer scientists from Saarbrücken and Stuttgart have now reached an important milestone together with psychologists from Australia. The software system they developed processes a person's eye movements to calculate whether they are vulnerable, sociable, tolerant, conscientious or curious.
"Not only do our eyes let us capture the environment, they also are the window to our soul, because they reveal who we are, how we feel and what we do," explains Andreas Bulling, who heads the research group "Perceptual User Interfaces" in Saarbrücken at the Max Planck Institute for Informatics and the Cluster of Excellence of Saarland University. With scientists in Stuttgart and Australia, Bulling has trained the group’s own software system so that it can evaluate eye movements and draw conclusions on the traits of a person. The research team used special computational methods of machine learning.
To obtain training and evaluation data, 50 students at Flinders University in Australia, including 42 women and eight men, with a mean age of 22 years, took part in a study. Upon arrival in the lab, the researchers equipped the students with a so-called "eye tracker." This device filmed the subject's eye movements as they strolled around the campus for about ten minutes, buying coffee or other items at the campus store. Afterwards, the scientists asked the students to take off their glasses and fill in special questionnaires to determine their personality and level of curiosity in a conventional way.
"In order to analyze the recorded eye data regardless of the duration of the recording, we worked with a shiftable time window without blurring any characteristics," explains Bulling. From each of the resulting windows, the researchers gained 207 characteristics. These included statistics on gaze fixation as well as the blink rate.
Based on eye data and the information from the questionnaires, the researchers developed for each character trait a classifier consisting of 100 decision trees, and trained it. The result: In the subsequent test with previously unused data, they were able to prove that the software system reliably recognizes traits such as emotional lability, sociability, compatibility and conscientiousness.
"We can transfer this knowledge about nonverbal behavior to robots so that they behave like humans. Such systems would then communicate with humans in a much more natural way, making them more efficient and more flexible," Bulling explains the benefits of the research. Together with Sabrina Hoppe from the University of Stuttgart, Tobias Loetscher from the University of South Australia in Adelaide, Australia and Stephanie Morey from Flinders University, also in Adelaide, Bulling has discussed the results in the article "Eye Movements During Everyday Behavior Predict Personality Traits" published in the journal "Frontiers in Human Neuroscience".
The project was funded by the Australian Research Council, the Cluster of Excellence "Multimodal Computing and Interaction" at Saarland University, and a doctoral scholarship from the German National Academic Foundation.
