Google Glass may have failed as a high-tech fashion trend, but it’s showing promise as a tool to help children with autism better navigate social situations.
A new smartphone app that pairs with a Google Glass headset uses facial recognition software to give the wearer real-time updates on which emotions people are expressing. In a pilot trial, described online August 2 in npj Digital Medicine, 14 children with autism spectrum disorder used this program at home for an average of just over 10 weeks. After treatment, the kids showed improved social skills, including increased eye contact and ability to decode facial expressions.
After her 9-year-old son, Alex, participated in the study, Donji Cullenbine described the Google Glass therapy as “remarkable.” She noticed within a few weeks that Alex was meeting her eyes more often — a behavior change that’s stuck since treatment ended, she says. And Alex enjoyed using the Google Glass app. Cullenbine recalls her son telling her excitedly, “Mommy, I can read minds.”
All the feels
A new Google Glass program can alert the wearer when someone is expressing a particular emotion (like happiness, sadness, anger, disgust, surprise, fear or contempt) by displaying an emoticon (top) on-screen. After using the program, parents and children can review a video (bottom) of their interaction accompanied by a color-coded timeline, which indicates when the Google Glass program ID’d different emotions.
Unlike most children, who naturally learn to read facial expressions by interacting with family and friends, children with autism often have to hone these skills through behavioral therapy. That typically involves a therapist leading the child through structured activities, like exercises with flash cards that depict faces wearing different expressions. But therapists are so few and far between that a child diagnosed with autism can spend 18 months on a waiting list before starting treatment.
Dennis Wall, a biomedical data scientist specializing in pediatrics at Stanford University, and colleagues built the new Google Glass program to offer children with autism at-home, on-demand behavioral therapy. The headset’s camera records the faces of people in the child’s field of view and feeds that information to a smartphone app. The app, trained on hundreds of thousands of face photos, is designed to recognize eight core expressions: happiness, sadness, anger, disgust, surprise, fear, contempt and calm. When the app recognizes an expression of one of these feelings, it sends the information to the Google Glass wearer — either by naming the emotion through the headset speaker or by displaying an emoticon on a small screen in the corner of the right spectacle frame.
In the pilot trial, children ages 3 to 17 with autism used this Google Glass program around their families for at least three 20-minute sessions per week. Before and after treatment, parents completed questionnaires that rated their children’s social skills. On this Social Responsiveness Scale, scores below 60 fall within the “normal” range, whereas scores 60 to 65, 65 to 75, and above 75 indicate mild, moderate and severe autism, respectively. Over the course of treatment, the children’s average score dropped from 80.07 to 72.93.
Eleven of the 14 children also completed an emotion recognition exam at the start and end of treatment. In this test, an examiner acted out each of the eight core emotions five times, and the child guessed which emotion the examiner was expressing. Before therapy, kids got an average 28.45 out of 40 questions right; afterward, they averaged 38 correct responses.
While these results are encouraging, the study did not include a control group of children who didn’t receive the treatment. So it’s not entirely clear whether the Google Glass program was the only reason that children showed improved social skills over the course of treatment, says Ned Sahin, a neuroscientist at Brain Power, a company that develops wearable technology to help people with autism, in Boston. A randomized controlled trial, where children are randomly assigned to receive treatment or not, could provide further insight into the therapy’s effects, Sahin says.
Wall and his team are currently working on one such experiment with 74 children ages 6 to 12. If the Google Glass therapy performs well in future clinical trials and is cleared for widespread use, it could be a powerful learning aid for many children with autism — which affects an estimated 1 in 59 children in the United States, according to the U.S. Centers for Disease Control and Prevention.
Already, Cullenbine expects that Alex will have better relationships with people, “and that’s life changing.”
