Can robot-inspired computer-assisted support benefit autistic children?

A new study published in the Journal of Computer-Assisted Learning introduces a novel Robot-Inspired Computer-Aided Adaptive Autism Therapy (RoboCA3T) that leverages the natural affinity of children with autism spectrum disorder for technology and robots.

RoboCA3T harnesses the potential of robot-assisted therapies by integrating robot avatars with computer-assisted therapies through a web-based solution.

When investigators assessed Childhood Autism Rating Scale scores before and after the intervention, they noted significant enhancement in joint attention, or the ability to coordinate attention and share a point of reference with another person. Scores also indicated improvements in imitation skills, indicating that the treatment helped children to observe and mirror the behaviours of others.

“The research contributes significantly to the ongoing effort to develop cost-effective, time-efficient, evidence-based treatments for children with autism spectrum disorder,” said corresponding author Sara Ali, PhD, of the National University of Sciences and Technology, in Pakistan. “RoboCA3T prioritizes personalized content delivery along with the integration of AI-based automatic gaze and pose detection algorithms.”

Multiple sclerosis – robotic system feeds people with severe mobility limitations

“Feeding individuals with severe mobility limitations with a robot is difficult, as many cannot lean forward and require food to be placed directly inside their mouths,”
“Feeding individuals with severe mobility limitations with a robot is difficult, as many cannot lean forward and require food to be placed directly inside their mouths”

Cornell researchers have developed a robotic feeding system that uses computer vision, machine learning and multimodal sensing to safely feed people with severe mobility limitations, including those with spinal cord injuries, cerebral palsy and multiple sclerosis. 

 Tapomayukh “Tapo” Bhattacharjee, assistant professor of computer science in the Cornell Ann S. Bowers College of Computing and Information Science and senior developer behind the system, went on to say, “The challenge intensifies when feeding individuals with additional complex medical conditions.” 

A paper on the system, “Feel the Bite: Robot-Assisted Inside-Mouth Bite Transfer using Robust Mouth Perception and Physical Interaction-Aware Control,” was presented at the Human-Robot Interaction conference, It received a Best Paper Honorable Mention recognition, while a demo of the research team’s broader robotic feeding system received a Best Demo Award. 

A leader in assistive robotics, Bhattacharjee and his EmPRISE Lab have spent years teaching machines the complex process by which humans feed themselves. It’s a complicated challenge to teach a machine everything from identifying food items on a plate to picking them up and then transferring them inside the mouth of a care recipient.  

“This last 5 centimetres, from the utensil to inside the mouth, is extremely challenging,” Bhattacharjee said. 

Some care recipients may have very limited mouth openings, measuring less than 2 centimetres, while others experience involuntary muscle spasms that can occur unexpectedly, even when the utensil is inside their mouth, Bhattacharjee said. Further, he said that some can only bite food at specific locations inside their mouth, which they indicate by pushing the utensil using their tongue. 

“Current technology only looks at a person’s face once and assumes they will remain still, which is often not the case and can be very limiting for care recipients,” said Rajat Kumar Jenamani, the paper’s lead author and a doctoral student in the field of computer science. 

To address these challenges, researchers developed and outfitted their robot with two essential features: real-time mouth tracking that adjusts to users’ movements, and a dynamic response mechanism that enables the robot to detect the nature of physical interactions as they occur, and react appropriately. This enables the system to distinguish between sudden spasms, intentional bites and user attempts to manipulate the utensil inside their mouth, researchers said. 

The robotic system successfully fed 13 individuals with diverse medical conditions in a user study spanning three locations: the EmPRISE Lab on the Cornell Ithaca campus, a medical center in New York City, and a care recipient’s home in Connecticut. Users of the robot found it to be safe and comfortable, researchers said. 

“This is one of the most extensive real-world evaluations of any autonomous robot-assisted feeding system with end-users,” Bhattacharjee said.  

The team’s robot is a multi-jointed arm that holds a custom-built utensil at the end that can sense the forces being applied on it. The mouth tracking method – trained on thousands of images featuring various participants’ head poses and facial expressions – combines data from two cameras positioned above and below the utensil. This allows for precise detection of the mouth and overcomes any visual obstructions caused by the utensil itself, researchers said. This physical interaction-aware response mechanism uses both visual and force sensing to perceive how users are interacting with the robot, Jenamani said. 

“We’re empowering individuals to control a 20-pound robot with just their tongue,” he said. 

He cited the user studies as the most gratifying aspect of the project, noting the significant emotional impact of the robot on the care recipients and their caregivers. During one session, the parents of a daughter with schizencephaly quadriplegia, a rare birth defect, witnessed her successfully feed herself using the system.  

“It was a moment of real emotion; her father raised his cap in celebration, and her mother was almost in tears,” Jenamani said. 

While further work is needed to explore the system’s long-term usability, its promising results highlight the potential to improve care recipients’ level of independence and quality of life, researchers said. 

“It’s amazing,” Bhattacharjee said, “and very, very fulfilling.”