It’s difficult to design for motor tasks because it calls for fast movement adaptation. Extended reality (XR) technology can help in this situation by allowing people who are learning motor skills to preserve their mobility while exercising, receiving physical treatment, or even dancing.
Computer scientist Misha Sra, an assistant professor at UC Santa Barbara, has been awarded a National Science Foundation (NSF) Early CAREER Award to develop head-worn XR gadgets that facilitate motor skill training. Embodied extended reality agents (ERAs) are used in Sra’s interactive design to mimic the efficiency of learning from a human expert. Funding of $606,000 over five years is included in the award.
In the context of motor-skill learning, the project will provide fundamental knowledge regarding interactions between people and AI agents that can mimic the verbal and nonverbal behaviors of human expert trainers.
The goal of this research, according to Sra, is to employ XR and AI to improve human physical capabilities. It aims to move artificial intelligence (AI) technology off of two-dimensional screens and into the real world, providing vital support and direction for learning and regaining motor abilities.
Increased access to motor-skill development by making it more affordable at scale is one of Sra’s research’s potential outcomes. Her study utilizes a variety of XR-related technologies, fusing the real and virtual worlds to produce immersive and interactive experiences.
Multimodal guidance and feedback through visual, aural, and haptic channels that provide explanations and demonstrations is a crucial part of Sra’s technology. The ERAs are created to appear and behave like humans, including body language like nodding and other facial emotions, and to identify and react to a user’s physical activities in real time or very close to it.
Sra mulls over a potential usage, such as helping a person recovering from an accident who needs physical therapy but can only access care every few months due to the distance to the nearest physical therapist.
“The person might not know what to do between appointments to help speed up their own recovery,” she explained. “Or, if they follow (often printed) instructions, they may not be sure if what they are doing is correct or will, possibly, cause injury.” Losing motivation may result from not being able to monitor one’s own development.
Imagine an AI system that is capable of offering direction and feedback in real time, she said. Wasn’t that wonderful? Making the most of the time in between visits with your clinician or therapist or knowing what to do after your therapy is officially ended but you are still far from recovered would help you recover significantly faster.
Sra’s research combines artificial intelligence (AI) with extended reality (XR), which is a group of technologies that merge the real and virtual worlds to produce engaging and immersive experiences. Virtual reality (VR), which produces a completely digital environment that users can interact with, augmented reality (AR), which superimposes digital content over the actual world, and mixed reality (MR), which combines virtual and real aspects, are all examples of XR. The ability for robots or computer programs to learn, reason, and make judgments based on data and algorithms is provided by artificial intelligence (AI).
The real-time feedback system faces numerous challenges and knowledge gaps, and Sra acknowledged as much. She expressed hope that her lab can “address some of them in our work and address others through collaborations with researchers in areas like computer vision, natural language processing, and cognitive science. The creation of AI agents with practical applications and significant societal benefits will be facilitated by ongoing developments in these fields.
