AI in Therapy Virtual Therapy Strengthens Social Skills for Autism

Related Vendors

FAULHABER Drive Systems

SIMTOS (Seoul International Machine Tool Show)

Researchers at Graz University of Technology are using virtual reality and large language models to help people with autism spectrum disorder train social skills. The system is intended to make therapy options more widely accessible.

An increasing number of people worldwide are affected by autism spectrum disorder (ASD), with studies showing that one in 44 children is diagnosed with the condition. A central symptom is so-called 'social blindness', i.e. the inability to recognize emotions in others and react appropriately to social situations. The appropriate therapy for this is usually based on 1:1 or small group support, which are only available to a limited extent and are cost-intensive. Researchers at the Institute of Human-Centred Computing at TU Graz are using computer game technology to create an effective supplement that is inexpensive and available at any time. Initial studies show that this approach helps people with ASD to get through everyday life more safely.

Everyday Situations Without Social Consequences

The specially developed virtual environment Simville uses virtual reality, large language models (LLMs), speech recognition and speech generation to make social training location-independent and therefore more accessible for those affected. In this computer world, users train for realistic everyday situations, such as conversations with work colleagues or meeting people in a coffee shop. As this takes place in a controlled environment, users can act freely without having to fear social consequences. These training scenarios make them better prepared for similar interactions in everyday life.

"Our system is not intended to replace traditional therapies, but to complement and reinforce them," says Christian Poglitsch from the Institute of Human-Centred Computing at TU Graz, who implemented the project as part of his dissertation. The immersive but playful approach is of central importance for Simville: tasks, storytelling and immediate feedback after a played-out scene motivate participants to practise regularly. In addition, the number of stimuli acting on the user can be controlled so that beginners can start with a small number and increase this over time through their training and reduce it again if they become overwhelmed.

Language Model Conveys Emotions

By integrating LLMs and speech recognition and generation, users can speak to the avatars in the game world as normal. What is said is translated into text by the speech recognition system, a large language model generates a reaction tailored to the situation and the avatar responds accordingly in spoken language. The team used Google's Gemini 12B model to create and play out the response. "What was fascinating was that the model was also able to convey a certain emotion. Depending on the context of the spoken word, you can definitely hear the right undertone," says Christian Poglitsch.

Test Subjects Feel Safer

Initial studies show that training with Simville has positive effects. For example, a study with 25 participants showed that after just a few sessions, many felt much more confident in social situations. Simville is now being incorporated into the international ETAP project led by Furtwangen University. The simulation interface is combined with extensive sensor technology in order to reduce or increase the intensity of the experience based on the user's reaction. In addition, the Game Lab Graz at Graz University of Technology wants to make Simville available as a demonstrator so that affected people can train with it themselves.

AI in the industry

ETMM Dossier – Artificial Intelligence