001. THESES & DISSERTATIONS

Permanent URI for this community

https://testres.nu.edu.kz/handle/123456789/8123

Browse

Browsing 001. THESES & DISSERTATIONS by Author "Issa, Ilyas"

Now showing 1 - 1 of 1
  • Thumbnail Image
    ItemOpen Access
    ADAPTING TO LEARNER’S COGNITIVE DIFFERENCES USING REINFORCEMENT LEARNING
    (Nazarbayev University School of Engineering and Digital Sciences, 2023-05-02) Nurgazy, Symbat; Issa, Ilyas; Kassymbekov, Saparkhan; Kuangaliyev, Zholaman
    This report explores the benefits and challenges of creating adaptive robotic systems by integrating open-source software and reinforcement learning algorithms. The aim is to develop robotic systems that adapt to cognitive differences of the child to increase the engagement time and develop social skills using robot-assisted therapy. The Furhat and NAO robots are used as the platforms for receiving inputs and outputs, while a reinforcement learning agent selects the order of ac- tivities. The state space includes the pose landmarks from the Mediapipe and OpenPose libraries to recognize the engagement level and some static parameters such as age, autism level, verbality, and the co-diagnosis of Attention Deficit Hyperactivity Disorder. The actions space contains the act of changing the current activity. In the current stage, we have eight types of activities the robot choose to interact with the child. Furhat robot is programmed to have a full pipeline for a conversational robot: automatic speech recognition, text-to-speech, machine translation, and OpenAI language model. The results of this project provide insight into the potential of using open-source software and reinforcement learning algorithms to create more advanced interactive robots and highlight the importance of continued research in the field of Human-Robot Interaction.