Project name: Autism Diagnostic Observation with Robot Evaluator
Project acronym: ADORE
Dates: 01.09.2014 - 31.08.2018
Total budget: 1,000,000.00 HRK
Funding: Croatian Science Foundation (HRZZ)
Project Number: 93743-2014
Project name: Autism Diagnostic Observation with Robot Evaluator
The main goal of the ADORE project is the development of a robot-based diagnostic protocol and validation of a robotic diagnostic assistant in clinical settings. The robots are used as ASD co-evaluators to help a human evaluator to assess a child's behavior objectively. To the best of our knowledge, such a robot-based ASD diagnostics system is the first of its kind.
The robot-assisted ASD diagnostic protocol consists of four tasks and is based on the Autism Diagnostic Observation Schedule (ADOS) which is considered to be the “golden standard” of ASD diagnostics. Within the tasks, the robot observes social cues such as eye contact, verbal activity and ability to imitate. Since our goal was to make the robot as autonomous as possible, we modelled the tasks using the Mixed Observability Markov Decision Processes (MOMDPs), which enables the robot to infer the information about the child and control task execution autonomously. On the protocol level, we use the standard Partially Observable Markov Decision Process (POMDP) model which enables the robot to choose tasks of the protocol autonomously and combine observations from tasks into a final assessment of a child’s behavior.
NAO robot demonstrates frog jumping gestures to the child
Preparing the robots for a session at the Child Communication Research Laboratory
Clinical trials are performed at the Child Communication Research Laboratory, which is located at the Croatian Institute for Brain Research. During various development stages, more than 50 children have been introduced to the robot. Fifteen of them (7 developing typically, 8 with ASD) were a part of the latest study which showed that, given perfect observations of social cues, our robots are capable of identifying the behavior of the child.
- University of Zagreb, Faculty of Electrical Engineering and Computing (HR)
- University of Zagreb, Faculty of Education and Rehabilitation Sciences (HR)
On 22 June 2019, Frano Petric presented our experiences with the ADORE project at the first Assistive technologies and communication conference (ASTEK) in Belgrade, Serbia.
Following our collaboration with Alfanum and professor Delic from the Faculty of Technical Sciences in Novi Sad, we were invited to present our experience of using humanoid robots with children with autism. Through the talk "Diagnosing ASD with a robot evaluator - experiences from the ADORE project" Frano talked about the technology behind the robot-assisted ASD diagnostics, challenges we faced but also about mixed reactions of the children to the robot and overwhelmingly positive reactions from the parents and caregivers.
We would like to thank ASTEK organizers and especially professor Delic for the invitation and are hoping we will be able to take part in future ASTEK events.
Another RADDAR workshop, organized in collaboration with the Fifth Gymnasium in Zagreb (MIOC), was held at UNIZG-FER this weekend, on 9th March 2019, during which RADDAR programme attendees went on a tour of our robotics laboratory and learned about how our robots work and what can be done with them. A lot of focus was put on our Pepper robot, so prof. Zdenko Kovačić made sure to help the high-school students get to know more about the interesting field of humanoid robots and the services they can offer. Find out more information on this workshop by following this link (in Croatian).
We have some great news for the final month of ADORE project! The paper entitled “Hierarchical POMDP Framework for a Robot-assisted ASD Diagnostic Protocol” has been accepted for oral presentation at the prestigious international conference ACM/IEEE International Conference on Human-Robot Interaction HRI 2019. The conference will be held on March 11-14, 2019 in Daegu, South Korea, and our dr. sc. Frano Petric will do the presentation.
Yesterday our Matko, Frano and NAO robot went to the Advent in Maksimir, where they took part in an art workshop titled "Overcoming the barriers of autism - together", organized by the Autism Association Zagreb, who we collaborated with on our recent Christmas charity event. During the workshop, visitors had the opportunity to meet members of the Association, see what they've been doing and take a look at the crafts they exhibited, as well as to get to know our NAO robot and learn about its role in diagnosing autism. You can read some more about the event here (in Croatian) and see some of the Association's photos here. We would like to thank the Association for the invitation and wish them a successful new year!
Our colleagues from the Faculty of Education and Rehabilitation Sciences have published an article in the journal Croatian Review of Rehabilitation Research entitled “Reaction to Robots in Social and Non-Social Contexts - Comparison of Children with Autism Spectrum Disorders and their Typical Peers”. In this article they have described their experience gained through numerous sessions with a group of typically developing children and children diagnosed with ASD interacting with NAO robots. The article is accessible at the following open-access link.
Prof. Zdenko Kovačić was invited by the partners of theproject to give a presentation about the ADORE project at the that was held in the popular Bulgarian ski resort Bansko. The aim of the workshop was to bring together experts from several European countries in order to exchange ideas leading to possible scientific collaboration on future joint projects.
Our professor Zdenko Kovačić attended the 26th International Conference on Software, Telecommunications and Computer Networks (SoftCOM 2018), which was held in Supetar-Split, Croatia, where he presented the paper entitled “Object Classification for Child Behavior Observation in the Context of Autism Diagnostics Using a Deep Learning-based Approach”. The paper brought to attention the results of a LARICS student, Mihael Presečan, which demonstrate the effectiveness of a deep learning approach for object detection and classification using a mono-vision feedback of a NAO humanoid robot for assessing a child’s behavior during free play with standardized toys, which is one of the tasks contained in the standard ADOS-2 autism spectrum disorder diagnostic protocol used by clinicians.
Dr. sc. Frano Petric attended the 26th Mediterranean Conference on Control and Automation, held on June 19-22, 2018 in Zadar, Croatia, where he presented the paper entitled “Classification of Child Vocal Behavior for a Robot-Assisted Autism Diagnostic Protocol”. The paper contained the results of a group of people including two LARICS students - Mirko Kokot and Ivan Bejić. The main contribution of the paper is an empirically determined minimal set of sound features, which allow efficient vocal behavior classification of preschool children, relevant in the context of autism diagnostics.
The half-day workshop on Assistive Technologies for People with ASD and other Disabilities held at the MED'18 Conference in Zadar, Croatia, brought together top experts - trainers from the field of robotics, information technology, human-robot interaction, rehabilitation and other related areas from France, Luxembourg, Serbia, Slovenia and Croatia. The keynote speaker was the famous European roboticist Alexandre Mazel who is a Software Innovation Director at Aldebaran - Softbank Group, Paris, France. Prof. Zdenko Kovačić gave a presentation about the ADORE project while Dr. Frano Petric presented the latest results stemming from his doctoral dissertation. Prof. Sanja Šimleša presented the results of ADORE from the clinician’s point of view. More information about the workshop program can be found at this link.
On March 29, 2018 LARICS researcher Frano Petric successfully defended his doctoral dissertation and received a Doctor of Science degree in Electrical Engineering. Frano’s thesis was entitled “Robot-assisted Autism Spectrum Disorder diagnostics using Partially Observable Markov Decision Processes”. Frano will continue his research work within the ADORE project until January 2019. Congratulations!
The results of work done by LARICS researcher Dr. sc. Frano Petric and his coauthors on autism disorder observation with a robot evaluator have been published in the International Journal of Humanoid Robotics (World Scientific) in the article entitled “POMDP-Based Coding of Child–Robot Interaction within a Robot-Assisted ASD Diagnostic Protocol”. The existing procedures for autism spectrum disorder (ASD) diagnosis are often time consuming and tiresome both for highly-trained human evaluators and children, which may be alleviated by using humanoid robots in the diagnostic process. Hence, this paper proposes a framework for robot-assisted ASD evaluation based on partially observable Markov decision process (POMDP) modeling, specifically POMDPs with mixed observability (MOMDPs). POMDP is broadly used for modeling optimal sequential decision-making tasks under uncertainty. The article is accessible at the following IJHR link.
Dr. sc. Frano Petric attended the 2018 ACM/IEEE International Conference on Human-Robot Interaction held in Chicago, USA, where he presented a poster entitled “No data? No problem! Expert System Approach to Designing a POMDP Framework for Robot-assisted ASD Diagnostics”. The poster contained a discussion about the clinical tests of an improved robot-assisted diagnostic protocol with a group of children with ASD, which has drawn the attention of HRI 2018 reviewers who suggested to include Frano’s work into the session entitled Late-Breaking Reports.
Prof. Zdenko Kovačić and Frano Petric were guests in the radio show "Znanstveni koncentrat” (Scientific concentrate) of the Croatian National Radio 1 (Hrvatski radio 1. Program). They answered questions about and discussed the purpose, methodology and significance of our ADORE project for the broader population with ASD.
Frano Petric attended the 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017) held in Lisbon, Portugal, on 28 to 31 August 2017, where he presented a paper titled “Functional Imitation Tasks in the Context of Robot-assisted Autism Spectrum Disorder Diagnostics: Preliminary Investigations”. In this paper, instead of pure motoric imitation, imitation tasks involving real objects in the environment are proposed. The introduction of physical objects strongly emphasizes joint attention skills, another area that is typically impaired in children with ASD. Furthermore, simple object detection, manipulation, tracking and gesture recognition algorithms, suitable for real-time, onboard execution on the small-scale humanoid robot NAO were presented.
Our Frano Petric attended the 2017 ACM/IEEE International Conference on Human-Robot Interaction held in Vienna, Austria, on March 6-9, 2017, where he presented the poster entitled “Robot-assisted autism spectrum disorder diagnostics using POMDPs”. The poster contained the description of the proposed research aimed towards developing models for control and assessment of child-robot interaction within a robot-assisted autism spectrum disorder diagnostic protocol. Although the poster contained only preliminary information, it was recommended by the HRI reviewers to be included in the HRI PIONEERS 2017 Workshop.
Our Frano took part in the Fifth Croatian Computer Vision Workshop, held on October 11, 2016, in Osijek. Frano presented his work titled “Probabilistic Eye-Contact Detection for the Robot-assisted ASD Diagnostic protocol”, done within the ADORE project. We would like to thank Visage Technologies for providing us with their face-tracking software.
The local news portal Zg-Magazin published a story about our ADORE project. Our Damjan Miklić was interviewed about the project, the NAO robots, the project's results so far, the team working on it etc. You can read the full story here.
Job Fair is a career fair organized by the student association KSET and FER, and from the students' perspective, it is a unique opportunity to meet potential future employers. This year, our Faculty was also present as an employer, offering a number of vacant positions, from doctoral to postdoctoral. Since our postdoctorand Domagoj Tolić left for Dubrovnik, LARICS is offering a postdoctoral position with the project ADORE.
We are happy to have been a part of European Researchers’ Night, a manifestation that takes place every year on the last Friday of September across Europe, with more than 30 countries and 300 cities involved. This year we were present with our NAO robots, showcasing our research on the ADORE project. Additionally, our ex-students presented the hexapod robot they developed while working on the graduate thesis and used this opportunity to promote their company and learning platform STEMI.
Frano Petric presented the paper entitled “Towards a Robot-assisted Autism Diagnostic Protocol: Modelling and Assessment with POMDP” at the 8th International Conference on Intelligent Robots and Applications (ICIRA), held in Portsmouth, UK. The paper presented the system of automatic encoding of diagnostic tasks and is co-authored by Damjan Miklić, Domagoj Tolić, Zdenko Kovačić, and our colleagues from the Faculty of Education and Rehabilitation Sciences, Maja Cepanec and Sanja Šimleša.
On May 07, 2015, one of the best-selling Croatian weekly magazines Globus published an article about the use of NAO H25 humanoid robots for autism diagnostics in the LARICS project ADORE funded by the Croatian Science Foundation. The article was entitled “My doctor is a robot”. We would like to thank Tomislav Novak, a Globus journalist, for his enthusiasm to present our work and thus popularize robotics among the readers of the magazine.
The newly started regional television N1 hosted Maja Cepanec and Frano Petric in their show “Novi dan”, where they discussed ASD prevalence rates across Croatia, Bosnia and Herzegovina and Serbia along with the first results of the ADORE project. Maja and Frano were accompanied by one of our NAO robots which demonstrated some of its features. A video from the show is available in Croatian here.
ADORE project team members from the Faculty of Education and Rehabilitation Sciences presented a poster entitled “Challenges of using humanoid robots in the assessment of autism spectrum disorder“ at the AUTISM 2014: Geneva Centre for Autism International Symposium that was held from October 22-24 in Toronto, Canada.
Frano Petric attended the IEEE Global Humanitarian Technology Conference, held in San Jose (CA), USA. Frano presented the initial efforts and results of first clinical tests on the ADORE project with his paper entitled “Four Tasks of a Robot-assisted Autism Spectrum Disorder Diagnostic Protocol: First Clinical Tests“.
Professor Zdenko Kovačić and Frano Petric went to Düsseldorf to present ADORE project on the REHACARE fair. We would like to thank Miroslav Vrankić from the University of Rijeka, founder, and CEO of E-GLAS, for the invitation to join him at the fair. Professor Kovačić used this opportunity to talk with Miroslav about possible uses of the speech processing part of their Serwantess software in our ADORE applications.
The kickoff meeting for the ADORE project was held on 5th Septemeber 2018 at UNIZG-FER. The project logo has been inaugurated and organizational topics have been discussed. Our team members are optimistic and eager to get to work!
The ADORE: Autism Diagnostic Observation with Robot Evaluator project funded by Croatian Science Foundation (HRZZ-93743-2014) started today, with the signing of the contract between LARICS and HRZZ. The main goal of the project is to help clinicians from the Faculty of Education and Rehabilitation Sciences diagnose autism spectrum disorders more quickly and efficiently by using NAO robots made by Aldebaran Robotics. Through this project, the LARICS team will be working on developing a new set of motoric and cognitive skills for our humanoid robots.
Prof. Bojan Jerbić from the Faculty of Mechanical Engineering and Naval Architecture and our prof. Zdenko Kovačić were guests in the Croatian national television (HRT1) broadcast “Društvena mreža”, where they discussed the cognitive abilities of today's and future robot generations. Prof. Kovačić briefly presented the goals of the project ADORE – Autism Diagnostic Observation with Robot Evaluator, accepted for funding by the Croatian Science Foundation.
The Croatian daily TV show Dobro jutro, Hrvatska (Good morning, Croatia) invited assist. prof. Maja Cepanec (ERF-Faculty of Education and Rehabilitation Sciences), prof. Zdenko Kovačić and Frano Petric to say a few words about the new project ADORE – Autism Diagnostic Observation with Robot Evaluator, financed by the Croatian Science Foundation for the period of 2014-2018, in which LARICS’ NAO robots will play a central role.
The daily TV show Hrvatska uživo (Croatia live) aired a feature about a new project in the making in which LARICS’ NAO robots play a central role. The proposed project is based on interdisciplinary collaboration between researchers from the Faculty of Education and Rehabilitation Sciences and LARICS laboratory and aims to facilitate the diagnosing and treatment of Autism spectrum disorders by using robots as assistants.