Teaching and Learning
Pioneering Augmented Reality and Generative AI to Enhance Dysphagia Education
Dr Cecilia Kong Hoi-mei
|
Senior Lecturer
By 2050, according to the United Nations, Hong Kong will be a region with the highest ageing population in the world. According to the forecast by the Census and Statistics Department, 36% of its population will be 65 or above in 20461.
A 2020 study conducted in the UK states that up to 70% of care home residents are affected by dysphagia – swallowing difficulties – which in turn increases their risks of hospital admissions and morbidity.2 Most common causes of dysphagia in the older adults are strokes or neurodegenerative diseases such as Alzheimer’s, Parkinson’s disease. Timely diagnosis and management by an interprofessional team, including nurses, speech-language pathologists, dietitians and doctors, can help patients maintain dignity, eat safely and prevent aspiration pneumonia, thereby relieving burden in the healthcare system.3
With lightening the burden in mind, a team of experienced nursing educators and simulation teaching experts from the School of Nursing and from the Swallowing Research Lab in the Faculty of Education, have collaborated to pioneer the development of an Augmented Reality (AR) and Generative Artificial Intelligence (GenAI) Integrated Simulation Learning Platform for Dysphagia Management to enhance dysphagia education.
This project draws on successful traditional methods of teaching dysphagia management, and applies AR and GenAI to create hyper-realistic 3D avatar patients with high mannequins that display dynamic physiological responses, such as coughing, choking and swallowing difficulties. To bridge the gap between theory and practice, virtual patients and anatomical overlays are projected into the student’s real-world environment through tablets or smartphones. This allows the student to visualise the swallowing mechanism, identify aspiration risks, and practise clinical decision-making anytime, anywhere, without the anxiety of real-world consequences.
This immersive, scalable and replicable Learning Platform with enhanced realism and spacing and retrieval practice4 will help transform how healthcare professionals learn, identify, and manage swallowing disorders, and also build their confidence and competency.
The project underscores the importance of collaboration across disciplines in managing complex patient conditions. In these shared simulation scenarios, students from two disciplines come together to jointly assess swallowing function, plan interventions and prevent aspiration — mirroring real-world multidisciplinary teamwork.
Furthermore, students can observe in close proximity interprofessional education in action — the project team, comprising lecturers, senior lecturers, speech-language specialists, 3D designers, instructional designers and educational technologists, collaborates in design and assessments of competencies such as communication skills, teamwork, and confidence in clinical decision-making. This approach can provide valuable lessons applicable to undergraduate, postgraduate, and continuing professional development programmes.
Apart from enriching the body of knowledge of interprofessional education, the findings from this initiative are expected to guide the adaptation of AI-enhanced simulation models across other healthcare disciplines, by demonstrating how to integrate human expertise with digital intelligence to better prepare future professionals to deliver quality care in an AI-empowered clinical environment.
The project can open opportunities to further expand collaborative research within and beyond HKU, and foster formation of win-win, cross-institutional partnerships.
Last but not least, the outcomes can highlight the leadership of the two collaborating Faculties at HKU in advancing interdisciplinary education and promoting evidence-based practice to enhance the quality of care for patients with dysphagia.
References