Enhancing student communication using speech technology and digital tools

Over the past two decades, computer-assisted learning has transformed English as a second language (ESL) classrooms, particularly for teaching pronunciation. However, existing tools often lack advanced feedback suited to professional and academic contexts and are often aimed at beginner learners. Developing effective science-based tools requires specialised knowledge in linguistics, phonetics, digital humanities and data science.

This collaborative project addresses the existing gap by developing a pronunciation feedback app. This web-based app will provide advanced, self-guided pronunciation feedback for ESL learners. It will initially be implemented in the graduate ESL subject ESLA90001 Professional Speaking Communication.

Using OpenAI's Whisper technology, the app will support assessment tasks and strengthen students’ ability to practice individually, guided by speech science expertise. User experience will be evaluated through student feedback.

After the pilot, the app will expand to other ESL and Master of Applied Linguistics subjects, enhancing feedback for graduates entering global and Australian workplaces. The app also has the potential commercial viability, as currently there are no effective pronunciation feedback technologies based on scientific principles for advanced learners, despite widespread interest.

MDAP will provide technical expertise to create the app, including capabilities in working with academics on computational needs and building platforms using audio and large language models.

The key outcomes of the project include the development of an app using innovative pedagogical tools, the deployment of an open-source app on Hugging Face (compatible with Transformers.js) for easy course website embedding, and the creation of a comprehensive database of speech recordings.