Researchers in the field of Natural Language Processing have started to recognise the ease with which data and the application of data for algorithmic purposes can be used and misused, potentially leading to harm. The field has begun to identify the need to take responsibility and mitigate harm and misuse through documentation for their AI output, such as through "Model Cards" (Mitchell et al., 2019) and "Data Statements" (Bender et al., 2018). This project aims to undertake the early stages of co-design of an AI-powered search and moderation support for an online mental health forum. The forum is managed by SANE Australia, a mental health charity, who we will work with. The project team will use the findings from this case study to evaluate and enhance existing models of delivering systems in ethical AI. They propose that through thorough documentation of data provenance, intent of purpose and limitations of models, as well as a well-defined plan of deployment, we can mitigate the potential casualties of the "Crumple Zone"(Elish, 2019).
During the case study, they will assess every step of the AI development process from data creation to delivery to advocate for best practice in AI deployment. The study involves co-design with mental health professionals as well as people with lived experience of mental illness, followed by an evaluation of the technology design and of our design process. In this way data annotation will be designed in conjunction with professionals in the field as well as those it serves.
Model Cards aim to reduce potential harm by stating the limitations and intended use of a ready-built model and Data Statements aim to identify potential biases in the data by documenting the provenance of an annotated corpus. Through this proof-of-concept, the project team aims to learn if Model Cards and Data Statements are sufficient or lack information; and whether there should be an additional requirement to document the delivery of an AI model for real-world deployment.
Research Team
- Dr Mel Mistica
Research Data Specialist
Melbourne Data Analytics Platform
University of Melbourne
- Dr Greg Wadley
Senior Lecturer
School of Computing and Information Systems
University of Melbourne
- Professor Nicola Reavley
Principal Research Fellow, Mental Health Literacy Program
School of Population and Global Health
University of Melbourne
