Melbourne CSHE Teaching and Learning Summit 2024

On Wednesday 5 June 2024, academic and professional staff from The University of Melbourne (and some keynote speakers from outside the University), came together to showcase and discuss the broader implications of artificial intelligence (AI) for teaching, learning and assessment. This article provides a brief overview of some of the presentations and key messages delivered.

We were lucky to have both some great spotlights on individual practice and key-notes from leaders within the assessment and AI ethics research space. Presentations and panel sessions covered key themes related to AI; literacy and employment; the future of assessment and AI; and ethical challenges. The summit themes explored the opportunities and challenges posed by generative AI for the sector and the value of education overall.

Key takeaways

Generative AI will continue to grow in power and disrupt Higher Education. If we focus on mitigating cheating risks we are missing the bigger picture (both positive and negative) around AI in higher education. We need to move to nuanced rather than binary discussions in this space. There are several challenges with the use of Generative AI that pose ethical and other risks, but Higher Education can also help shape the future of AI. Evaluative judgement and other ‘human’ skills will become equally if not more important in an age of AI.

Key highlights

Professor Christopher Ziguras opened and set the scene for the day by acknowledging Country and also acknowledging challenges in the area of AI and how rapidly it is changing, which is a significant catalyst for the 2024 summit.

AI literacy, employment and universities

Associate Professor Jason Lodge (The University of Queensland) discussed AI literacy, employment and universities. There are fundamental changes happening now,  and we need to question whether this is for the better. We should be guiding change as opposed to tech companies. We currently focus on the acute problem of cheating, but not enough on the chronic problem of how and what do we teach now? We need to address both, as both problems are linked.

Some key messages from the presentation were:

  • Comparing AI to a RAV4 car. We don’t need to know the inner workings of the machinery in order to drive a car. Current Generative AI allows us to ask the technology how to use it. If so, is learning to understand the technology still important, or is it more important to instead understand ourselves? We can’t expect students to make the most of these tools if they don’t understand their own learning first.
  • Literacy skills are important but need to be considered in balance. Students are using AI for explaining things, brainstorming ideas, and a smaller amount for assessment writing. Student usage trends suggest that, as with other technology trends, there are large gaps between use of AI – some students with strong literacy are using AI innovatively, some use it superficially but don’t trust it and will fact-check, and some are scared to use it, often due to a fear of being labelled as cheating.
  • Universities are preparing students for a world that was 10 years ago. There is disagreement as to when AI will surpass human intelligence. But rapid changes in AI tools make focusing solely on technology ineffective.
  • Humans vs robots - what are each good at? Robots are not embodied humans so can’t make the same inferences based on situational, cultural and social data. To make the most of these differences and to prepare students effectively for the future, we should focus on teaching evaluative judgement, sense-making and self-regulated learning.

The future of assessment and AI

Professor Margaret Bearman, Centre for Research in Assessment and Digital Learning (CRADLE), Deakin University discussed the need for assessment reform and the importance of evaluative judgment.

  • Discourse around Chat GPT tends to be polarised between excited or scared. But these types of polarisations remove the space needed for the nuanced conversations that we need to have.
  • Academic integrity on both sides: There is often an assumption that students are cheating, but Wu et al. (2024) showed that the highest Reddit threads around education and student use were students seeking help for false accusations of cheating. Assessment integrity is a nuanced issue that cuts both ways.
  • In a time of AI we need to think about assessment design. As highlighted in Jason Lodge’s presentation, Bearman argues we need to be considering both short term acute challenges and the long-term purpose of Higher Education and use this to shape assessment design including graded and non-graded tasks. Assessment should promote feedback, iterative opportunities and discourage mechanical approaches.
  • Machines don’t arbitrate quality, we do. Assessment is an ideal modality for understanding quality. Assessment both assures and promotes learning.
  • The need to build evaluative judgement. What counts as “good” is a socially derived, contextual evolving process, and changes over time and location. To better evaluate what ‘good’ looks like requires evaluative judgement, which is a key skill needed in the age of AI.
  • Assessment – it's complicated. Assessment design towards authenticity is suggested to avoid AI challenges. However, it often raises other challenges. Building better purpose and authenticity might increase motivation but doesn’t solve academic integrity or cheating. However invigilation is costly and stressful, and only allows for a narrow band of capabilities to be tested, so we need to change assessment design as good practice regardless.
  • Thinking beyond the individual unit. A single assessment design is unlikely to be cheat proof and not every assessment can be cost and labour intensive in terms of marking and feedback load or invigilation costs. We need to look beyond the individual unit/subject to think and design programmatically across the course. In this model, we can put resources into key units and for validating integrity and security at meaningful points.

Engaging with ethical challenges of AI

Professor Jeannie Patterson, Centre for AI and Digital Ethics (CAIDE), University of Melbourne focused on the broader ethical implications of AI in education and the workplace including opportunities and risks, and what The University of Melbourne is doing in response to these.

  • It’s not just about cheating: Patterson reiterated Lodge and Bearman, noting we should not just be talking about cheating. There are broader implications for the workplace, education and society that we need to be discussing and addressing.
  • Risks aren’t only to education: AI is changing the fundamental expected skills in the workplace. Deepfakes, AI bias, privacy, data leaks and IP and copyright risks affect business and the community. If AI is changing workplace skills and contexts, then what is the role and purpose of Higher Education in this new landscape?
  • On the ground responses: Training, guidelines and policies, technological innovation are needed to address ethical concerns. Government and University policy and law is moving to support this. Locally, The University of Melbourne AI Principles outline AI literacy, academic integrity training, privacy and security.
  • Transparency: The importance of responsible AI that respects human rights and autonomy. AI vendors aren’t necessarily transparent about how their software works or how accurate it is. They are in the business of obfuscating, we need to be holding them to account.
  • Ethics is a conversation not destination but important in context of discipline. Ethics don’t provide the answer, they engage in the conversations and encourage us to keep up with changes.
  • Mutual opportunities for research: what AI can contribute to research projects, and what researchers and artists can bring to AI.

Spotlight sessions

Spotlight sessions from academic staff and students within The University of Melbourne showcased a range of examples of how AI can be used within education:

  • Dr Solange Glasser highlighted the challenges of balancing efficiency with ethical and pedagogical concerns as part of redesigned subject and assessments and incorporating Generative AI as a study bot in her Music, Mind and Wellbeing subject (FlexAP redesign).
  • Allison Clarke piloted SparkAI and noted the opportunities and efficiencies for qualitative research, while also noting the important role researchers and their ‘human’ skills still play in ensuring accuracy and ethics of Gen AI data.
  • Oliver Cucanic (PhD Candidate) provided an overview of Sindy, AI software he is developing with support from University of Melbourne InnovatEd.
  • Dr Wasana Karunarathne demonstrated a case study on the use of a chatbot integration within subjects for improved student experience and resources access, as part of FlexAP project and Learning Environments team support.
  • Dr Christopher Honig discussed the importance of accuracy in AI and the different expectations around accuracy between AI chatbots and humans.
  • Dr Marc Cheong and colleague demonstrated that philosophers and ethicists also have a comedic side in their humourous presentation, while also presenting how they have iteratively redesigned their subject matter and assessments in ethics subjects due to the evolving nature of AI in disrupting both assessment integrity and what students need to know.

Find out more about AI at University of Melbourne

Further resources