Enhancing Engagement and Learning in Computing Education: Automated Moodle-Based Problem-Solving Assessments
summary
The gist
This paper presents the design and refinement of automated Moodle-based Problem-Solving Assessments (PSAs) deployed across large-scale computing units to replace traditional exams with a scalable…
In short
The research designed and improved automated Moodle-based Problem-Solving Assessments (PSAs) to replace traditional exams in large computing units. These PSAs use scenario-based, parameterised tasks delivered weekly workshops to boost learning authenticity, cognitive rigor, and resilience against Generative AI while ensuring scalability.
Key concepts
- Problem-Solving Assessments (PSAs)
- These are scenario-based tasks designed to simulate real-world computing problems. They focus on deeper learning rather than rote memorization by requiring students to apply knowledge in specific contexts.
- Parameterisation
- This design element ensures fairness and maintains academic integrity by adjusting task difficulty based on student performance or context. It allows for varied challenges while keeping the assessment rigorous and relevant.
- Spacing and Testing Effects
- PSAs are scheduled at key points during the semester (Weeks 5, 9, and 12). This strategic scheduling leverages psychological principles to improve long-term retention of material by testing knowledge at increasing intervals.
Terminology used across episodes
This episode discusses
- Enhancing Engagement and Learning in Computing Education: Automated Moodle-Based Problem-Solving Assessments · Paper Radio
The paper
Enhancing Engagement and Learning in Computing Education: Automated Moodle-Based Problem-Solving Assessments · Read on arXiv
Charith Jayasekara, Carlo Kopp, Vincent Lee, Chetan Arora
Department of Software Systems and Cybersecurity, Monash University · Department of Data Science and AI, Monash University
This paper presents the design and refinement of automated Moodle-based Problem-Solving Assessments (PSAs) deployed across large-scale computing units. Developed to replace traditional exams, PSAs assess applied problem-solving skills through parameterised, real-world tasks delivered via Moodle's quiz engine. Integrated with interactive workshops, this approach supports authentic learning, mitigates academic integrity risks, and reduces inconsistencies in marking. Iterative improvements have enhanced scalability, fairness, and alignment with learning outcomes. The model offers a practical and sustainable alternative for modern computing and engineering education.
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Enhancing Engagement and Learning in Computing Education".
Jane: Here's the gist of the paper. This paper presents the design and refinement of automated Moodle-based Problem-Solving Assessments (PSAs) deployed across large-scale computing units to replace traditional exams with a scalable, authentic alternative Assessment Model Philosophy The Moodle-based PSA model was purposefully developed to tackle challenges in authenticity, cognitive rigour, academic integrity, GenAI resilience, and scalability in computing assessments Unlike conventional digital exams, PSAs focus on deeper learning by engaging students in parameterised, scenario-based tasks aligned with real-world computing contexts Grounded in principles such as the spacing and testing effects, PSAs are scheduled at key points during the semester (Weeks 5, 9, and 12) Parameterisation is one of the key design pillars here, ensuring fairness while maintaining challenge and academic integrity All PSAs are delivered within weekly workshops to create a cohesive learning and assessment environment that promotes continuous participation Question Design and Automation The PSA model leverages a diverse set of Moodle question types to simulate realistic computing tasks while supporting automated marking at scale The following question types are prominently used in PSAs:: Numerical Calculated (short and long) i.e.
Paper summary: Tom: So, to summarize this paper's main point, they are designing and refining automated Moodle-based Problem-Solving Assessments. The goal is to swap out old exams for something that tests applied problem-solving skills using parameterised, real-world tasks delivered through Moodle’s quiz engine.
Jane: What they claim is that this model supports authentic learning by embedding these assessments within interactive workshops, and it aims to mitigate academic integrity risks while also cutting down on the marking workload because of the automated grading.
Lu: The paper focuses heavily on how these assessments are structured to assess threshold concepts in realistic scenarios, and they analyze how well this alignment between question structure and learning outcomes actually works through student feedback data.
Meng: I’m wondering about the mechanism itself; they list several question types, including numerical calculated questions for system-level tasks, multiple-choice single-answer questions, and even cloze questions. That variety sounds complex to set up consistently across a large course.
Lalam: The paper specifically highlights that numerical and calculated numerical questions are used for tasks like memory calculations or estimating disk access time estimation, and each student gets different parameter values to keep things unique.
Jane: That parameterization seems crucial because it directly addresses the risk of cheating by making sure no two students get the exact same test, which is a key feature they want to emphasize for fairness.
Tom: Yeah, and they are using these different question types—numerical, multiple-choice calculated and non-calculated—to simulate different aspects of real computing tasks while still keeping the assessment automated at scale. It's about simulating complexity without overwhelming the student in one go.
Lu: And they analyze the effectiveness of this alignment between what the questions are designed to test and what students actually learn, looking at performance data and student feedback to see if it’s working as intended.
Meng: What I want to know is how robust this setup is when you try to scale it up. If you deploy this across a really large unit, does the automated marking system handle the sheer volume of varied inputs efficiently?
Lalam: They state that iterative improvements have enhanced scalability and fairness, suggesting they've worked on making the deployment more sustainable and efficient over time.
Jane: So, they are building this as an evolving system rather than a static tool. This suggests they recognize that the initial design needs refinement based on how it actually performs in a real classroom setting.
Tom: And what matters is that they present it as a practical and sustainable alternative for modern computing education, moving away from those rigid, time-pressured final exams we talked about earlier.
Conclusion: Tom: So, wrapping up this discussion on the paper "Enhancing Engagement and Learning in Computing Education: Automated Moodle-Based Problem-Solving Assessments," the main thing is that they’ve put together a design for PSAs that aims to be a scalable, authentic replacement for traditional exams.
Jane: It boils down to using parameterised, scenario-based tasks delivered through Moodle to test applied problem-solving skills instead of just surface recall, which they claim helps keep students engaged throughout the semester.
Lu: The authors are showing how structuring the questions—using a mix of numerical and multiple-choice types tailored to specific computing tasks—can align better with what students need to learn in a meaningful way.
Meng: From my side, it seems like the real win is reducing marking overhead while increasing the complexity of what you can test at scale, but we still need solid data showing it holds up under heavy traffic.
Lalam: Lalam believes the system's strength lies in its ability to provide varied challenges through those parameter values while maintaining a structure that supports automated grading effectively.
Tom: Overall, this paper is pushing for an approach where assessments are continuous and tied to learning outcomes, suggesting a future where testing isn't just at the end of the course but woven into the learning process itself.
Jane: It suggests that by focusing on realistic problem-solving scenarios in a flexible platform like Moodle, we can create an assessment method that feels more authentic and less stressful for students while still giving us better data on their actual abilities.
More episodes
- 2505.16254-Human-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMs
- 2506.10913-Choreographic Quick Changes: First-Class Location (Set) Polymorphism
- 2507.14812-A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation
- 2507.15448-Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes
- 2508.21572-NewsReX: An Open-Source Multi-Framework for Neural News Recommendation
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines
- 2601.15318-On the closest balanced game