Enhancing Engagement and Learning in Computing Education: Automated Moodle-Based Problem-Solving Assessments

summary

The gist

This paper presents the design and refinement of automated Moodle-based Problem-Solving Assessments (PSAs) deployed across large-scale computing units to replace traditional exams with a scalable…

In short

The research designed and improved automated Moodle-based Problem-Solving Assessments (PSAs) to replace traditional exams in large computing units. These PSAs use scenario-based, parameterised tasks delivered weekly workshops to boost learning authenticity, cognitive rigor, and resilience against Generative AI while ensuring scalability.

Key concepts

Problem-Solving Assessments (PSAs)
These are scenario-based tasks designed to simulate real-world computing problems. They focus on deeper learning rather than rote memorization by requiring students to apply knowledge in specific contexts.
Parameterisation
This design element ensures fairness and maintains academic integrity by adjusting task difficulty based on student performance or context. It allows for varied challenges while keeping the assessment rigorous and relevant.
Spacing and Testing Effects
PSAs are scheduled at key points during the semester (Weeks 5, 9, and 12). This strategic scheduling leverages psychological principles to improve long-term retention of material by testing knowledge at increasing intervals.

Terminology used across episodes

This episode discusses

The paper

Enhancing Engagement and Learning in Computing Education: Automated Moodle-Based Problem-Solving Assessments · Read on arXiv

Charith Jayasekara, Carlo Kopp, Vincent Lee, Chetan Arora

Department of Software Systems and Cybersecurity, Monash University · Department of Data Science and AI, Monash University

This paper presents the design and refinement of automated Moodle-based Problem-Solving Assessments (PSAs) deployed across large-scale computing units. Developed to replace traditional exams, PSAs assess applied problem-solving skills through parameterised, real-world tasks delivered via Moodle's quiz engine. Integrated with interactive workshops, this approach supports authentic learning, mitigates academic integrity risks, and reduces inconsistencies in marking. Iterative improvements have enhanced scalability, fairness, and alignment with learning outcomes. The model offers a practical and sustainable alternative for modern computing and engineering education.

DOI: 10.47722/imrj.2001.49

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Enhancing Engagement and Learning in Computing Education".

Jane: Here's the gist of the paper. This paper presents the design and refinement of automated Moodle-based Problem-Solving Assessments (PSAs) deployed across large-scale computing units to replace traditional exams with a scalable, authentic alternative Assessment Model Philosophy The Moodle-based PSA model was purposefully developed to tackle challenges in authenticity, cognitive rigour, academic integrity, GenAI resilience, and scalability in computing assessments Unlike conventional digital exams, PSAs focus on deeper learning by engaging students in parameterised, scenario-based tasks aligned with real-world computing contexts Grounded in principles such as the spacing and testing effects, PSAs are scheduled at key points during the semester (Weeks 5, 9, and 12) Parameterisation is one of the key design pillars here, ensuring fairness while maintaining challenge and academic integrity All PSAs are delivered within weekly workshops to create a cohesive learning and assessment environment that promotes continuous participation Question Design and Automation The PSA model leverages a diverse set of Moodle question types to simulate realistic computing tasks while supporting automated marking at scale The following question types are prominently used in PSAs:: Numerical Calculated (short and long) i.e.

Paper summary: Tom: So, to summarize this paper's main point, they are designing and refining automated Moodle-based Problem-Solving Assessments. The goal is to swap out old exams for something that tests applied problem-solving skills using parameterised, real-world tasks delivered through Moodle’s quiz engine.

Jane: What they claim is that this model supports authentic learning by embedding these assessments within interactive workshops, and it aims to mitigate academic integrity risks while also cutting down on the marking workload because of the automated grading.

Lu: The paper focuses heavily on how these assessments are structured to assess threshold concepts in realistic scenarios, and they analyze how well this alignment between question structure and learning outcomes actually works through student feedback data.

Meng: I’m wondering about the mechanism itself; they list several question types, including numerical calculated questions for system-level tasks, multiple-choice single-answer questions, and even cloze questions. That variety sounds complex to set up consistently across a large course.

Lalam: The paper specifically highlights that numerical and calculated numerical questions are used for tasks like memory calculations or estimating disk access time estimation, and each student gets different parameter values to keep things unique.

Jane: That parameterization seems crucial because it directly addresses the risk of cheating by making sure no two students get the exact same test, which is a key feature they want to emphasize for fairness.

Tom: Yeah, and they are using these different question types—numerical, multiple-choice calculated and non-calculated—to simulate different aspects of real computing tasks while still keeping the assessment automated at scale. It's about simulating complexity without overwhelming the student in one go.

Lu: And they analyze the effectiveness of this alignment between what the questions are designed to test and what students actually learn, looking at performance data and student feedback to see if it’s working as intended.

Meng: What I want to know is how robust this setup is when you try to scale it up. If you deploy this across a really large unit, does the automated marking system handle the sheer volume of varied inputs efficiently?

Lalam: They state that iterative improvements have enhanced scalability and fairness, suggesting they've worked on making the deployment more sustainable and efficient over time.

Jane: So, they are building this as an evolving system rather than a static tool. This suggests they recognize that the initial design needs refinement based on how it actually performs in a real classroom setting.

Tom: And what matters is that they present it as a practical and sustainable alternative for modern computing education, moving away from those rigid, time-pressured final exams we talked about earlier.

Conclusion: Tom: So, wrapping up this discussion on the paper "Enhancing Engagement and Learning in Computing Education: Automated Moodle-Based Problem-Solving Assessments," the main thing is that they’ve put together a design for PSAs that aims to be a scalable, authentic replacement for traditional exams.

Jane: It boils down to using parameterised, scenario-based tasks delivered through Moodle to test applied problem-solving skills instead of just surface recall, which they claim helps keep students engaged throughout the semester.

Lu: The authors are showing how structuring the questions—using a mix of numerical and multiple-choice types tailored to specific computing tasks—can align better with what students need to learn in a meaningful way.

Meng: From my side, it seems like the real win is reducing marking overhead while increasing the complexity of what you can test at scale, but we still need solid data showing it holds up under heavy traffic.

Lalam: Lalam believes the system's strength lies in its ability to provide varied challenges through those parameter values while maintaining a structure that supports automated grading effectively.

Tom: Overall, this paper is pushing for an approach where assessments are continuous and tied to learning outcomes, suggesting a future where testing isn't just at the end of the course but woven into the learning process itself.

Jane: It suggests that by focusing on realistic problem-solving scenarios in a flexible platform like Moodle, we can create an assessment method that feels more authentic and less stressful for students while still giving us better data on their actual abilities.

More episodes

← Home