Enhancing Engagement and Learning in Computing Education: Automated Moodle-Based Problem-Solving Assessments
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Enhancing Engagement and Learning in Computing Education".
Jane: Here's the gist of the paper. This paper presents the design and refinement of automated Moodle-based Problem-Solving Assessments (PSAs) deployed across large-scale computing units to replace traditional exams with a scalable, authentic alternative Assessment Model Philosophy The Moodle-based PSA model was purposefully developed to tackle challenges in authenticity, cognitive rigour, academic integrity, GenAI resilience, and scalability in computing assessments Unlike conventional digital exams, PSAs focus on deeper learning by engaging students in parameterised, scenario-based tasks aligned with real-world computing contexts Grounded in principles such as the spacing and testing effects, PSAs are scheduled at key points during the semester (Weeks 5, 9, and 12) Parameterisation is one of the key design pillars here, ensuring fairness while maintaining challenge and academic integrity All PSAs are delivered within weekly workshops to create a cohesive learning and assessment environment that promotes continuous participation Question Design and Automation The PSA model leverages a diverse set of Moodle question types to simulate realistic computing tasks while supporting automated marking at scale The following question types are prominently used in PSAs:: Numerical Calculated (short and long) i.e.
Paper summary: Tom: So, to summarize this paper's main point, they are designing and refining automated Moodle-based Problem-Solving Assessments. The goal is to swap out old exams for something that tests applied problem-solving skills using parameterised, real-world tasks delivered through Moodle’s quiz engine.
Jane: What they claim is that this model supports authentic learning by embedding these assessments within interactive workshops, and it aims to mitigate academic integrity risks while also cutting down on the marking workload because of the automated grading.
Lu: The paper focuses heavily on how these assessments are structured to assess threshold concepts in realistic scenarios, and they analyze how well this alignment between question structure and learning outcomes actually works through student feedback data.
Meng: I’m wondering about the mechanism itself; they list several question types, including numerical calculated questions for system-level tasks, multiple-choice single-answer questions, and even cloze questions. That variety sounds complex to set up consistently across a large course.
Lalam: The paper specifically highlights that numerical and calculated numerical questions are used for tasks like memory calculations or estimating disk access time estimation, and each student gets different parameter values to keep things unique.
Jane: That parameterization seems crucial because it directly addresses the risk of cheating by making sure no two students get the exact same test, which is a key feature they want to emphasize for fairness.
Tom: Yeah, and they are using these different question types—numerical, multiple-choice calculated and non-calculated—to simulate different aspects of real computing tasks while still keeping the assessment automated at scale. It's about simulating complexity without overwhelming the student in one go.
Lu: And they analyze the effectiveness of this alignment between what the questions are designed to test and what students actually learn, looking at performance data and student feedback to see if it’s working as intended.
Meng: What I want to know is how robust this setup is when you try to scale it up. If you deploy this across a really large unit, does the automated marking system handle the sheer volume of varied inputs efficiently?
Lalam: They state that iterative improvements have enhanced scalability and fairness, suggesting they've worked on making the deployment more sustainable and efficient over time.
Jane: So, they are building this as an evolving system rather than a static tool. This suggests they recognize that the initial design needs refinement based on how it actually performs in a real classroom setting.
Tom: And what matters is that they present it as a practical and sustainable alternative for modern computing education, moving away from those rigid, time-pressured final exams we talked about earlier.
Conclusion: Tom: So, wrapping up this discussion on the paper "Enhancing Engagement and Learning in Computing Education: Automated Moodle-Based Problem-Solving Assessments," the main thing is that they’ve put together a design for PSAs that aims to be a scalable, authentic replacement for traditional exams.
Jane: It boils down to using parameterised, scenario-based tasks delivered through Moodle to test applied problem-solving skills instead of just surface recall, which they claim helps keep students engaged throughout the semester.
Lu: The authors are showing how structuring the questions—using a mix of numerical and multiple-choice types tailored to specific computing tasks—can align better with what students need to learn in a meaningful way.
Meng: From my side, it seems like the real win is reducing marking overhead while increasing the complexity of what you can test at scale, but we still need solid data showing it holds up under heavy traffic.
Lalam: Lalam believes the system's strength lies in its ability to provide varied challenges through those parameter values while maintaining a structure that supports automated grading effectively.
Tom: Overall, this paper is pushing for an approach where assessments are continuous and tied to learning outcomes, suggesting a future where testing isn't just at the end of the course but woven into the learning process itself.
Jane: It suggests that by focusing on realistic problem-solving scenarios in a flexible platform like Moodle, we can create an assessment method that feels more authentic and less stressful for students while still giving us better data on their actual abilities.
Charith Jayasekara, Carlo Kopp, Vincent Lee, Chetan Arora
Department of Software Systems and Cybersecurity, Monash University · Department of Data Science and AI, Monash University
cs.CY
Submitted: 2025-08-24
Updated: 2025-08-24
Comments: Accepted and presented at the Asia-Pacific Conference on Education, Teaching & Technology (AP-EduTeach 2025), Bangkok, Thailand. This is the author-accepted manuscript version. The final version will appear in the official conference proceedings. (ISBN 978-1-988652-89-4)
Journal ref: International Multidisciplinary Research Journal 3 (2025) 1-12
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 86/100
The gist: This paper presents the design and refinement of automated Moodle-based Problem-Solving Assessments (PSAs) deployed across large-scale computing units to replace traditional exams with a scalable…
Key concepts
- Problem-Solving Assessments (PSAs)
- These are scenario-based tasks designed to simulate real-world computing problems. They focus on deeper learning rather than rote memorization by requiring students to apply knowledge in specific contexts.
- Parameterisation
- This design element ensures fairness and maintains academic integrity by adjusting task difficulty based on student performance or context. It allows for varied challenges while keeping the assessment rigorous and relevant.
- Spacing and Testing Effects
- PSAs are scheduled at key points during the semester (Weeks 5, 9, and 12). This strategic scheduling leverages psychological principles to improve long-term retention of material by testing knowledge at increasing intervals.
Terminology
Summary
This paper presents the design and refinement of automated Moodle-based Problem-Solving Assessments (PSAs) deployed across large-scale computing units to replace traditional exams with a scalable, authentic alternative
Assessment Model Philosophy
The Moodle-based PSA model was purposefully developed to tackle challenges in authenticity, cognitive rigour, academic integrity, GenAI resilience, and scalability in computing assessments Unlike conventional digital exams, PSAs focus on deeper learning by engaging students in parameterised, scenario-based tasks aligned with real-world computing contexts Grounded in principles such as the spacing and testing effects, PSAs are scheduled at key points during the semester (Weeks 5, 9, and 12) Parameterisation is one of the key design pillars here, ensuring fairness while maintaining challenge and academic integrity All PSAs are delivered within weekly workshops to create a cohesive learning and assessment environment that promotes continuous participation
Question Design and Automation
The PSA model leverages a diverse set of Moodle question types to simulate realistic computing tasks while supporting automated marking at scale The following question types are prominently used in PSAs:
**: Numerical Calculated (short and long) i.e.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Automated Problem-Solving Assessments (PSAs) can be deployed across large-scale computing units to assess applied problem-solving skills through parameterised, real-world tasks delivered via Moodle’s quiz engine. This supports authentic learning and reduces inconsistencies in marking by incorporating automated marking logic and partial scoring rules.
-
The AI system can generate unique versions of each task for students with varied inputs, values, or sequences to ensure fairness while maintaining academic integrity through parameterisation. This eliminates the need for traditional exam halls or seating plans by using randomised parameters and shuffled orderings.
-
The system can integrate assessments directly within interactive workshops, creating a cohesive learning and assessment environment that promotes continuous participation. This structure enhances engagement and strengthens the alignment between learning activities and assessment tasks throughout the semester.
-
Question types can be leveraged to simulate realistic computing tasks by using numerical calculated questions for algorithmic tasks or system-level metrics, as seen in the example involving calculating flow rules or traffic intensity. This moves beyond rote recall by requiring multi-step calculations involving throughput, resource allocation, and performance evaluation.
-
The system can utilize Cloze (embedded) questions to present multi-step reasoning within a single narrative, which enables a controlled build-up of complexity to assess problem-solving processes. This scaffolded approach supports both accessibility and challenge while requiring students to demonstrate conceptual understanding in applied contexts like paging operations or multi-stage computations.
-
The AI system can employ adaptive sequencing based on performance data, as seen in the iterative refinement process, to increase diagnostic value and fairness of assessments. This allows the model to adjust question complexity over successive offerings, ensuring that average scores improve through spaced assessment and progressive challenge.
-
The system can provide consistent partial marking logic for nuanced evaluation via automation within Moodle, which enhances marking reliability and reduces long term administrative overhead. This provides timely feedback while supporting grading transparency, unlike legacy methods prone to delays or inconsistencies.
Abstract
This paper presents the design and refinement of automated Moodle-based Problem-Solving Assessments (PSAs) deployed across large-scale computing units. Developed to replace traditional exams, PSAs assess applied problem-solving skills through parameterised, real-world tasks delivered via Moodle's quiz engine. Integrated with interactive workshops, this approach supports authentic learning, mitigates academic integrity risks, and reduces inconsistencies in marking. Iterative improvements have enhanced scalability, fairness, and alignment with learning outcomes. The model offers a practical and sustainable alternative for modern computing and engineering education.
Related papers
- AI-Mediated Communication Can Steer Collective Opinion
- Epistemic Infrastructures of Science in AI Era Should Rebalance Costs of Generation and Verification
- AI out-persuades expert humans in text-based dialogue
- Evaluating Ambient Clinical Scribes in India: The Need for Multilingual Real-World Clinical Conversation Data
- AI Value Alignment for Evolving Social Norms
- Voting Biases in Decentralized Autonomous Organization (DAO) Governance