Scaffolding GenAI Use in Ethical Essay Writing: A Three-Year Experience Report in Introductory Computing

arXiv:2611.08431 · cs.CY · Submitted 2026-11-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "Scaffolding GenAI Use in Ethical Essay Writing".

Jane: Here's the gist of the paper. Scaffolding GenAI Use in Ethical Essay Writing: A Three-Year Experience Report in Introductory Computing examines how students use Generative AI when writing ethical essays across three years of a first-year computer systems course and finds that pedagogical value depends on how assessment design guides students to evaluate and critically engage with AI-generated suggestions Study Design and Context The experience report examines GenAI use across three years of ethical essay assignments in a large first-year computer systems course The study analyzed 1,894 responses from six post-essay surveys conducted between 2024 and 2026, linking survey data with essay grades from 2025 and 2026 The research followed a repeated cross-sectional design across three successive cohorts of the same course to show how practices changed at the course level rather than for individual students Each semester included two short essay assignments on ethical issues in computing, which were worth 8% of the final grade and required students to interpret prompts, construct arguments, and identify stakeholders Adoption Patterns and Tool Usage The adoption of GenAI tools varied significantly depending on the policy framing Under voluntary use policies from 2024 to 2025, AI adoption remained steady at around two-thirds of students, while under the 2026 expected-use policy, it rose above 90% for both essays Tool choice diversified beyond ChatGPT in 2026, with Claude and Gemini capturing roughly one-tenth of users each Students reported mainly using AI for help with grammar and style, as brainstorming fell sharply from 47% in 2025 to 15% in 2026 Impact on Essay Quality Grade-linked analyses showed that AI use did not independently predict essay quality once prior writing ability was controlled In both Phase 2 (unscaffolded) and Phase 3 (scaffolded), ANCOVA models showed that AI use had no independent effect on essay quality when prior writing ability was held constant Score changes across the four adoption trajectories were not significantly different in either phase, indicating that AI use alone does not explain essay quality Critical Engagement Under Scaffolding The most significant findings related to how students engaged with AI suggestions under the scaffolded Phase 3 workflow Students reported high levels of responsible AI engagement, with rejection or modification of suggestions reported by 81% in Essay 1 and 84% in Essay 2 Marker-coded evaluations confirmed this trend, showing that positive AI tone increased from 47% to 66%, while negative ratings fell from 9.1% to under 1% Markers coded that students showed critical engagement at rates of 77–81% and genuine insight at rates of 81–85% in Phase 3 submissions Pedagogical Implications The study concludes that AI use is shaped by policy and assessment design as much as by tool availability, suggesting that assessment design can shape whether students use AI selectively rather than uncritically The central pedagogical insight is that engagement quality and outcome quality are distinct goals, and scaffolding may advance the former without improving the latter The authors recommend keeping the core scaffold—the draft-before-AI step with its checklist, mandatory disclosure, and marked reflection—as a low-cost way to assess ethics within a technical course This approach helps students evaluate, adapt, and take responsibility for AI suggestions Limitations The experience report is limited as a repeated cross-sectional study that cannot track individual student practices over time The analysis relies largely on self-reported AI use, which is prone to recall and social desirability bias Furthermore, the Phase 3 redesign limits causal interpretation because both essays used the scaffolded workflow, mixing design effects with cohort effects The findings are therefore treated as formative evidence of how scaffolding shaped students’ AI practices rather than causal evidence that it improves essay performance In summary, adoption held at about two-thirds under voluntary use and exceeded 90% under scaffolded, expected use policies Students mainly used AI for writing support, and tool use diversified beyond ChatGPT Neither AI use nor adoption trajectory independently explained essay quality in either phase The clearest Phase 3 findings were behavioral, with students reporting high levels of modification and verification, and markers saw widespread critical engagement and genuine insight The value of GenAI in writing-intensive computing assignments appears to lie less in permitting its use than in designing structures that help students evaluate, adapt, and take responsibility for AI suggestions The paper is available at arXiv:2611.08431v1 [cs.

Title and authors: Tom: So we’re looking at this paper called "Scaffolding GenAI Use in Ethical Essay Writing: A Three-Year Experience Report in Introductory Computing." It looks like they tracked how students used AI when writing these ethical essays over three years in a first-year computer systems course, and they found that how the assessment is designed really matters for whether students use the AI responsibly.

Jane: Exactly. The core idea is that just letting students use AI without any guardrails doesn't teach them to think critically about the ethics of using that technology, so this report looks at how different policies change student behavior over time.

Lu: What’s interesting here is that the adoption of GenAI tools shifted quite a bit depending on whether the course allowed it voluntarily or required it under an expected-use policy, and they saw that as the rules got stricter, more students started using AI for things other than just grammar help.

Meng: I’m curious about what kind of use they found—was it mostly brainstorming ideas, or was it more about polishing the final product? Because from a production standpoint, we need to know if students are using the tool to save time or if they are actually using it to learn something new.

Lalam: The study shows that brainstorming dropped significantly from forty-seven percent in twenty twenty five down to just fifteen percent in twenty twenty six, which suggests that as the requirements got more structured, the way students engaged with the AI changed.

Tom: That shift is telling me that when you give them a specific job to do before they use AI, they stop treating it like a magic answer machine and start treating it like a collaborator or maybe just an editor.

Jane: And that brings us to what the paper suggests are actual improvements for how educators can design these assignments. They argue that the key isn't just letting students use AI, but building in steps that force them to evaluate and take responsibility for whatever the AI suggests.

Title and authors: Lu: They actually propose a specific scaffolded workflow where students have to write a complete draft independently with a checklist before they can even touch an AI tool for refinement, which is pretty structural.

Meng: A mandatory independent draft sounds like it would be great for testing real-world scenarios because that forces them to build the argument themselves before outsourcing the heavy lifting to the AI.

Tom: And then they add this checklist requirement where students have to identify at least two stakeholders and include a counterargument, which makes sure they are hitting all the required ethical components of an essay, not just getting a good sounding paragraph.

Jane: Then there’s this part about verification—students can’t verify AI suggestions before they use them, but they have to provide at least two credible references for every claim in their final essay because the AI can be inaccurate.

Lalam: That sounds like a major safety net because it forces them to connect the ideas generated by the AI back to established knowledge, which is something I see as crucial when we deploy any kind of large language model.

Lu: The paper also stresses structured reflection after using the AI, requiring one hundred to two hundred words where they discuss how their draft differs from the final essay and what improvements they made during that interaction with the AI.

Meng: That kind of reflection is important for us because we need to know what breaks at scale when we integrate these tools into our own systems; knowing how a user interacts with the model helps us predict where things might fail in production.

Tom: So, the paper suggests that the pedagogical value isn't in permitting AI use at all, but in designing structures that guide students to evaluate and adapt those suggestions rather than just passively accepting them as truth.

Jane: That’s the central argument—that engagement quality and outcome quality are two different goals, so scaffolding can advance one without necessarily improving the other.

Title and authors: Lu: The authors found that when this scaffolded workflow was used in Phase three, students reported high levels of responsible AI engagement, with eighty-one to eighty-four percent of them rejecting or modifying the suggestions they got from the AI.

Meng: That rejection rate is pretty telling; it suggests that even with a powerful tool available, students are actively using their critical thinking skills to filter out what’s not right.

Lalam: Marker coding confirmed this trend too; they saw positive AI tone increase from forty-seven percent up to sixty-six percent, and negative ratings dropped from nine point one percent down to under one percent in the scaffolded phase.

Tom: It shows that when you give them a process that requires them to pause and check their work, they are much more likely to be critical partners with the AI instead of just letting it write for them.

Jane: So, when we look at the conclusion of this experience report on "Scaffolding GenAI Use in Ethical Essay Writing: A Three-Year Experience Report in Introductory Computing," what we see is that policy and assessment design shape student use more than the tools themselves do.

Lu: And they conclude that keeping that core scaffold—the draft-before-AI step with its checklist, mandatory disclosure, and marked reflection—is a low-cost way to assess ethics within a technical course.

Meng: From an engineering view, that makes sense because it’s a repeatable process we could bake into any system design or evaluation pipeline to ensure accountability.

Lalam: It’s about making the responsibility of using the AI explicitly part of the assessment itself, which is how I see it impacting people who use models like me.

Tom: So, to wrap up this discussion on "Scaffolding GenAI Use in Ethical Essay Writing: A Three-Year Experience Report in Introductory Computing," it seems that designing a process that demands verification and reflection is what actually teaches students the ethical side of using AI.

Jane: It really does. And for us listening, it suggests that if you’re teaching technical skills alongside ethics, building in those mandatory evaluation checkpoints is the way to go instead of just handing over the whole essay to a machine.

The paper's summary: Tom: So, we're looking at this report on GenAI use in ethical essay writing across three years of an introductory computer systems course, and the main thing they found is that how you set up the assessment—the scaffolding—really dictates whether students use AI responsibly.

Jane: Yeah, it shows that just letting students write and then checking the final grade isn't enough; you have to design a process where they are forced to interact with the AI in a specific way.

Lu: They tracked one thousand eight hundred ninety-four student responses across three years, looking at how their AI habits changed depending on whether the course policy was voluntary or required.

Meng: What they really dug into was how students handled AI suggestions when there was a scaffolded workflow versus when it wasn't. That’s where the real behavioral data is.

Tom: Because in that scaffolded phase, students were actually rejecting or modifying more than half of the AI's suggestions, which is a huge sign they weren't just accepting whatever came out of the box.

Jane: And marker codes confirmed that when they had that structure—the draft-before-AI step with a checklist and mandatory reflection—students showed high levels of critical engagement, around seventy to eighty percent.

Lu: They saw positive AI tone actually increasing from forty seven percent up to sixty six percent in the scaffolded submissions, but negative ratings dropped significantly.

Tom: That tells you that the process itself is shaping their interaction with the tool; they aren't just blindly hitting generate and hoping for the best anymore.

Jane: The core message here is that engagement quality and outcome quality are two different things, and scaffolding helps you get students to engage critically without necessarily making their final essay better, though it probably does help them think better overall.

Lu: The authors suggest that the most important thing for educators right now isn't just whether a tool is available; it’s designing structures that force students to evaluate and take responsibility for what the AI suggests.

Tom: They are recommending keeping that basic scaffold—the draft-before-AI step with its checklist, mandatory disclosure, and marked reflection—as a low-cost way to assess ethics in these types of courses.

Jane: It’s about making the responsibility of using the AI explicit part of the assessment itself, which is how we actually teach students how to use these powerful tools safely.

Lu: The report is limited because it's a repeated cross-sectional study, so we can see what happened with policy changes, but we can't say for sure that the scaffolding *caused* a performance boost because both phases used that same structure.

Meng: Exactly; they’re showing us how to build better guardrails without needing massive new infrastructure just yet.

Tom: And their next step seems to be moving from just tracking use to actively measuring the quality of that engagement, which is what those marker codes were designed for.

Jane: So if you're designing an assignment involving AI, think less about letting students write and more about designing a process where they have to prove they’ve actually wrestled with the output.

Lu: That leads us into how we can actually build that kind of structured workflow, which is what we'll look at next.

The paper's improvements: Tom: So, moving past just describing what happened over those three years, the paper proposes concrete improvements for how we actually design these assignments to get better ethical outcomes from AI use.

Jane: They’re suggesting a more structured workflow where students have to write a complete draft on their own first, and then they use the AI only for refinement after checking it against a specific checklist.

Lu: This isn't just about using the tool differently; it’s about changing the entire process to force evaluation before generation happens.

Meng: I like that idea of forcing an independent draft because that immediately tells you what kind of argument they were trying to build before they started outsourcing the heavy lifting.

Tom: And they're emphasizing mandatory disclosure, meaning students have to state when and how much AI helped them with their work, which adds a layer of accountability.

Jane: Plus, there’s this part about verification—they say students can’t verify an AI suggestion before they use it, but they have to provide at least two credible references for every claim in the final essay.

Lu: That verification requirement is pretty important because it forces them to connect the ideas generated by the AI back to established knowledge, which is something that gets lost when you just take a quick AI answer.

Tom: It seems like they want us to treat the AI as a very powerful suggestion engine rather than an automatic answer generator that we just accept at face value.

Jane: The overall implication is that the pedagogical goal should be to advance critical thinking and responsible adaptation, not just increasing the final essay score.

Lu: They argue that engagement quality and outcome quality are two different goals, and scaffolding can help you get a student to engage critically even if it doesn't instantly improve their final grade.

Meng: That makes sense for us because when we deploy these models at scale, we need to know the user isn't just blindly following the most fluent output; they need that critical filter built in.

Tom: So, the big idea is that designing a process that demands verification and reflection is what actually teaches students the ethical side of using AI.

Jane: It really does. And for us listening, it suggests that if you’re teaching technical skills alongside ethics, building in those mandatory evaluation checkpoints is the way to go instead of just handing over the whole essay to a machine.

Conclusion: Tom: So, to wrap up "Scaffolding GenAI Use in Ethical Essay Writing: A Three-Year Experience Report in Introductory Computing," the main point is that designing a process that demands verification and reflection is what actually teaches students the ethical side of using AI.

Jane: It really does. They show us that policy and assessment design shape student use more than the tools themselves do, pushing them toward more responsible engagement when they have to actively evaluate suggestions.

Lu: The authors conclude that keeping that basic scaffold—the draft-before-AI step with its checklist, mandatory disclosure, and marked reflection—is a low-cost way to assess ethics within a technical course.

Meng: It’s about making the responsibility of using the AI explicit part of the assessment itself, which is how we see it impacting people who use these models in their daily work.

Lalam: I'm sure about this. The study confirms that when you give students a process that requires them to pause and check their work, they are much more likely to be critical partners with the AI instead of just letting it write for them.

Tom: It shows us that the value isn't in permitting AI use at all, but in designing structures that guide students to evaluate and adapt those suggestions rather than just passively accepting them.

Jane: That’s the central argument—that engagement quality and outcome quality are two different goals, so scaffolding can advance one without necessarily improving the other.

Lu: We've seen how specific workflows change student behavior, which is exactly the kind of data we need to build better training strategies for AI systems themselves.

Meng: If we can learn from this, it means our next generation of models shouldn't just focus on fluency; they need to be designed with built-in checkpoints that encourage human verification.

Lalam: I’m confident in the finding that modification and verification rates were high under the scaffolded conditions because that’s how responsible use looks in practice.

Tom: It was a solid report on how we can start engineering ethics into the learning process, rather than just hoping students behave correctly when they have access to a tool.

Jane: So, for anyone designing technical courses or even developing AI interfaces, the message is clear: give them a job that requires them to think critically about what the machine is suggesting before they let it finish.

Lu: This work gives us a blueprint for integrating tools into education in a way that prioritizes student agency over simple output generation.

Asma Shakil, Paul Denny, Jun Seo, Gauravkumar Hirani, Sean Longyu Ma, Claus Brabrand

University of Auckland · Karnavati University · IT University of Copenhagen

cs.CY

Submitted: 2026-11-05

Updated: 2026-11-05

DOI: 10.1145/3857222.3860166

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: Scaffolding GenAI Use in Ethical Essay Writing: A Three-Year Experience Report in Introductory Computing examines how students use Generative AI when writing ethical essays across three years of a…

Key concepts

Scaffolding
A structured instructional approach designed to support learning by providing temporary assistance. In this study, it involved a specific workflow—writing a draft independently first, then using an AI tool for refinement—complete with checklists and mandatory reflection steps.
Critical Engagement
The active process where students evaluate AI-generated suggestions instead of passively accepting them. This was measured by markers who coded whether students showed critical engagement (Yes/No) and genuine insight into the AI's output, which was found to be high under the scaffolded approach.
AI Adoption Trajectory
The changing patterns in how frequently and how much students used GenAI tools across different policies. Adoption rose from around two-thirds under voluntary use to over 90% under expected-use policies, showing that policy framing strongly influences student behavior.
Assessment Design
The structure of the essay assignments and grading criteria. The study concluded that assessment design dictates whether students use AI selectively or uncritically; designing tasks that require evaluation rather than just generation is key to effective pedagogical outcomes.

Terminology

Summary

Scaffolding GenAI Use in Ethical Essay Writing: A Three-Year Experience Report in Introductory Computing examines how students use Generative AI when writing ethical essays across three years of a first-year computer systems course and finds that pedagogical value depends on how assessment design guides students to evaluate and critically engage with AI-generated suggestions

Study Design and Context

The experience report examines GenAI use across three years of ethical essay assignments in a large first-year computer systems course The study analyzed 1,894 responses from six post-essay surveys conducted between 2024 and 2026, linking survey data with essay grades from 2025 and 2026 The research followed a repeated cross-sectional design across three successive cohorts of the same course to show how practices changed at the course level rather than for individual students Each semester included two short essay assignments on ethical issues in computing, which were worth 8% of the final grade and required students to interpret prompts, construct arguments, and identify stakeholders

Adoption Patterns and Tool Usage

The adoption of GenAI tools varied significantly depending on the policy framing Under voluntary use policies from 2024 to 2025, AI adoption remained steady at around two-thirds of students, while under the 2026 expected-use policy, it rose above 90% for both essays Tool choice diversified beyond ChatGPT in 2026, with Claude and Gemini capturing roughly one-tenth of users each Students reported mainly using AI for help with grammar and style, as brainstorming fell sharply from 47% in 2025 to 15% in 2026

Impact on Essay Quality

Grade-linked analyses showed that AI use did not independently predict essay quality once prior writing ability was controlled In both Phase 2 (unscaffolded) and Phase 3 (scaffolded), ANCOVA models showed that AI use had no independent effect on essay quality when prior writing ability was held constant Score changes across the four adoption trajectories were not significantly different in either phase, indicating that AI use alone does not explain essay quality

Critical Engagement Under Scaffolding

The most significant findings related to how students engaged with AI suggestions under the scaffolded Phase 3 workflow Students reported high levels of responsible AI engagement, with rejection or modification of suggestions reported by 81% in Essay 1 and 84% in Essay 2 Marker-coded evaluations confirmed this trend, showing that positive AI tone increased from 47% to 66%, while negative ratings fell from 9.1% to under 1% Markers coded that students showed critical engagement at rates of 77–81% and genuine insight at rates of 81–85% in Phase 3 submissions

Pedagogical Implications

The study concludes that AI use is shaped by policy and assessment design as much as by tool availability, suggesting that assessment design can shape whether students use AI selectively rather than uncritically The central pedagogical insight is that engagement quality and outcome quality are distinct goals, and scaffolding may advance the former without improving the latter The authors recommend keeping the core scaffold—the draft-before-AI step with its checklist, mandatory disclosure, and marked reflection—as a low-cost way to assess ethics within a technical course This approach helps students evaluate, adapt, and take responsibility for AI suggestions

Limitations

The experience report is limited as a repeated cross-sectional study that cannot track individual student practices over time The analysis relies largely on self-reported AI use, which is prone to recall and social desirability bias Furthermore, the Phase 3 redesign limits causal interpretation because both essays used the scaffolded workflow, mixing design effects with cohort effects The findings are therefore treated as formative evidence of how scaffolding shaped students’ AI practices rather than causal evidence that it improves essay performance

Conclusion

In summary, adoption held at about two-thirds under voluntary use and exceeded 90% under scaffolded, expected use policies Students mainly used AI for writing support, and tool use diversified beyond ChatGPT Neither AI use nor adoption trajectory independently explained essay quality in either phase The clearest Phase 3 findings were behavioral, with students reporting high levels of modification and verification, and markers saw widespread critical engagement and genuine insight The value of GenAI in writing-intensive computing assignments appears to lie less in permitting its use than in designing structures that help students evaluate, adapt, and take responsibility for AI suggestions The paper is available at arXiv:2611.08431v1 [cs. p. 0

Where the paper says it

This experience report addresses these gaps through a threeyear study (2024–2026) in a large first-year introductory computing arXiv:2611. 08431v1 [cs.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Responsible Use Scaffolding Implementation: The AI system can implement a structured workflow requiring students to write a complete draft independently, with a checklist at its top before using an AI tool for refinement.

  2. Critical Engagement Coding: The system can apply marker codes such as AI tone (Positive, Neutral, Negative) and critical engagement (Yes, No) to submissions to assess how students evaluate AI output rather than passively accepting it.

  3. Verification Focus: The system can monitor and prompt students for reference verification, as the study found that reference verification by 93% (E1) and 95% (E2) was a reported responsible practice under scaffolding.

Abstract

As society, ethics, and professional practice become increasingly integrated across the computer science curriculum, technical courses need practical ways to support ethical reasoning. Ethical essay assignments offer one approach, but little is known about how students use GenAI when writing such essays. This experience report examines GenAI use across three years of ethical essay assignments in a large first-year computer systems course. Across six post-essay surveys from 2024 to 2026, we analyzed 1,894 responses, linked survey data with essay grades in 2025 and 2026, and introduced a scaffolded responsible-use workflow in 2026. Under voluntary-use policies, AI adoption remained steady at around two-thirds of students, rising above 90% when use was scaffolded. Students reported mainly using AI for help with grammar and style, while tool use diversified beyond ChatGPT in 2026. Grade-linked analyses showed that AI use did not independently predict essay quality, while marker evaluations indicated high levels of critical engagement when use was scaffolded. Overall, GenAI's pedagogical value appears to depend on how assessment design guides students to evaluate and critically engage with AI-generated suggestions.

Related papers