Predicting Student Attrition in Competitive Programming: A Large-Scale Study Integrating Survey Insights and Global Behavioral Logs

summary

The gist

Predicting Student Attrition in Competitive Programming: A Large-Scale Study Integrating Survey Insights and Global Behavioral Logs is a dual-layer predictive framework that examines CP attrition by…

In short

The study created a dual-layer predictive framework combining large-scale Codeforces activity logs with localized student surveys to predict competitive programming attrition. It found that true attrition is preceded by an 83.71% drop in contest participation and increased struggle time, and identified a 'Skill-Application Paradox' where stopped students report higher confidence despite weaker independent practice.

Key concepts

Skill-Application Paradox
This counterintuitive finding shows that students who quit CP often overestimate their mathematical or data structure confidence compared to active peers. They may feel more capable theoretically but lack the consistent, independent practice habits needed to sustain engagement.
Intensity Ratio (IR)
This engineered feature measures a user's independent practice relative to contest participation. It captures how much a student practices on their own versus how often they participate in contests, providing a better signal for attrition than just raw activity counts.
Editorial Dependency Index (EDI)
This feature hypothesizes that students who stop upsolving frequently but without genuine independent effort are at risk. It operationalizes the idea that simply doing extra work on problems might not be enough if the underlying habit of independent practice is missing.
Early Warning System (EWS)
A proof-of-concept AI system deployed over active students that assigns continuous attrition risk scores based on survey data and behavioral trends. It helps identify specific high-risk individuals for proactive mentoring before they fully disengage.

Terminology used across episodes

This episode discusses

The paper

Predicting Student Attrition in Competitive Programming: A Large-Scale Study Integrating Survey Insights and Global Behavioral Logs · Read on arXiv

Department of Computer Science and Engineering, Metropolitan University, Sylhet, Bangladesh

Competitive programming (CP) offers computer science students an environment for developing algorithmic reasoning skills. However, sustained participation remains a challenge, as many students disengage after encountering skill plateaus or performance anxiety. While educational data mining (EDM) has studied dropout in MOOCs and academic courses, CP attrition remains understudied. This paper presents a dual-layer framework combining large-scale Codeforces activity logs (n=1,816) with a multi-institutional psychographic survey across 10 universities in Bangladesh (n=64). Analysis reveals that true attrition is preceded by an 83.71% reduction in contest participation and a 15.6% increase in struggle time. We identify a "Skill-Application Paradox": stopped students self-report higher mathematical confidence (3.88 vs. 3.41) and data structure understanding (3.57 vs. 3.09) than active peers, yet their independent practice and upsolving habits are significantly weaker (p < 0.001). Benchmark evaluations show that a Soft-Voting Ensemble achieves a 0.737 CV F1-score on behavioral logs, while Random Forest achieves 0.924 CV F1-score on psychographic data (an exploratory pilot). We deploy the survey-trained model as a proof-of-concept Early Warning System over 22 active students, identifying 4 at high risk. These findings show that behavioral and psychographic signals can support data-driven mentoring in computing programs.

DOI: 10.53799/vka5w161

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "Predicting Student Attrition in Competitive Programming".

Jane: Here's the gist of the paper. Predicting Student Attrition in Competitive Programming: A Large-Scale Study Integrating Survey Insights and Global Behavioral Logs is a dual-layer predictive framework that examines CP attrition by combining large-scale platform behavioral logs with localized psychographic survey data to support proactive, data-driven mentoring Pruning redundant reasoning tokens The paper presents a dual-layer predictive framework that examines CP attrition by combining large-scale platform behavioral logs with localized psychographic survey data The authors collected raw activity traces from 6,927 Codeforces profiles and refined these into a balanced modeling dataset of 1,816 users—908 persistently active and 908 exhibiting true attrition In parallel, they conducted a multi-institutional survey across 10 universities in Bangladesh (n = 96; n = 64 for predictive modeling after label refinement) Their behavioral analysis reveals that true attrition is preceded by an "83.71% reduction in contest participation and a 15.6% increase in problem-solving struggle time" Crucially, they identify a ”Skill-Application Paradox”: stopped students self-report higher mathematical confidence (3.88 vs.

Tom: 3.41) and data structure understanding (3.57 vs.

Jane: 3.09) than their active peers How the training signal is built The methodology involves engineering two novel domain-specific interaction features—the Intensity Ratio (IR) and Editorial Dependency Index (EDI)—that outperform raw activity counts in capturing attrition-relevant behavioral signals The authors engineered the Intensity Ratio (IR) to capture a user’s independent practice relative to contest participation, defined as IRi = Psolved,i − Pcontest,i / Ctotal,i + ϵ The Editorial Dependency Index (EDI) operationalizes the hypothesis that attrition-bound users upsolved frequently but without genuine independent effort Activity decay is captured through the Activity Trend Ratio (ATR), defined as AT Ri = P3 / m=1 Am,i P6 / m=4 Am,i + ϵ The model architecture evaluated nine classifier architectures including a custom Soft-Voting Ensemble which aggregates classifier probability outputs as Pˆ(y = 1x) = X M m=1 wm · Pm(y = 1x) All classifiers used class weight=’balanced’ to handle residual imbalance in the survey dataset Where the gains come from The behavioral analysis showed that true attrition users exhibited a higher burnout rate, with "45.01% had activity declining by more than 50% in the final three months" compared to active users The survey findings revealed that "Upsolving habit (χ 2 = 27.70, p < 0.001), peer circle density (χ 2 = 60.46, p < 0.001), and long break history (χ 2 = 73.00, p < 0.01) all differ significantly between active and stopped students" Feature importance analysis showed that "Upsolving habit (28.81%) and peer circle density (21.01%) together account for approximately 50% of predictive weight" The Soft-Voting Ensemble achieved the highest cross-validation F1-score of 0.737 on the global behavioral dataset Pruning redundant reasoning tokens The convergence across data layers indicates that disengagement is preceded by declining contest participation and increased problem-solving friction, which are described consistently from different vantage points The authors suggest that university programming clubs would benefit most from focusing on "upsolving discipline and peer network density" rather than just running more contests The deployment of the Early Warning System identified 4 students as High Risk whose profiles matched the SkillApplication Paradox signature Pruning redundant reasoning tokens The study acknowledges limitations, noting that the survey dataset is small and geographically limited to Bangladesh, which may not generalize to other academic or cultural contexts A critical limitation is retrospective recall bias regarding psychographic features collected from stopped students after they had already disengaged The Early Warning System deployment constitutes a proof-of-concept demonstration only, and no longitudinal follow-up data is currently available to verify whether the 4 flagged high-risk individuals subsequently experienced attrition Future work will focus on larger, longitudinal cohorts spanning multiple universities and academic years Pruning redundant reasoning tokens The paper concludes that the SkillApplication Paradox is validated through both feature importance rankings and statistically significant group-level tests The engineered Intensity Ratio ranked as the third most predictive Codeforces feature, validating its utility in capturing independent practice effort beyond raw activity counts The results are presented as an exploratory baseline rather than a production-ready deployment Future work will focus on larger, longitudinal cohorts spanning multiple universities and academic years The authors thank the faculty members and students across the ten participating universities in Bangladesh for their cooperation in completing the survey Pruning redundant reasoning tokens The single most informative sentence about the paper's subject and finding is that true attrition is preceded by an 83.71% reduction in contest participation and a 15.6% increase in problem-solving struggle time The authors present these results as an exploratory baseline rather than a production-ready deployment The paper presents a dual-layer predictive framework that examines CP attrition by combining large-scale platform behavioral logs with localized psychographic survey data to support proactive, data-driven mentoring The authors collected raw activity traces from 6,927 Codeforces profiles and refined these into a balanced modeling dataset of 1,816 users—908 persistently active and 908 exhibiting true attrition The authors conducted a multi-institutional survey across 10 universities in Bangladesh (n = 96; n = 64 for predictive modeling after label refinement) The authors identified that true attrition is preceded by an "83.71% reduction in contest participation and a 15.6% increase in problem-solving struggle time" The authors identified the "Skill-Application Paradox," where stopped students self-report higher mathematical confidence (3.88 vs.

Tom: 3.41) and data structure understanding (3.57 vs.

Title and authors: Tom: So we're looking at this paper today, "Predicting Student Attrition in Competitive Programming: A Large-Scale Study Integrating Survey Insights and Global Behavioral Logs." It sounds like they've put together a dual-layer system to figure out why students drop out of competitive programming.

Jane: Exactly. The core idea is they are mixing massive amounts of activity data from platforms like Codeforces with some local survey results from universities in Bangladesh to find patterns that predict when someone will quit CP.

Lu: What's interesting is how they defined the groups initially—they took six thousand nine hundred twenty-seven raw activity traces and ended up with a balanced modeling dataset of one thousand eight hundred sixteen users, split into nine hundred eight active people and another nine hundred eight who are truly at risk of attrition.

Meng: So, what's the real question they're trying to answer here? Is it just about activity levels, or is there something deeper going on with these students?

Tom: It’s definitely deeper because they found that true attrition isn't random. Their behavioral analysis shows that it’s preceded by an eighty-three point seven one percent reduction in contest participation and a fifteen point six percent increase in the time it takes to solve a problem.

Jane: That sounds like a real measurable signal, which is crucial because it moves this beyond just guessing based on how many contests someone entered. And then they introduce this concept called the Skill Application Paradox.

Lu: Right. The paradox is that when students stop, they actually report higher mathematical confidence—three point eight eight compared to three point four one for active peers—and better data structure understanding, which is what stops them from practicing independently in the first place.

Meng: So, they're saying the problem isn't just lack of skill; it’s a mismatch between what they *think* they know and how much independent practice they are actually doing. That makes sense for someone trying to study something self-directed like CP.

Tom: And to capture that mismatch, they engineered two new features called the Intensity Ratio and the Editorial Dependency Index, which seem better than just looking at raw activity counts alone.

Jane: Let me unpack those a bit for us. The Intensity Ratio seems to measure how much practice a user is doing compared to contest participation, defined as solved problems minus contest problems divided by total contests plus some little epsilon value.

Lu: And the Editorial Dependency Index operationalizes this idea that students who are about to quit might be upsolving frequently but without actual independent effort, which is why they engineered that feature.

Title and authors: Meng: From a production standpoint, I’m curious how these features actually perform when we try to run them against real traffic instead of just the curated dataset. What do those interaction features tell us about what breaks at scale?

Tom: The model architecture they tested included nine different classifiers, including a custom Soft Voting Ensemble that combines all the probabilities into one final prediction. And they used class weight equal balanced to handle any imbalance in their survey data.

Jane: The results on the global behavioral dataset showed that this Soft Voting Ensemble got a cross validation F1 score of zero point seven three seven, and a test recall of zero point seven six nine. That’s a solid number for identifying true attrition users.

Lu: Now, look at the psychographic survey data from those ten universities in Bangladesh; they found that upsolving habit and peer circle density both had significant differences between active and stopped students, with chi square values of zero point two seven seven and zero point sixty four respectively.

Meng: So, when you look at what's driving the prediction most strongly, what features actually matter? Is it the raw activity or something else?

Tom: Feature importance analysis showed that upsolving habit and peer circle density together account for about fifty percent of the predictive weight in their model. That really validates why they focused on those specific behavioral signals.

Jane: It seems like the combination of self-reported confidence, reduced contest participation, and weak independent practice habits are all pointing to the same thing: a student is starting to burn out or feel stuck.

Lu: The whole point of this paper is establishing this dual layer—the global behavioral telemetry and the localized psychographic survey data—to create a reproducible framework for identifying at-risk students.

Meng: It’s a proof of concept, though, right? They deployed an Early Warning System over twenty two active students to see if it worked practically before claiming it's ready for widespread deployment.

Tom: And they did, because the system identified four specific students as high risk whose profiles matched that Skill Application Paradox signature. That’s a tangible outcome for proactive mentoring.

Jane: So, what does this mean for the world of competitive programming education? It suggests that universities or programming clubs shouldn't just focus on running more contests; they should focus on building up the upsolving discipline and strengthening peer network density among students.

Lu: That aligns with their suggestion, and it’s validated by the fact that their Intensity Ratio was ranked as the third most predictive Codeforces feature, showing its utility beyond just counting submissions.

Title and authors: Meng: From my side, if we were to build this system for a real startup environment, I'd be worried about the cost of running those complex interaction features like IR and EDI on live data streams and ensuring they don't break when things get chaotic.

Tom: That’s a valid concern, Meng, but the framework itself is sound because it uses these specific metrics to target the intervention. The model suggests that early warning systems could actually lead to targeted mentoring recommendations for those high-risk students.

Jane: It’s about moving from reactive support after someone quits, to proactive conversation or a mentor referral when you see those behavioral patterns emerge in real time.

Lu: The limitation they mention is the survey itself being small and geographically limited to Bangladesh, which means we have to be careful generalizing these findings across different academic or cultural contexts.

Meng: And there’s also that retrospective recall bias with the psychographic features, because they collected those survey answers from students who had already disengaged. That's a real hurdle for any long-term study.

Tom: So, to wrap up on this paper, "Predicting Student Attrition in Competitive Programming: A Large-Scale Study Integrating Survey Insights and Global Behavioral Logs," the main implication is that we can use a dual layer of data—behavioral logs and survey insights—to build an early warning system for student disengagement.

Jane: It gives us a concrete way to say, "Hey, this student is showing signs of trouble based on their activity trends and how they talk about their skills."

Lu: The Skill Application Paradox is the most counterintuitive finding here—students feel more confident but are actually doing less independent practice than their active peers.

Meng: And the practical application is that we should focus mentoring efforts on things like upsolving discipline and peer network density, rather than just increasing contest frequency.

Tom: It’s a solid exploratory baseline that proves these specific behavioral signals have predictive power, even when combined with survey data from a smaller sample.

Jane: A great piece of work showing how large-scale platform data and localized human insights can combine to make sense of complex student behavior.

Lu: This framework provides a structure for future research, focusing on larger cohorts across different universities and academic years to test its robustness more thoroughly.

Meng: We’ll need those longitudinal studies to see if this Early Warning System actually works reliably when deployed in a high-pressure environment like university computing programs.

Tom: We'll keep an eye on that work, but for now, this paper shows us exactly what to look for when trying to spot someone about to drop out of CP.

The paper's summary: Tom: So this paper is essentially showing us how to build an early warning system for competitive programming students by merging two different kinds of data—massive activity logs from platforms like Codeforces and detailed survey results from universities in Bangladesh.

Jane: It’s about spotting that student who is really about to drop out before they actually quit, by looking at a combination of their on-platform behavior and what they tell you in a survey.

Tom: The big finding they highlight is that true dropping out isn't just random; it happens after an eighty-three point seven one percent drop in contest participation and fifteen point six percent more time spent struggling with problems.

Jane: And then they introduce this concept called the Skill Application Paradox, which is pretty interesting because it means that when students stop, they actually report feeling more confident about things like math and data structure understanding than their active peers do.

Tom: That’s the part where you have to look closer at why that happens; it suggests that stopped students aren't just bad at the material, but they are also not practicing independently as much.

Jane: Exactly; they might be upsolving frequently, but it isn't real independent practice, which is what their model tries to capture using these new features like the Intensity Ratio and the Editorial Dependency Index.

Tom: Those features are supposed to be better than just counting how many problems someone solved or how many contests they entered because they try to catch that specific pattern of behavior that signals burnout.

Jane: The results show that when you combine those behavioral trends with those survey insights—like upsolving habits and peer network density—the model gets really good at predicting who is at risk.

Tom: They used nine different types of machine learning classifiers, including a custom one called the Soft Voting Ensemble, and it actually got a cross validation F1 score of zero point seven three seven on the global data set.

Jane: It’s important to remember that they also have some limitations; they admit their survey is small and only covers ten universities in Bangladesh, so we need to be careful about how much those findings generalize elsewhere.

Tom: They also noted a limitation with the survey itself, which is that they collected those feelings about stopping students *after* they had already disengaged, so it’s a bit like looking backward at something.

Jane: But even with those caveats, they deployed an Early Warning System on twenty two active students and flagged four as high risk whose profiles matched that skill application paradox signature.

Tom: So what this means for us in the world is that we can start moving toward proactive mentoring in competitive programming programs instead of just waiting for students to quit.

Jane: It suggests that university clubs should focus more on building up the upsolving discipline and strengthening peer networks, rather than just running more contests every week.

Tom: They’re essentially saying this dual-layer approach—behavioral logs and survey data—is a reproducible way to identify at-risk students, even if it's only a proof of concept right now.

Jane: The next step is for researchers to take this framework and test it on much larger groups across different academic years and universities to see if those findings hold up in the real world.

Lalam: I’m confident in this direction; the model architecture they used, combining classifiers with those engineered features, provides a solid empirical baseline for this problem.

The paper's improvements: Tom: So we’re talking about how this study suggests we can actually build an improved system for predicting student attrition in competitive programming by focusing on specific things like upsolving habits and peer circles, instead of just looking at raw activity counts.

Jane: It means that the next version of this early warning system won't just flag a drop in contests; it will look for those behavioral signatures we talked about earlier, like low Intensity Ratio scores combined with high self-reported confidence.

Tom: The authors suggest that university programming clubs should shift their focus away from just running more contests and toward building up that upsolving discipline and strengthening the peer network density among students.

Jane: That’s a huge implication for mentoring; it suggests that the intervention should be about structured post-contest review sessions or mentor-buddy systems, tailored to those specific weaknesses.

Tom: And they also point out that because they engineered those new features like the Intensity Ratio and the Editorial Dependency Index, this approach is more robust because it captures signals raw activity counts just don't show.

Jane: That makes sense; it’s about capturing that gap between what a student *thinks* they know and how much independent practice they are actually doing, which is where the Skill Application Paradox comes in.

Tom: The authors also emphasize that this dual-layer empirical baseline—combining global data with localized survey data—is a reproducible framework, which is important because it shows the method can work across different contexts if you use these specific features.

Jane: But we have to remember the paper flagged a major limitation: they used a small sample size from just ten universities in Bangladesh, so we can't assume this exact model will work perfectly in every single academic setting out there.

Tom: Right, and they also admitted that because they collected those survey answers retrospectively, there’s a potential for recall bias regarding those psychographic features after the students had already decided to stop participating.

Jane: So what this study changes for someone who only listens to the show is that it gives us a blueprint—a structural way to think about using both big data and human insight together to predict student disengagement.

Tom: It moves the goal from just measuring activity levels to actually understanding the underlying psychological friction causing that attrition.

Jane: And looking ahead, the authors themselves say their next steps will involve testing this framework on much larger groups spanning multiple universities and several academic years to see if it stays reliable over time.

Tom: That’s a good point; proof of concept is one thing, but longitudinal data is what you need to prove that the system actually predicts future attrition, not just past behavior.

Jane: So, the paper gives us a framework for proactive mentoring based on behavioral signals and self-reported skills, even with those known limitations about sample size and timing.

Conclusion: Tom: So we’re wrapping up this look at "Predicting Student Attrition in Competitive Programming: A Large-Scale Study Integrating Survey Insights and Global Behavioral Logs" by Ruhan and his team.

Jane: Basically, the paper shows a dual-layer way to predict if a student will quit competitive programming by mixing activity logs with survey data from universities.

Tom: The main implication is that we can start using these specific behavioral signals—like the reduction in contest participation—as an early warning system for at-risk students.

Jane: It suggests that university clubs need to focus more on mentoring and building up peer networks rather than just increasing the contest frequency, which is a pretty practical change.

Tom: The study proves that engineering those interaction features, the Intensity Ratio and the Editorial Dependency Index, gives you better results than just looking at raw activity numbers alone.

Jane: And they validated that finding through their Soft Voting Ensemble model getting a zero point seven three seven cross validation F1 score on the global behavioral dataset.

Tom: That number is solid for identifying those true attrition users in the global data set, and it shows that combining different machine learning approaches can help you get a better prediction.

Jane: The main caveat is that since they used a small survey group from Bangladesh, we have to be cautious about how much these findings generalize to other academic or cultural contexts.

Tom: And they also noted the limitation with retrospective recall bias, meaning the survey answers are collected after the students have already stopped participating.

Jane: So what this study changes for us is that it gives us a reproducible framework for using both big data and human insight to spot at-risk students in computer science programs.

Tom: It’s an exploratory baseline, so we know it’s a starting point, but the four high-risk students they flagged based on the Skill Application Paradox signature are concrete evidence that the system has some predictive power.

Jane: That leads us into future work where they plan to test this framework on much larger groups spanning multiple universities and academic years to see if it holds up.

Tom: Exactly; we need those longitudinal studies to confirm that this Early Warning System actually works reliably in a real, long-term environment.

Jane: It's a great piece of work showing how localized human insights and global platform data can combine to make sense of complex student behavior patterns.

More episodes

← Home