Longitudinal Learner Participation Profiling in Online Learning: Heterogeneous Consensus Clustering under Unequal Observation Windows

arXiv:2610.17494 · cs.LG, cs.CY · Submitted 2026-10-12 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Longitudinal Learner Participation Profiling in Online Learning".

Jane: The gist: Course-grouped adjustment substantially reduced this problem:

Tom: First, who's behind it and why it matters.

Title and authors: Tom: We’ve seen how this paper tackles the issue of unequal observation windows, and now we’re going to talk about what they actually did in the "Longitudinal Learner Participation Profiling in Online Learning: Heterogeneous Consensus Clustering under Unequal Observation Windows" paper >

Jane: They started by setting up this study using longitudinal Moodle records from a public university in South Africa to see if their clustering approach could handle the variability in observation windows >

Lu: The core idea they developed is this consensus-clustering framework that accounts for unequal observation windows by first checking if participation measures are affected by the number of observed weeks and then adjusting them where necessary >

Meng: Adjusting them sounds complicated. Can you explain what "duration adjustment" means in plain terms for someone who isn't deep into the statistical stuff?

Jane: Okay, imagine you have a measure of participation, like how many times a learner logged in. If some courses are short and some are long, those raw numbers aren't comparable >

Tom: Right. So they estimate what that participation measure *should* look like based on the course length, and then they subtract that expected value from the actual observed value to get this duration-relative value >

Lu: That’s exactly it; subtracting the expected indicator values from the observed value to calculate a duration-relative value >

Meng: So, if you're looking at a long course and a short course, this adjustment helps make them comparable?

Jane: It helps make them comparable by reducing the influence of how many weeks were actually observed in that specific record >

Tom: And once they have these adjusted measures, they use those five retained indicators—the ones that stayed good after the adjustment—to run their clustering algorithms >

Lu: They used K-means, Gaussian mixtures, Ward agglomerative clustering, and K-medoids on both the standardized original measures and these new duration-relative indicators >

Meng: So it’s not just one method they are using; they are testing different tools to see which ones give them the best picture of participation patterns >

Jane: That’s right. They were comparing different representations, like standardized original indicators versus standardized duration-relative indicators, to see what worked better for clustering >

Tom: It shows they aren't just relying on one mathematical trick; they’re building a system that checks its own assumptions about the data structure >

The paper's summary: Tom: So we know the setup involved these complex adjustments and multiple clustering techniques, but what is the actual finding here in terms of what those profiles look like? >

Jane: They found evidence supporting a broad two-profile grouping, but they also detailed a more specific four-profile consensus that they felt was more detailed >

Lu: Those four profiles are Gradually declining, Early-concentrated, Cyclical variable, and Sustained active >

Meng: What do those names mean in the context of learner behavior? Are we talking about students who drop out fast or those who stay active forever?

Jane: It’s important to understand that these profiles describe patterns of participation within particular courses rather than just fixed learner types >

Tom: So, if a student falls into the "Gradually declining" profile, it means their participation trend is showing a consistent downward slope over time in the courses they take >

Lu: Yeah. It’s about that pattern of decline being observable across different course sequences rather than just being an isolated event >

Meng: That makes sense for production because if we can identify a pattern like this, we know when to intervene with a student in a certain context >

Jane: And they measured the mean assignment support at zero point seven three nine; that means the profiles they assigned had pretty good agreement across their different clustering results >

Tom: But that support isn't perfect either; eight point three five percent of assignments fell below a support of zero point five zero, with the greatest uncertainty in the Sustained active profile >

Lu: That uncertainty in Sustained active is something we need to keep an eye on because it suggests that pattern isn't as clearly defined by the data alone >

Meng: So, how does this change how we think about learner engagement at scale? Are these profiles just descriptive labels or can they actually drive action?

Jane: They are meant to be interpreted as patterns, so they give you a starting point for understanding what’s happening in the data, not necessarily a fixed risk label >

The paper's improvements: Tom: Let’s talk about the improvements they suggest. Beyond just creating these profiles, what did this work actually add to the field? >

Lu: The main improvement is that they developed this entire consensus-clustering framework which accounts for unequal observation windows, which addresses a problem that previous methods ignored >

Meng: So it’s not just a new profile; it’s a new way of doing the whole analysis because it tackles the measurement problem upfront >

Jane: They showed how to combine results from different clustering algorithms—K-means, Gaussian mixture, Ward agglomerative clustering, and K-medoids—and different representations >

Tom: And they compared those across standardized original indicators versus those duration-relative indicators to see which ones provided better information for the final profile assignment >

Lu: They showed that sequence clustering favored the broader two-profile grouping, but multichannel agreement showed strong consistency between the active and inactive results from different clustering methods >

Meng: That level of cross-checking sounds important for building a reliable system, especially when you're worried about scale and deployment issues >

Jane: It provides a more transparent and robust basis for profiling participation over time because they check the influence of observation-window length, compare results across clustering methods and forms of participation measures, and report assignment consistency >

Tom: So the takeaway here is that to get a reliable profile, you can't just trust one single metric or one single clustering method alone >

Conclusion: Jane: So to wrap up the whole discussion on this paper, they introduced and evaluated a consensus framework that combines several clustering methods and participation measures while successfully accounting for observation-window length issues >

Tom: The results supported both a broad two-profile grouping and the more detailed four-profile grouping: Gradually declining, Early-concentrated, Cyclical variable, and Sustained active >

Lu: These profiles should be understood as patterns that combine several aspects of participation rather than fixed learner types >

Meng: They are definitely tools for understanding how students interact with their learning environment over time in a specific course context >

Jane: By checking the influence of observation-window length, comparing results across clustering methods and forms of participation measures, and reporting assignment consistency, this framework provides a more transparent and robust basis for profiling participation over time in higher education >

Tom: Yeah. It’s about moving toward a method that is less sensitive to arbitrary data choices when you're trying to get reliable insights from LMS traces >

Lu: Indeed. The paper "Longitudinal Learner Participation Profiling in Online Learning: Heterogeneous Consensus Clustering under Unequal Observation Windows" gives us a better tool for handling the messy reality of longitudinal data >

Meng: It’s a solid piece of work that shows how to build something that is reliable enough to actually be used in a real production setting, even if it has some uncertainty with certain profiles >

David Otoo-Arthur, Terence L. van Zyl

University of the Witwatersrand · University of Johannesburg

cs.LG, cs.CY

Submitted: 2026-10-12

Updated: 2026-10-12

Comments: Accepted for presentation at SACAIR 2026 and publication in the conference proceedings

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

The gist: The gist: Course-grouped adjustment substantially reduced this problem: profile membership accounted for 38.18% of the variation in observation-window length before adjustment and 2.87% after

Key concepts

Observation Window Length: This refers to the different amounts of time recorded for each course or sequence. Because some courses are shorter than others, simply measuring participation rates doesn't allow for fair comparisons across all courses.
Consensus Clustering Framework: This method compares participation profiles derived from different summary methods and clustering techniques. It aims to find a stable grouping by checking how consistently each learner-course record is assigned across various analyses.
Duration-Relative Value: This adjustment technique estimates what the participation value *should* have been based on the expected observation window length. By subtracting this expected value from the actual observed value, it normalizes participation measures for differences in course duration.

Terminology

Clustering — 1× in this paper · explained in 2 episodes

Summary

The gist: Course-grouped adjustment substantially reduced this problem: profile membership accounted for 38.18% of the variation in observation-window length before adjustment and 2.87% after adjustment

Methodology and Data

The study used longitudinal Moodle records from a public South African university to investigate participation profiles The data covered February 2013–August 2019 and comprised 1,172,894 learner–course–week observations from 5,020 learners, 179 courses and 30,610 learner–course units The profiling cohort comprised 16,448 sequences, 4,599 learners, 152 courses and 587,276 weekly observations. Eight indicators were calculated to describe participation frequency and how participation changed over time.

Addressing Observation Window Length

A primary methodological finding was that converting participation measures into rates or shares was not enough to make records from shorter and longer courses comparable Before adjustment, profile membership accounted for 38.18% of the variation in observation-window length; after adjustment, this fell to 2.87%. This reduction was achieved by estimating expected indicator values using a spline–ridge model and calculating the duration-relative value by subtracting the expected value from the observed value. Pearson and Spearman correlations, together with variance inflation factors, were used to identify five indicators that provided similar information after this adjustment.

Consensus Clustering Framework

The framework developed accounts for unequal observation windows by first checking whether participation measures are affected by the number of observed weeks and adjusting them where necessary. It then compares profiles across different summaries of participation and clustering methods and assesses how consistently each learner–course record is assigned. The study evaluated eight indicators in four forms for clustering, including standardised original indicators, standardised duration-relative indicators, three principal components from the original indicators, rescaled to have equal variance, and three equivalent components from the duration-relative indicators.

Learner Participation Profiles

The analysis identified a broad two-profile grouping and a more detailed four-profile consensus: Gradually declining, Early-concentrated, Cyclical variable and Sustained active. The four profiles were Gradually declining (n = 4, 873), Early-concentrated (n = 5, 191), Cyclical variable (n = 3, 594) and Sustained active (n = 2, 790). Mean assignment support was measured at a mean of 0.739. The profiles were interpreted as patterns of participation within particular courses rather than as fixed learner types.

Robustness and Interpretation

The study assessed the stability of the four-profile solution by removing one algorithm at a time and comparing results across different indicator forms. Sequence clustering favoured the broader two-profile grouping, while multichannel agreement showed strong consistency between active–inactive and multichannel clustering results. The profiles should therefore be understood as patterns that combine several aspects of participation that are not fully reflected in week-by-week patterns alone. The framework provides a more transparent and robust basis for profiling participation over time by checking the influence of observation-window length, comparing results across clustering methods and forms of participation measures, and reporting assignment consistency.

Limitations

The study did not capture offline participation and included only records with at least two active weeks. Duration adjustment reduced associations with observation-window length but did not account for course design, discipline, assessment timing or access conditions. Agreement with sequence analyses was modest, indicating that week-by-week sequences reproduced only part of the detailed profiles. Future research should test the framework across institutions, account for learners appearing in multiple courses and course differences and examine links with achievement and course design without treating the profiles as fixed risk labels.

Conclusion

This study introduced and evaluated a consensus framework that combines several clustering methods and forms of participation measures while accounting for observation-window length. The results supported a broad two-profile grouping and a more detailed four-profile grouping: Gradually declining, Early-concentrated, Cyclical variable and Sustained active. The profiles should therefore be understood as patterns that combine several aspects of participation, rather than as fixed learner types. By checking the influence of observation-window length, comparing results across clustering methods and forms of participation measures, and reporting assignment consistency, the framework provides a more transparent and robust basis for profiling participation over time in higher education.

--- Page 1 ---

Longitudinal Learner Participation Profiling in Online Learning: Heterogeneous Consensus Clustering under Unequal Observation Windows David Otoo-Arthur1[0000−0003−2057−088X] and Terence L. van Zyl2[0001−4281−63] 1 University of the Witwatersrand, Johannesburg, South Africa david.otoo-arthur1@students.wits.ac.za 2 University of Johannesburg, Johannesburg, South Africa tvanzyl@uj.ac.za Abstract Learning management system traces are clustered to identify participation profiles, but unequal observation windows may cause profiles to reflect duration rather than participation alone Using 16,448 complete learner–course sequences from 4,599 learners across 152 courses at a South African public university we evaluated eight participation measures All remained related to observation-window length even when expressed as rates or shares Course-grouped adjustment reduced these relationships Five retained measures were evaluated using K-means, Gaussian mixtures, Ward agglomerative clustering and K-medoids with standardised measures and principal-component summaries Evidence supported a broad two-profile grouping and a more detailed adjusted four-profile consensus: Gradually declining, Early-concentrated, Cyclical variable and Sustained active Profile membership accounted for 2.87% of the variation in duration in the primary consensus versus 38.18% using original measures Mean assignment support support was 0.739; 8.35% of assignments fell below 0.50, with greatest uncertainty in Sustained active Two week-by-week sequence analyses agreed strongly with each other but only modestly with the four-profile consensus indicating that the profiles combined multiple aspects of participation rather than distinct sequence patterns Profiling should therefore assess duration effects, compare clustering methods and participation summaries and report assignment consistency Keywords: Learning analytics · Behavioural participation · Consensus clustering · Longitudinal profiling · Duration adjustment · Assignment uncertainty 1 Introduction Learning management systems (LMSs) are widely used to support teaching, assessment and course administration in higher education [32] As learners access resources, submit assessments, participate in forums and use other platform tools they generate time-stamped traces of their interactions with the arXiv:2610.17494v1 [cs.

Improvements for AI systems

The AI system can generate longitudinal participation profiles by clustering learner-course sequences using a consensus framework that accounts for unequal observation windows, thereby reducing reliance on any single grouping method or summary measure.
The improved system can perform duration adjustment by calculating the duration-relative value of participation indicators, which is defined as subtracting the value expected for its observation-window length from its observed value, and then using these adjusted measures for primary profiling because they are less affected by course duration.
The AI system can produce detailed, context-specific profiles by combining results from different clustering algorithms (K-means, Gaussian mixture, Ward agglomerative clustering, K-medoids) and different representations (standardised original indicators vs. standardised duration-relative indicators).
The system can generate four interpretable participation profiles: Gradually declining, Early-concentrated, Cyclical variable, and Sustained active. These profiles describe patterns of participation within particular courses rather than fixed learner types, as they are interpreted from high-support cores identified through assignment consistency.
The AI system can assess the reliability of its profiling by reporting assignment support, which is defined as the mean proportion of eight clustering results that agreed with the final profile assigned to each record, allowing users to distinguish between well-supported assignments and less certain ones, particularly for the Sustained active profile.
The system can perform sequence-based sensitivity analysis by comparing participation patterns at discrete points along the course progress, using measures like Hamming distance on activity states or Euclidean distance on multichannel representations to test whether profiles are reproduced from temporal shape.