Predictive Modeling for High Impact Active Learning Classrooms

arXiv:2603.14335 · physics.ed-ph · Submitted 2026-03-15 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.

Kai: We're looking at the paper: "Predictive Modeling for High Impact Active Learning Classrooms".

Mira: Here's the gist of the paper. Predictive Modeling for High Impact Active Learning Classrooms: This study uses a predictive model to map classroom time spent on different activities to student conceptual learning and identifies specific combinations of active learning strategies that consistently demonstrate exceptional student learning gains in undergraduate physics courses.

Kai: Predictive Model Development The researchers created a predictive model using multiple linear regression to map the fraction of twominute class intervals in which specific COPUS codes were present to the concept inventory effect size in that class.

Mira: The model was trained on 57 classes from extant data sources, and then used to predict effect sizes for 12 new physics and astronomy courses, yielding a mean absolute error of 0.16 after removing unreliable predictions.

Kai: To handle the complex relationships between classroom activities and student learning outcomes, the model included features such as Lec squared, Lec cubed, and all second and third order cross-terms like W G multiplied by CG.

Paper summary: Kai: So, looking at "Predictive Modeling for High Impact Active Learning Classrooms" by Ross, Sundstrom, and Holmes, the core message is that there’s a measurable way to predict which classroom activity mixes lead to the most significant student learning improvements in physics and astronomy courses.

Mira: The authors use a predictive model trained on over ten thousand students and twenty-four institutions to map time spent on lecture, group worksheets, clicker questions, and student questions to concept inventory effect sizes <ref:2603.14335#pg3>.

Lev: What’s important is that they found that the combination of spending twenty to forty percent of class time on group clicker questions and ten to twenty percent on group worksheets, along with having two or more student questions per hour, consistently predicts high learning outcomes <ref:2603.14335#pg1>.

Kai: So, the implication is that instructors don't just need to guess which active strategies are good; they can use this model to predict a specific strategy mix before they even start teaching the course <ref:2603.14335#pg2>.

Mira: It suggests a practical way for instructors to get actionable data, moving away from just anecdotal evidence about what works in the classroom <ref:2603.14335#pg1>.

Lev: From my side, this modeling tells us where the real effort should go; if you want maximum impact, you focus on that specific range of group work and questioning time rather than trying to maximize every single activity at once <ref:2603.14335#pg1>.

Kai: It really shows that even with a lot of variables in a science class, the core predictive power comes down to those four measurable activities <ref:2603.14335#pg1>.

Mira: So, this paper gives us a framework for what high impact active learning actually looks like in practice, based on empirical data from a large sample size <ref:2603.14335#pg3>.

Conclusion: Kai: So, this paper is about using a predictive model to figure out which classroom activities actually lead to big learning gains in physics and astronomy classes.

Mira: It’s titled "Predictive Modeling for High Impact Active Learning Classrooms," and the authors are Ross, Sundstrom, and Holmes.

Kai: Basically, they used data from fifty-seven existing classes to build this model that predicts concept inventory effect sizes based on how much time a class spends on different things like lectures or group worksheets.

Mira: The main finding is that you don't need to test every possible activity; there’s a specific mix—a certain percentage of time on group clicker questions, some time on worksheets, and a certain number of student questions per hour—that consistently shows exceptional learning gains.

Kai: It’s not just about doing *some* active learning; it’s about finding this precise combination that works better than just having a long lecture.

Mira: That’s the implication for instructors that they can use this model to predict which strategies are likely to work best before they even start teaching the course.

Kai: It suggests we can move past just guessing what works and get a data-driven idea of how to structure those active learning sessions.

Mira: But Ross and his team also pointed out some limitations, like the model only looks at these four specific activity types and doesn't account for everything happening in a real classroom.

Kai: They also mentioned that the data they used is quite varied across different class sizes and student groups, so applying this prediction everywhere might need some careful checking.

Mira: So, while this gives us a strong starting point for active learning design, we still have to consider if these four activities are truly the only things that matter for deep conceptual understanding.

Olive Ross, Meagan Sundstrom, N.G. Holmes

Cornell University · Drexel University

physics.ed-ph

Submitted: 2026-03-15

Updated: 2026-11-06

Comments: 20 pages, 8 figures, 2 tables

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 73/100

The gist: Predictive Modeling for High Impact Active Learning Classrooms: This study uses a predictive model to map classroom time spent on different activities to student conceptual learning and identifies…

Key concepts

Predictive Model Development
The study used multiple linear regression to create a mathematical model. It mapped how much time students spent on different class activities (like lectures or group work) related to the actual improvement in their conceptual understanding, using historical data from 57 classes.
COPUS Codes
These are codes used to categorize specific student and instructor behaviors observed during class sessions. The researchers focused on four key codes: Lec (lecture), WG (working on worksheets), CG (clicker questions in groups), and SQ (student asking a question).
Effect Size (Cohen's d)
This is a statistical measure used to quantify the magnitude of student learning gain. It shows how much better students performed after learning compared to before the test. A larger effect size indicates a stronger impact from the instructional method.
Optimal Strategy Identification
The model predicted an ideal teaching mix: 20-40% time on group clicker questions, 10-20% on group worksheets, and at least two student questions per hour. This combination showed the highest likelihood of achieving significant student learning gains.

Terminology

Summary

Predictive Modeling for High Impact Active Learning Classrooms: This study uses a predictive model to map classroom time spent on different activities to student conceptual learning and identifies specific combinations of active learning strategies that consistently demonstrate exceptional student learning gains in undergraduate physics courses. p. 0

Where the paper says it

edu; Abstract Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning classrooms; however, the term “active learning” lacks definition and little research has examined which types and combinations of active learning strategies are most effective. In this study, we use a dataset representing over 10,000 students and 24 institutions to create a predictive model that maps classroom time spent on different activities to student conceptual learning.

Predictive Model Development

The researchers created a predictive model using multiple linear regression to map the fraction of twominute class intervals in which specific COPUS codes were present to the concept inventory effect size in that class. The model was trained on 57 classes from extant data sources, and then used to predict effect sizes for 12 new physics and astronomy courses, yielding a mean absolute error of 0. p. 2

Where the paper says it

We first trained the model on 57 classes from extant data sources [11, 26, 28]. In order to test how accurately the model is able to make predictions for physics and astronomy classes outside of the training data, we collected COPUS and concept inventory data from 12 new physics and astronomy courses.

16 after removing unreliable predictions. p. 2
Where the paper says it

The model was used to make effect size predictions for these classes based on their COPUS profiles and these predictions were compared to the measured effect sizes as shown in Fig. After removing unreliable predictions, the mean absolute error was 0.

To handle the complex relationships between classroom activities and student learning outcomes, the model included features such as Lec squared, Lec cubed, and all second and third order cross-terms like W G multiplied by CG. p. 9
Where the paper says it

, Lec2 , Lec3 , Lec4 ), and all second and third order cross-terms (e. , W G ∗ CG, W G2 ∗ CG, and W G ∗ CG ∗ SQ), as features in the model.

Key Predictive Variables

The study found that four variables—classroom time spent on lecture, group worksheets, clicker questions, and student questions—are sufficient to reliably predict student learning as measured by concept inventory scores. One type of class consistently demonstrates exceptional student learning gains (effect sizes greater than 2): those that spend 10-20% of class time on group worksheets, 20-40% of class time on group clicker questions, and average two or more student questions per hour of class time. p. 0

Where the paper says it

We find that four variables – classroom time spent on lecture, group worksheets, clicker questions, and student questions – are sufficient to reliably predict student learning, as measured by concept inventory scores. We identify one type of class that consistently demonstrates exceptional student learning gains (effect sizes greater than 2): those that spend 10-20% of class time on group worksheets, 20-40% of class time on group clicker questions, and average two or more student questions per hour of class time.

Furthermore, classes which do not utilize group worksheets consistently have learning outcomes comparable to fully lecture classes. p. 0
Where the paper says it

We identify one type of class that consistently demonstrates exceptional student learning gains (effect sizes greater than 2): those that spend 10-20% of class time on group worksheets, 20-40% of class time on group clicker questions, and average two or more student questions per hour of class time. We also find that classes which do not utilize group worksheets consistently have learning outcomes comparable to fully lecture classes.

Optimal Strategy Identification

The model predicts one “optimal” combination of active learning strategies that should be tested in controlled studies: classes that spend 20-40% of class time on group clicker questions, spend 10-20% of class time on group worksheets, and have 2 or more student questions per hour consistently have high learning outcomes. The results also indicate that less class time spent on lecture is not necessarily better for student learning. p. 4

Where the paper says it

, up to about 60% of class time) is spent lecturing. Lastly, the results indicate that less class time spent on lecture is not necessarily better for student learning, as suggested by some prior work [e.

Model Limitations and Future Work

Limitations of the study include the fact that an undergraduate science class has many more variables than are captured in the model, focusing solely on conceptual learning. The researchers also did not account for activities occurring outside of regularly scheduled lecture time or for in-class variables not captured by the four COPUS codes. p. 5

Where the paper says it

Future work should examine the differential impact of active learning strategies on other student outcomes. We also do not account for activities that occur outside of regularly scheduled lecture time (such as homework, quizzes, projects, laboratory sections, discussion sections, or time spent studying) or for in-class variables not captured by the four COPUS codes (CG, SQ, Lec, and WG) included in this study.

Future work should investigate whether the presence of these activities directly impacts student learning or if they serve as proxies for other instructional or interpersonal variables. p. 5
Where the paper says it

Notably, this work focuses on the statistical accuracy of model predictions and not the theoretical underpinnings or causality of the relationships represented by the model. Future work should evaluate whether the presence of these activities directly impacts student learning, or if they serve as proxies for other instructional or interpersonal variables.

The sparsity of data in certain regions of this parameter space severely limits the ability to make reliable predictions for classes that spend more than 50% of class time on group worksheets. p. 5
Where the paper says it

Even within the four variables considered in this work, the sparsity of our data in certain regions of this parameter space severely limits our ability to make reliable predictions. While one of the highest effect sizes represented in our dataset was from a class that spent 100% of class time on group worksheets, we were unable to make reliable predictions for classes that spent more than 50% of class time on group worksheets due to the sparsity of the data in that range (see Fig.

Quantifying Learning Metrics

Student learning is quantified using Cohen’s d, a standardized mean difference between pre-test and post-test scores on concept inventories. This metric provides a dimensionless measure of student learning in each course, allowing for the comparison of scores across different disciplines and concept inventories. p. 7

Where the paper says it

This metric provides a dimensionless measure of student learning in each course, allowing us to combine student scores across a range of disciplines, courses, student populations, and concept inventories. The ability to compare scores across concept inventories is particularly important in this work, with 28 different concept inventories represented across our dataset.

Effect size is strongly correlated with gain, which aids in interpretation for readers familiar with gain as a summary statistic. p. 7
Where the paper says it

5, effect size is strongly correlated with gain. This may aid in interpretation for readers with more familiarity with gain as a summary statistic.

The relationship between classroom activities and student learning outcomes is complex, showing that some activities may have simple linear relationships while others may have more complicated relationships. p. 8
Where the paper says it

The relationship between classroom activities and student learning outcomes is complex. Some activities may have simple linear relationships with learning (e.

Data Sources and Methodology

The dataset combines data from three extant studies spanning 24 institutions, 3 scientific disciplines, and class sizes ranging from 11 to 576 students with new data collected by the authors. Classroom activities are characterized using the Classroom Observation Protocol for Undergraduate STEM (COPUS), which consists of 25 different codes marking student and instructor behaviors in two-minute segments. p. 1

Where the paper says it

We characterize classroom activities using the Classroom Observation Protocol for Undergraduate STEM (COPUS) [24]. The COPUS is a structured observation protocol that consists of 25 different codes, or student and instructor behaviors, and observers mark which codes occur in two-minute segments of class time.

The COPUS codes included in this analysis are Lec (instructor lecturing), WG (students working on worksheets in groups), CG (students working on clicker questions in groups), and SQ (student asking a question). p. 6
Where the paper says it

Multiple COPUS codes can be applied in each time interval; therefore, the sum of all COPUS code quantities is more than one. We include four COPUS codes in this study: Lec (instructor lecturing), WG (students working on worksheets in groups), CG (students working on clicker questions in groups), and SQ (student asking a question).

Model Uncertainty Assessment

To quantify how sensitive the model predictions are to sample bias, bootstrap sampling was performed by randomly selecting 1,000 different subsets of 85% of the data and training the model on each subset. The final prediction is taken as the mean of that distribution, and the model uncertainty is defined as the standard deviation of that distribution. p. 2

Where the paper says it

We apply bootstrap sampling to create a suite of functions rather than a single function, such that the model outputs a distribution of predicted effect sizes rather than a single predicted effect size. The final model prediction is then taken as the mean of this distribution and the prediction uncertainty (due to sample bias) is taken as the standard deviation of the distribution.

Predictions with uncertainty above 0. p. 4
Where the paper says it

Because student questions tend not to take up a whole two-minute COPUS interval, the right y-axis translates from percent of class time spent on SQ to average number of student questions per hour of class. Gray hatched regions are predictions that have high prediction uncertainty (bootstrap error >0.

3 (≈ 10% of the full range of effect sizes) are considered unreliable. p. 2
Where the paper says it

Second, any prediction with uncertainty above 0. 3 (≈ 10% of the full range of effect sizes present in the training set) is considered unreliable.

The mean absolute error for out-of-sample predictions, after removing unreliable ones, was 0. p. 2
Where the paper says it

The model was used to make effect size predictions for these classes based on their COPUS profiles and these predictions were compared to the measured effect sizes as shown in Fig. After removing unreliable predictions, the mean absolute error was 0.

18.

Data Characterization

The dataset represents more than 10,000 students and 40 unique instructors across 24 American and Canadian universities. The data sources include extant studies from works [11], [26], and [28], supplemented by new data collected from 12 introductory physics and astronomy class sections at a private, R1 university in the United States. p. 6

Where the paper says it

The dataset from existing studies represents 57 class sections, more than 8,000 undergraduate science students, and 24 American and Canadian Universities. To supplement these data, we also collected COPUS and concept inventory data from 12 introductory physics and astronomy class sections containing more than 1,900 undergraduate students at a private, R1 university in the United States.

Class sizes are represented across three categories: Less than 50, 50-150, and More than 150 students. p. 1
Where the paper says it

While this growing body of work has importantly started to disentangle the relative impacts of different active learning strategies, the impacts of different combinations of these strategies has yet to be rigorously analyzed in a way that produces testable predictions about student learning outcomes for future classes. In this study, we combine data from three extant studies [11, 26, 28]–spanning 24 institutions, 3 scientific disciplines, and class sizes ranging from 11 to 576 students– with new data collected by the authors to identify combinations of instructional strategies that coincide with the largest student learning gains.

The data is highly varied across class size, student demographics, and instructor experience. p. 5
Where the paper says it

Future work should evaluate whether the presence of these activities directly impacts student learning, or if they serve as proxies for other instructional or interpersonal variables. Finally, our data are highly varied across class size, student demographics, student academic level and preparedness, and instructor experience.

Data Availability

All data and source code are available at https://doi. p. 7

Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

org/10.5281/zenodo. p. 7
Where the paper says it

Source code is available at https: //doi. 5281/zenodo.

19598369. The research has been supported by the U. p. 0
Where the paper says it

edu; ngholmes@cornell. edu; Abstract Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning classrooms; however, the term “active learning” lacks definition and little research has examined which types and combinations of active learning strategies are most effective.

S. National Science Foundation Graduate Research Fellowship (DGE – 2139899). p. 0
Where the paper says it

edu; ngholmes@cornell. edu; Abstract Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning classrooms; however, the term “active learning” lacks definition and little research has examined which types and combinations of active learning strategies are most effective.

The authors declare no competing interests. p. 0
Where the paper says it

E-mail(s): ogr8@cornell. edu; Contributing authors: ms5629@drexel.

The author contributions are detailed, with O. p. 0
Where the paper says it

3Department of Physics, Cornell University, 410 Thurston Ave, Ithaca, 14853, New York, United States. *Corresponding author(s).

R. conceptualizing the study and performing the formal analysis and investigations. p. 5
Where the paper says it

Future work should seek to disentangle these conflicting claims. Several limitations of this study also prompt future investigations.

References

[1] Aiken JM, De Bin R, Lewandowski HJ, et al (2021) Framework for evaluating statistical models in physics education research. Physical Review Physics Education Research 17(2):020104. p. 2

Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

https://doi.org/10.1103/PhysRevPhysEducRes.17.

[5] Bojinova E, Oigara J (2013) Teaching and learning with clickers in higher education. p. 2

Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

International Journal of Teaching and Learning in Higher Education 25(2):154–165. p. 2
Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

[7] Chi MTH (2009) Active-constructive-interactive: A conceptual framework for differentiating learning activities. p. 1

Where the paper says it

Another study [28] related the extent to which different active learning activities were used during biology classes to student learning outcomes and found that spending more time on group worksheets was positively correlated with student learning. Other research has applied the ICAP (Interactive, Constructive, Active, and Passive) framework [7] to distinguish between different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8].

p. 1
Where the paper says it

Another study [28] related the extent to which different active learning activities were used during biology classes to student learning outcomes and found that spending more time on group worksheets was positively correlated with student learning. Other research has applied the ICAP (Interactive, Constructive, Active, and Passive) framework [7] to distinguish between different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8].

p. 1
Where the paper says it

Another study [28] related the extent to which different active learning activities were used during biology classes to student learning outcomes and found that spending more time on group worksheets was positively correlated with student learning. Other research has applied the ICAP (Interactive, Constructive, Active, and Passive) framework [7] to distinguish between different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8].

Topics in Cognitive Science 1(1):73–105. p. 1
Where the paper says it

Another study [28] related the extent to which different active learning activities were used during biology classes to student learning outcomes and found that spending more time on group worksheets was positively correlated with student learning. Other research has applied the ICAP (Interactive, Constructive, Active, and Passive) framework [7] to distinguish between different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8].

p. 1
Where the paper says it

Another study [28] related the extent to which different active learning activities were used during biology classes to student learning outcomes and found that spending more time on group worksheets was positively correlated with student learning. Other research has applied the ICAP (Interactive, Constructive, Active, and Passive) framework [7] to distinguish between different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8].

p. 1
Where the paper says it

Another study [28] related the extent to which different active learning activities were used during biology classes to student learning outcomes and found that spending more time on group worksheets was positively correlated with student learning. Other research has applied the ICAP (Interactive, Constructive, Active, and Passive) framework [7] to distinguish between different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8].

p. 1
Where the paper says it

Another study [28] related the extent to which different active learning activities were used during biology classes to student learning outcomes and found that spending more time on group worksheets was positively correlated with student learning. Other research has applied the ICAP (Interactive, Constructive, Active, and Passive) framework [7] to distinguish between different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8].

https: //doi. p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

org/10.1111/j.1756-8765.2008.01005.x

[9] Chin C, Osborne J (2008) Students’ questions: A potential resource for teaching and learning science. Studies in Science Education 44(1):1–39. p. 6

Where the paper says it

be selection bias effects both for instructors who volunteered their classes to participate in these studies (e. , instructors who are discipline-based education researchers themselves may be more likely to volunteer and may exhibit larger student learning gains than other instructors [2]) and for students who completed the pre-post surveys (e.

p. 6
Where the paper says it

be selection bias effects both for instructors who volunteered their classes to participate in these studies (e. , instructors who are discipline-based education researchers themselves may be more likely to volunteer and may exhibit larger student learning gains than other instructors [2]) and for students who completed the pre-post surveys (e.

https://doi.org/ 10.1080/03057260701828101

[26] Sundstrom M, Gambrell J, Green C, et al (2026) Relative benefits of different active learning methods to conceptual physics learning. p. 1

Where the paper says it

Other research has applied the ICAP (Interactive, Constructive, Active, and Passive) framework [7] to distinguish between different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8]. While this growing body of work has importantly started to disentangle the relative impacts of different active learning strategies, the impacts of different combinations of these strategies has yet to be rigorously analyzed in a way that produces testable predictions about student learning outcomes for future classes.

Nature Physics pp 1–6. p. 7
Where the paper says it

4 shows the relationships between concept inventory mean pre-test scores, mean post-test scores, and effect sizes present in the data. The nature of this work necessitates the use of effect size as a summary statistic, however, as demonstrated in Fig.

p. 7
Where the paper says it

4 shows the relationships between concept inventory mean pre-test scores, mean post-test scores, and effect sizes present in the data. The nature of this work necessitates the use of effect size as a summary statistic, however, as demonstrated in Fig.

p. 7
Where the paper says it

4 shows the relationships between concept inventory mean pre-test scores, mean post-test scores, and effect sizes present in the data. The nature of this work necessitates the use of effect size as a summary statistic, however, as demonstrated in Fig.

https://doi.org/10.1038/s41567-026-03307-2

[28] Weir LK, Barker MK, McDonnell LM, et al (2019) Small changes, big gains: A curriculum-wide study of teaching practices and student learning in undergraduate biology. PLOS ONE 14(8):e0220900. p. 0

Where the paper says it

We find that four variables – classroom time spent on lecture, group worksheets, clicker questions, and student questions – are sufficient to reliably predict student learning, as measured by concept inventory scores. We identify one type of class that consistently demonstrates exceptional student learning gains (effect sizes greater than 2): those that spend 10-20% of class time on group worksheets, 20-40% of class time on group clicker questions, and average two or more student questions per hour of class time.

p. 0
Where the paper says it

We find that four variables – classroom time spent on lecture, group worksheets, clicker questions, and student questions – are sufficient to reliably predict student learning, as measured by concept inventory scores. We identify one type of class that consistently demonstrates exceptional student learning gains (effect sizes greater than 2): those that spend 10-20% of class time on group worksheets, 20-40% of class time on group clicker questions, and average two or more student questions per hour of class time.

https://doi.org/10.1371/journal.pone

[3] Bazett T, Clough CL (2021) Course coordination as an avenue to departmental culture change. PRIMUS 31(3-5):467–482. https://doi.org/ 10.1080/10511970

[4] Beichner RJ, Saul JM, Abbott DS, et al (2007) The student-centered activities for large enrollment undergraduate programs (scale-up) project. Reviews in Physics Education Research 1(1):1–42. p. 2

Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

https://doi.org/10.1119/RevPERv1.

[6] Burkholder E, Walsh C, Holmes NG (2020) Examination of quantitative methods for analyzing data from concept inventories. p. 7

Where the paper says it

The ability to compare scores across concept inventories is particularly important in this work, with 28 different concept inventories represented across our dataset. 4 shows the relationships between concept inventory mean pre-test scores, mean post-test scores, and effect sizes present in the data.

Physical Review Physics Education Research 16(1):010141. p. 2
Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

p. 2
Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

p. 2
Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

https://doi.org/ 10.1103/PhysRevPhysEducRes.

[8] Chi MTH, Wylie R (2014) The icap framework: Linking cognitive engagement to active learning outcomes. p. 1

Where the paper says it

Another study [28] related the extent to which different active learning activities were used during biology classes to student learning outcomes and found that spending more time on group worksheets was positively correlated with student learning. Other research has applied the ICAP (Interactive, Constructive, Active, and Passive) framework [7] to distinguish between different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8].

Educational Psychologist 49(4):219–243. https://doi.org/ 10.1080/00461520.2014.965823

[17] Nissen J, Donatello R, Van Dusen B (2019) Missing data and bias in physics education research: A case for using multiple imputation. Physical Review Physics Education Research 15(2):020106. p. 2

Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

https://doi.org/ 10.1103/PhysRevPhysEducRes.

[24] Smith MK, Jones FHM, Gilbert SL, et al (2013) The classroom observation protocol for undergraduate stem (copus): A new instrument to characterize university stem classroom practices. p. 1

Where the paper says it

To do so, we analyze two data sources: classroom activities and concept inventory scores. We characterize classroom activities using the Classroom Observation Protocol for Undergraduate STEM (COPUS) [24].

p. 1
Where the paper says it

To do so, we analyze two data sources: classroom activities and concept inventory scores. We characterize classroom activities using the Classroom Observation Protocol for Undergraduate STEM (COPUS) [24].

CBE–Life Sciences Education 12(4):618–627. p. 2
Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

p. 2
Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

https://doi.org/ 10.1187/cbe.

[25] Stains M, Harshman J, Barker MK, et al (2018) Anatomy of stem teaching in american universities: A snapshot from a large-scale observation study. Science 359(6383):1468–1470. p. 0

Where the paper says it

edu; ngholmes@cornell. edu; Abstract Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning classrooms; however, the term “active learning” lacks definition and little research has examined which types and combinations of active learning strategies are most effective.

p. 0
Where the paper says it

edu; ngholmes@cornell. edu; Abstract Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning classrooms; however, the term “active learning” lacks definition and little research has examined which types and combinations of active learning strategies are most effective.

https://doi.org/ 10.1126/science. p. 0
Where the paper says it

edu; ngholmes@cornell. edu; Abstract Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning classrooms; however, the term “active learning” lacks definition and little research has examined which types and combinations of active learning strategies are most effective.

aap8892

[27] Theobald EJ, Hill MJ, Tran E, et al (2020) Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. p. 0

Where the paper says it

edu; ngholmes@cornell. edu; Abstract Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning classrooms; however, the term “active learning” lacks definition and little research has examined which types and combinations of active learning strategies are most effective.

Proceedings of the National Academy of Sciences 117(12):6476–6483. https://doi.org/ 10.1073/pnas.

[29] Connell GL, Donovan DA, Chambers TG (2016) Increasing the use of studentcentered pedagogies from moderate to high improves student learning and attitudes about biology. p. 1

Where the paper says it

One study [26] compared physics student learning outcomes in four research-based pedagogical approaches, finding that the SCALE-UP (Student-Centered Active Learning Environment with Upside-down Pedagogies) pedagogy [4] coincided with higher student learning gains than other commonly used active learning pedagogies. Another study [28] related the extent to which different active learning activities were used during biology classes to student learning outcomes and found that spending more time on group worksheets was positively correlated with student learning.

CBE–Life Sciences Education 15(1):ar3. p. 2
Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

https://doi.org/ 10.1187/cbe.

[3] Bazett T, Clough CL (2020) Course coordination as an avenue to departmental culture change. p. 7

Where the paper says it

We trained the model for both cases and found significantly higher prediction accuracy when using CG; therefore, CG was retained in our analysis and CQ was removed. 3 shows the prevalence of values for each COPUS code in the dataset as well as their relationships against the change in concept inventory score means (gain), the change in concept inventory score standard deviations, and effect sizes.

PRIMUS 31(3-5):467–482. https://doi.org/ 10.1080/10511970

[6] Burkholder E, Walsh C, Holmes NG (2020) Examination of quantitative methods for analyzing data from concept inventories. Physical Review Physics Education Research 16(1):010141. https://doi.org/ 10.1103/PhysRevPhysEducRes.

[7] Chi MTH (2009) Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science 1(1):73–105. https: //doi.org/ 10.1111/j.1756-8765.2008.01005.x

[8] Chi MTH, Wylie R (2014) The icap framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist 49(4):219–243. https://doi.org/ 10.1080/00461520.2014.965823

[7] Chi MTH (2009) Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science 1(1):73–105. https: //doi.org/ 10.1111/j.1756-8765.2008.01005.x

[9] Chin C, Osborne J (2008) Students’ questions: A potential resource for teaching and learning science. Studies in Science Education 44(1):1–39. https://doi.org/ 10.1080/030572607018281

[26] Sundstrom M, Gambrell J, Green C, et al (2026) Relative benefits of different active learning methods to conceptual physics learning. Nature Physics pp 1–6. https://doi.org/ 10.1038/s41567-026-03307-2

[28] Weir LK, Barker MK, McDonnell LM, et al (2019) Small changes, big gains: A curriculum-wide study of teaching practices and student learning in undergraduate biology. PLOS ONE 14(8):e0220900. https://doi.org/ 10.1371/journal.pone

[13] Dancy M, Henderson C, Apkarian N, et al (2024) Physics instructors’ knowledge and use of active learning has increased over the last decade but most still lecture too much. Physical Review Physics Education Research 20(1):010119. p. 2

Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

https://doi.org/ 10.1103/PhysRevPhysEducRes.

[24] Smith MK, Jones FHM, Gilbert SL, et al (2013) The classroom observation protocol for undergraduate stem (copus): A new instrument to characterize university stem classroom practices. CBE–Life Sciences Education 12(4):618–627.

[25] Stains M, Harshman J, Barker MK, et al (2018) Anatomy of stem teaching in american universities: A snapshot from a large-scale observation study. Science 359(6383):1468–1470.

[26] Sundstrom M, Gambrell J, Green C, et al (2026) Relative benefits of different active learning methods to conceptual physics learning. Nature Physics pp 1–6.

[27] Theobald EJ, Hill MJ, Tran E, et al (2020) Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences 117(12):6476–6483.

[18] Nissen JM, Talbot RM, Nasim Thompson A, et al (2018) Comparison of normalized gain and cohen’s d for analyzing gains on concept inventories. p. 2

Where the paper says it

, [26]], we use effect sizes between pre-test and post-test scores to quantify student learning. We particularly use Cohen’s d, a standardized mean difference between the two sets of scores, as this measure can be applied across different concept inventories and student populations [6, 18, 26].

Physical Review Physics Education Research 14(1):010115. p. 2
Where the paper says it

Concept inventories are commonly used by discipline-based education researchers to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core concepts in a given topic (e.

[22] Seabold S, Perktold J (2010) Statsmodels: Econometric and statistical modeling with python. p. 5

Where the paper says it

Future work should consider using continuous coding instead of two-minute intervals to capture more accurate fractions of class time spent on different activities. Notably, this work focuses on the statistical accuracy of model predictions and not the theoretical underpinnings or causality of the relationships represented by the model.

SciPy Conference Proceedings https://doi. p. 7
Where the paper says it

Modeling In this section, we describe our development of a predictive model that maps four COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group work), and SQ (student questions) – to effect sizes. Source code is available at https: //doi.

org/ 10.25080/Majora-92bf1922-011

[3] Bazett T, Clough CL (2021) Course coordination as an avenue to departmental culture change. PRIMUS 31(3-5):467–482. https://doi.org/ 10.1080/10511970

[6] Burkholder E, Walsh C, Holmes NG (2020) Examination of quantitative methods for analyzing data from concept inventories. Physical Review Physics Education Research 16(1):010141. https://doi.org/ 10.1103/PhysRevPhysEducRes.

[7] Chi MTH (2009) Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science 1(1):73–105. https: //doi.org/ 10.1111/j.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Predictive Modeling for High Impact Active Learning Classrooms: The improved AI system can predict student conceptual learning by mapping classroom time spent on lecture, group worksheets, clicker questions, and student questions to concept inventory effect sizes.

  2. Identification of Optimal Combinations: The system can identify a specific optimal combination of active learning strategies that should be tested in controlled studies, such as spending 10-20% of class time on group worksheets, 20-40% on group clicker questions, and two or more student questions per hour.

  3. Strategy Validation: The system can validate the importance of different activities by determining that in-class group worksheets are strong predictors of student learning, aligning with the ICAP framework's assertion that constructive and interactive activities lead to the highest learning gains.

Abstract

Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning classrooms; however, the term "active learning" lacks definition and little research has examined which types and combinations of associated strategies are most effective. This study uses data from 69 courses, 3 scientific disciplines, and 24 institutions to create a predictive model that maps classroom time spent on different activities to student conceptual learning. We find that a model using four classroom variables -- time spent on lecture, group worksheets, group clicker questions, and student questions -- can make predictions that account for 49% of the variance in effect sizes, as measured by R squared. We identify one type of class that is predicted to have exceptional student learning gains (effect sizes greater than 2): those that spend 10-20% of class time on group worksheets, 20-40% of class time on group clicker questions, and 10% or more of class time on student questions. We also find that classes that do not utilize group worksheets have learning outcomes comparable to fully lecture classes. These results provide testable recommendations for future controlled studies to investigate effective active learning implementation in undergraduate physics courses.

Related papers