Predictive Modeling for High Impact Active Learning Classrooms
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.
Kai: We're looking at the paper: "Predictive Modeling for High Impact Active Learning Classrooms".
Mira: Here's the gist of the paper. Predictive Modeling for High Impact Active Learning Classrooms: This study uses a predictive model to map classroom time spent on different activities to student conceptual learning and identifies specific combinations of active learning strategies that consistently demonstrate exceptional student learning gains in undergraduate physics courses.
Kai: Predictive Model Development The researchers created a predictive model using multiple linear regression to map the fraction of twominute class intervals in which specific COPUS codes were present to the concept inventory effect size in that class.
Mira: The model was trained on 57 classes from extant data sources, and then used to predict effect sizes for 12 new physics and astronomy courses, yielding a mean absolute error of 0.16 after removing unreliable predictions.
Kai: To handle the complex relationships between classroom activities and student learning outcomes, the model included features such as Lec squared, Lec cubed, and all second and third order cross-terms like W G multiplied by CG.
Paper summary: Kai: So, looking at "Predictive Modeling for High Impact Active Learning Classrooms" by Ross, Sundstrom, and Holmes, the core message is that there’s a measurable way to predict which classroom activity mixes lead to the most significant student learning improvements in physics and astronomy courses.
Mira: The authors use a predictive model trained on over ten thousand students and twenty-four institutions to map time spent on lecture, group worksheets, clicker questions, and student questions to concept inventory effect sizes <ref:2603.14335#pg3>.
Lev: What’s important is that they found that the combination of spending twenty to forty percent of class time on group clicker questions and ten to twenty percent on group worksheets, along with having two or more student questions per hour, consistently predicts high learning outcomes <ref:2603.14335#pg1>.
Kai: So, the implication is that instructors don't just need to guess which active strategies are good; they can use this model to predict a specific strategy mix before they even start teaching the course <ref:2603.14335#pg2>.
Mira: It suggests a practical way for instructors to get actionable data, moving away from just anecdotal evidence about what works in the classroom <ref:2603.14335#pg1>.
Lev: From my side, this modeling tells us where the real effort should go; if you want maximum impact, you focus on that specific range of group work and questioning time rather than trying to maximize every single activity at once <ref:2603.14335#pg1>.
Kai: It really shows that even with a lot of variables in a science class, the core predictive power comes down to those four measurable activities <ref:2603.14335#pg1>.
Mira: So, this paper gives us a framework for what high impact active learning actually looks like in practice, based on empirical data from a large sample size <ref:2603.14335#pg3>.
Conclusion: Kai: So, this paper is about using a predictive model to figure out which classroom activities actually lead to big learning gains in physics and astronomy classes.
Mira: It’s titled "Predictive Modeling for High Impact Active Learning Classrooms," and the authors are Ross, Sundstrom, and Holmes.
Kai: Basically, they used data from fifty-seven existing classes to build this model that predicts concept inventory effect sizes based on how much time a class spends on different things like lectures or group worksheets.
Mira: The main finding is that you don't need to test every possible activity; there’s a specific mix—a certain percentage of time on group clicker questions, some time on worksheets, and a certain number of student questions per hour—that consistently shows exceptional learning gains.
Kai: It’s not just about doing *some* active learning; it’s about finding this precise combination that works better than just having a long lecture.
Mira: That’s the implication for instructors that they can use this model to predict which strategies are likely to work best before they even start teaching the course.
Kai: It suggests we can move past just guessing what works and get a data-driven idea of how to structure those active learning sessions.
Mira: But Ross and his team also pointed out some limitations, like the model only looks at these four specific activity types and doesn't account for everything happening in a real classroom.
Kai: They also mentioned that the data they used is quite varied across different class sizes and student groups, so applying this prediction everywhere might need some careful checking.
Mira: So, while this gives us a strong starting point for active learning design, we still have to consider if these four activities are truly the only things that matter for deep conceptual understanding.
Olive Ross, Meagan Sundstrom, N.G. Holmes
Cornell University · Drexel University
physics.ed-ph
Submitted: 2026-03-15
Updated: 2026-11-06
Comments: 20 pages, 8 figures, 2 tables
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 73/100
The gist: Predictive Modeling for High Impact Active Learning Classrooms: This study uses a predictive model to map classroom time spent on different activities to student conceptual learning and identifies…
Key concepts
- Predictive Model Development
- The study used multiple linear regression to create a mathematical model. It mapped how much time students spent on different class activities (like lectures or group work) related to the actual improvement in their conceptual understanding, using historical data from 57 classes.
- COPUS Codes
- These are codes used to categorize specific student and instructor behaviors observed during class sessions. The researchers focused on four key codes: Lec (lecture), WG (working on worksheets), CG (clicker questions in groups), and SQ (student asking a question).
- Effect Size (Cohen's d)
- This is a statistical measure used to quantify the magnitude of student learning gain. It shows how much better students performed after learning compared to before the test. A larger effect size indicates a stronger impact from the instructional method.
- Optimal Strategy Identification
- The model predicted an ideal teaching mix: 20-40% time on group clicker questions, 10-20% on group worksheets, and at least two student questions per hour. This combination showed the highest likelihood of achieving significant student learning gains.
Terminology
Summary
Predictive Modeling for High Impact Active Learning Classrooms: This study uses a predictive model to map classroom time spent on different activities to student conceptual learning and identifies specific combinations of active learning strategies that consistently demonstrate exceptional student learning gains in undergraduate physics courses. p. 0 edu;
Abstract
Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning
classrooms; however, the term “active learning” lacks definition and little research
has examined which types and combinations of active learning strategies are most
effective. In this study, we use a dataset representing over 10,000 students and
24 institutions to create a predictive model that maps classroom time spent on
different activities to student conceptual learning.Where the paper says it
Predictive Model Development
The researchers created a predictive model using multiple linear regression to map the fraction of twominute class intervals in which specific COPUS codes were present to the concept inventory effect size in that class. The model was trained on 57 classes from extant data sources, and then used to predict effect sizes for 12 new physics and astronomy courses, yielding a mean absolute error of 0. p. 2 We first trained the model on 57 classes from extant data sources [11, 26, 28]. In order to test how accurately the model is able to make predictions for physics
and astronomy classes outside of the training data, we collected COPUS and concept
inventory data from 12 new physics and astronomy courses. The model was used
to make effect size predictions for these classes based on their COPUS profiles and
these predictions were compared to the measured effect sizes as shown in Fig. After
removing unreliable predictions, the mean absolute error was 0. , Lec2
, Lec3
, Lec4
), and all second and
third order cross-terms (e. , W G ∗ CG, W G2 ∗ CG, and W G ∗ CG ∗ SQ), as features
in the model.Where the paper says it
Where the paper says it
Where the paper says it
Key Predictive Variables
The study found that four variables—classroom time spent on lecture, group worksheets, clicker questions, and student questions—are sufficient to reliably predict student learning as measured by concept inventory scores. One type of class consistently demonstrates exceptional student learning gains (effect sizes greater than 2): those that spend 10-20% of class time on group worksheets, 20-40% of class time on group clicker questions, and average two or more student questions per hour of class time. p. 0 We find that four variables –
classroom time spent on lecture, group worksheets, clicker questions, and student
questions – are sufficient to reliably predict student learning, as measured by
concept inventory scores. We identify one type of class that consistently demonstrates exceptional student learning gains (effect sizes greater than 2): those that
spend 10-20% of class time on group worksheets, 20-40% of class time on group
clicker questions, and average two or more student questions per hour of class
time. We identify one type of class that consistently demonstrates exceptional student learning gains (effect sizes greater than 2): those that
spend 10-20% of class time on group worksheets, 20-40% of class time on group
clicker questions, and average two or more student questions per hour of class
time. We also find that classes which do not utilize group worksheets consistently
have learning outcomes comparable to fully lecture classes.Where the paper says it
Where the paper says it
Optimal Strategy Identification
The model predicts one “optimal” combination of active learning strategies that should be tested in controlled studies: classes that spend 20-40% of class time on group clicker questions, spend 10-20% of class time on group worksheets, and have 2 or more student questions per hour consistently have high learning outcomes. The results also indicate that less class time spent on lecture is not necessarily better for student learning. p. 4 , up to about 60% of class time) is spent lecturing. Lastly, the results indicate that less class time spent on lecture is not necessarily better for student learning, as suggested by some prior work [e.Where the paper says it
Model Limitations and Future Work
Limitations of the study include the fact that an undergraduate science class has many more variables than are captured in the model, focusing solely on conceptual learning. The researchers also did not account for activities occurring outside of regularly scheduled lecture time or for in-class variables not captured by the four COPUS codes. p. 5 Future
work should examine the differential impact of active learning strategies on other student outcomes. We also do not account for activities that occur outside of regularly
scheduled lecture time (such as homework, quizzes, projects, laboratory sections, discussion sections, or time spent studying) or for in-class variables not captured by the
four COPUS codes (CG, SQ, Lec, and WG) included in this study. Notably, this work focuses on the statistical accuracy of model predictions and
not the theoretical underpinnings or causality of the relationships represented by
the model. Future work should evaluate whether the presence of these activities
directly impacts student learning, or if they serve as proxies for other instructional or
interpersonal variables. Even within the
four variables considered in this work, the sparsity of our data in certain regions of
this parameter space severely limits our ability to make reliable predictions. While one
of the highest effect sizes represented in our dataset was from a class that spent 100%
of class time on group worksheets, we were unable to make reliable predictions for
classes that spent more than 50% of class time on group worksheets due to the sparsity of the data in that range (see Fig.Where the paper says it
Where the paper says it
Where the paper says it
Quantifying Learning Metrics
Student learning is quantified using Cohen’s d, a standardized mean difference between pre-test and post-test scores on concept inventories. This metric provides a dimensionless measure of student learning in each course, allowing for the comparison of scores across different disciplines and concept inventories. p. 7 This metric provides a dimensionless measure of student learning in each course, allowing us to combine student scores across a range
of disciplines, courses, student populations, and concept inventories. The ability to
compare scores across concept inventories is particularly important in this work, with
28 different concept inventories represented across our dataset. 5, effect size is strongly
correlated with gain. This may aid in interpretation for readers with more familiarity
with gain as a summary statistic. The relationship between classroom activities and student learning outcomes is
complex. Some activities may have simple linear relationships with learning (e.Where the paper says it
Where the paper says it
Where the paper says it
Data Sources and Methodology
The dataset combines data from three extant studies spanning 24 institutions, 3 scientific disciplines, and class sizes ranging from 11 to 576 students with new data collected by the authors. Classroom activities are characterized using the Classroom Observation Protocol for Undergraduate STEM (COPUS), which consists of 25 different codes marking student and instructor behaviors in two-minute segments. p. 1 We characterize classroom activities using the Classroom Observation Protocol for Undergraduate
STEM (COPUS) [24]. The COPUS is a structured observation protocol that consists
of 25 different codes, or student and instructor behaviors, and observers mark which
codes occur in two-minute segments of class time. Multiple COPUS codes can be applied in each time interval; therefore, the
sum of all COPUS code quantities is more than one. We include four COPUS codes in this study: Lec (instructor lecturing), WG (students working on worksheets in groups), CG (students working on clicker questions
in groups), and SQ (student asking a question).Where the paper says it
Where the paper says it
Model Uncertainty Assessment
To quantify how sensitive the model predictions are to sample bias, bootstrap sampling was performed by randomly selecting 1,000 different subsets of 85% of the data and training the model on each subset. The final prediction is taken as the mean of that distribution, and the model uncertainty is defined as the standard deviation of that distribution. p. 2 We apply bootstrap sampling to create a suite of functions rather than
a single function, such that the model outputs a distribution of predicted effect sizes
rather than a single predicted effect size. The final model prediction is then taken as
the mean of this distribution and the prediction uncertainty (due to sample bias) is
taken as the standard deviation of the distribution. Because student
questions tend not to take up a whole two-minute COPUS interval, the right y-axis translates from
percent of class time spent on SQ to average number of student questions per hour of class. Gray
hatched regions are predictions that have high prediction uncertainty (bootstrap error >0. Second, any prediction with uncertainty above
0. 3 (≈ 10% of the full range of effect sizes present in the training set) is considered
unreliable. The model was used
to make effect size predictions for these classes based on their COPUS profiles and
these predictions were compared to the measured effect sizes as shown in Fig. After
removing unreliable predictions, the mean absolute error was 0.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Data Characterization
The dataset represents more than 10,000 students and 40 unique instructors across 24 American and Canadian universities. The data sources include extant studies from works [11], [26], and [28], supplemented by new data collected from 12 introductory physics and astronomy class sections at a private, R1 university in the United States. p. 6 The dataset from
existing studies represents 57 class sections, more than 8,000 undergraduate science
students, and 24 American and Canadian Universities. To supplement these data,
we also collected COPUS and concept inventory data from 12 introductory physics
and astronomy class sections containing more than 1,900 undergraduate students at
a private, R1 university in the United States. While this growing
body of work has importantly started to disentangle the relative impacts of different active learning strategies, the impacts of different combinations of these strategies
has yet to be rigorously analyzed in a way that produces testable predictions about
student learning outcomes for future classes. In this study, we combine data from three extant studies [11, 26, 28]–spanning 24
institutions, 3 scientific disciplines, and class sizes ranging from 11 to 576 students–
with new data collected by the authors to identify combinations of instructional
strategies that coincide with the largest student learning gains. Future work should evaluate whether the presence of these activities
directly impacts student learning, or if they serve as proxies for other instructional or
interpersonal variables. Finally, our data are highly varied across class size, student demographics, student academic level and preparedness, and instructor experience.Where the paper says it
Where the paper says it
Where the paper says it
Data Availability
All data and source code are available at https://doi. p. 7 Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Source code is available at https:
//doi. 5281/zenodo. edu; ngholmes@cornell. edu;
Abstract
Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning
classrooms; however, the term “active learning” lacks definition and little research
has examined which types and combinations of active learning strategies are most
effective. edu; ngholmes@cornell. edu;
Abstract
Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning
classrooms; however, the term “active learning” lacks definition and little research
has examined which types and combinations of active learning strategies are most
effective. E-mail(s): ogr8@cornell. edu;
Contributing authors: ms5629@drexel. 3Department of Physics, Cornell University, 410 Thurston Ave, Ithaca,
14853, New York, United States. *Corresponding author(s). Future work should seek to
disentangle these conflicting claims. Several limitations of this study also prompt future investigations.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
References
[1] Aiken JM, De Bin R, Lewandowski HJ, et al (2021) Framework for evaluating statistical models in physics education research. Physical Review Physics Education Research 17(2):020104. p. 2 Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e.Where the paper says it
[5] Bojinova E, Oigara J (2013) Teaching and learning with clickers in higher education. p. 2 Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e. Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e.Where the paper says it
Where the paper says it
[7] Chi MTH (2009) Active-constructive-interactive: A conceptual framework for differentiating learning activities. p. 1 Another study [28] related the
extent to which different active learning activities were used during biology classes to
student learning outcomes and found that spending more time on group worksheets
was positively correlated with student learning. Other research has applied the ICAP
(Interactive, Constructive, Active, and Passive) framework [7] to distinguish between
different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8]. Another study [28] related the
extent to which different active learning activities were used during biology classes to
student learning outcomes and found that spending more time on group worksheets
was positively correlated with student learning. Other research has applied the ICAP
(Interactive, Constructive, Active, and Passive) framework [7] to distinguish between
different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8]. Another study [28] related the
extent to which different active learning activities were used during biology classes to
student learning outcomes and found that spending more time on group worksheets
was positively correlated with student learning. Other research has applied the ICAP
(Interactive, Constructive, Active, and Passive) framework [7] to distinguish between
different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8]. Another study [28] related the
extent to which different active learning activities were used during biology classes to
student learning outcomes and found that spending more time on group worksheets
was positively correlated with student learning. Other research has applied the ICAP
(Interactive, Constructive, Active, and Passive) framework [7] to distinguish between
different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8]. Another study [28] related the
extent to which different active learning activities were used during biology classes to
student learning outcomes and found that spending more time on group worksheets
was positively correlated with student learning. Other research has applied the ICAP
(Interactive, Constructive, Active, and Passive) framework [7] to distinguish between
different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8]. Another study [28] related the
extent to which different active learning activities were used during biology classes to
student learning outcomes and found that spending more time on group worksheets
was positively correlated with student learning. Other research has applied the ICAP
(Interactive, Constructive, Active, and Passive) framework [7] to distinguish between
different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8]. Another study [28] related the
extent to which different active learning activities were used during biology classes to
student learning outcomes and found that spending more time on group worksheets
was positively correlated with student learning. Other research has applied the ICAP
(Interactive, Constructive, Active, and Passive) framework [7] to distinguish between
different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8]. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[9] Chin C, Osborne J (2008) Students’ questions: A potential resource for teaching and learning science. Studies in Science Education 44(1):1–39. p. 6 be selection bias effects both for instructors who volunteered their classes to participate in these studies (e. , instructors who are discipline-based education researchers
themselves may be more likely to volunteer and may exhibit larger student learning
gains than other instructors [2]) and for students who completed the pre-post surveys
(e. be selection bias effects both for instructors who volunteered their classes to participate in these studies (e. , instructors who are discipline-based education researchers
themselves may be more likely to volunteer and may exhibit larger student learning
gains than other instructors [2]) and for students who completed the pre-post surveys
(e.Where the paper says it
Where the paper says it
[26] Sundstrom M, Gambrell J, Green C, et al (2026) Relative benefits of different active learning methods to conceptual physics learning. p. 1 Other research has applied the ICAP
(Interactive, Constructive, Active, and Passive) framework [7] to distinguish between
different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8]. While this growing
body of work has importantly started to disentangle the relative impacts of different active learning strategies, the impacts of different combinations of these strategies
has yet to be rigorously analyzed in a way that produces testable predictions about
student learning outcomes for future classes. 4 shows the relationships between concept inventory mean pre-test scores, mean post-test scores, and
effect sizes present in the data. The nature of this work necessitates the use of effect
size as a summary statistic, however, as demonstrated in Fig. 4 shows the relationships between concept inventory mean pre-test scores, mean post-test scores, and
effect sizes present in the data. The nature of this work necessitates the use of effect
size as a summary statistic, however, as demonstrated in Fig. 4 shows the relationships between concept inventory mean pre-test scores, mean post-test scores, and
effect sizes present in the data. The nature of this work necessitates the use of effect
size as a summary statistic, however, as demonstrated in Fig.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[28] Weir LK, Barker MK, McDonnell LM, et al (2019) Small changes, big gains: A curriculum-wide study of teaching practices and student learning in undergraduate biology. PLOS ONE 14(8):e0220900. p. 0 We find that four variables –
classroom time spent on lecture, group worksheets, clicker questions, and student
questions – are sufficient to reliably predict student learning, as measured by
concept inventory scores. We identify one type of class that consistently demonstrates exceptional student learning gains (effect sizes greater than 2): those that
spend 10-20% of class time on group worksheets, 20-40% of class time on group
clicker questions, and average two or more student questions per hour of class
time. We find that four variables –
classroom time spent on lecture, group worksheets, clicker questions, and student
questions – are sufficient to reliably predict student learning, as measured by
concept inventory scores. We identify one type of class that consistently demonstrates exceptional student learning gains (effect sizes greater than 2): those that
spend 10-20% of class time on group worksheets, 20-40% of class time on group
clicker questions, and average two or more student questions per hour of class
time.Where the paper says it
Where the paper says it
[3] Bazett T, Clough CL (2021) Course coordination as an avenue to departmental culture change. PRIMUS 31(3-5):467–482. https://doi.org/ 10.1080/10511970
[4] Beichner RJ, Saul JM, Abbott DS, et al (2007) The student-centered activities for large enrollment undergraduate programs (scale-up) project. Reviews in Physics Education Research 1(1):1–42. p. 2 Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e.Where the paper says it
[6] Burkholder E, Walsh C, Holmes NG (2020) Examination of quantitative methods for analyzing data from concept inventories. p. 7 The ability to
compare scores across concept inventories is particularly important in this work, with
28 different concept inventories represented across our dataset. 4 shows the relationships between concept inventory mean pre-test scores, mean post-test scores, and
effect sizes present in the data. Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e. Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e. Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[8] Chi MTH, Wylie R (2014) The icap framework: Linking cognitive engagement to active learning outcomes. p. 1 Another study [28] related the
extent to which different active learning activities were used during biology classes to
student learning outcomes and found that spending more time on group worksheets
was positively correlated with student learning. Other research has applied the ICAP
(Interactive, Constructive, Active, and Passive) framework [7] to distinguish between
different types of engagement, finding that activities promoting higher levels of cognitive engagement are associated with better learning outcomes [8].Where the paper says it
[17] Nissen J, Donatello R, Van Dusen B (2019) Missing data and bias in physics education research: A case for using multiple imputation. Physical Review Physics Education Research 15(2):020106. p. 2 Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e.Where the paper says it
[24] Smith MK, Jones FHM, Gilbert SL, et al (2013) The classroom observation protocol for undergraduate stem (copus): A new instrument to characterize university stem classroom practices. p. 1 To do so, we analyze
two data sources: classroom activities and concept inventory scores. We characterize classroom activities using the Classroom Observation Protocol for Undergraduate
STEM (COPUS) [24]. To do so, we analyze
two data sources: classroom activities and concept inventory scores. We characterize classroom activities using the Classroom Observation Protocol for Undergraduate
STEM (COPUS) [24]. Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e. Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[25] Stains M, Harshman J, Barker MK, et al (2018) Anatomy of stem teaching in american universities: A snapshot from a large-scale observation study. Science 359(6383):1468–1470. p. 0 edu; ngholmes@cornell. edu;
Abstract
Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning
classrooms; however, the term “active learning” lacks definition and little research
has examined which types and combinations of active learning strategies are most
effective. edu; ngholmes@cornell. edu;
Abstract
Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning
classrooms; however, the term “active learning” lacks definition and little research
has examined which types and combinations of active learning strategies are most
effective. edu; ngholmes@cornell. edu;
Abstract
Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning
classrooms; however, the term “active learning” lacks definition and little research
has examined which types and combinations of active learning strategies are most
effective.Where the paper says it
Where the paper says it
Where the paper says it
[27] Theobald EJ, Hill MJ, Tran E, et al (2020) Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. p. 0 edu; ngholmes@cornell. edu;
Abstract
Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning
classrooms; however, the term “active learning” lacks definition and little research
has examined which types and combinations of active learning strategies are most
effective.Where the paper says it
[29] Connell GL, Donovan DA, Chambers TG (2016) Increasing the use of studentcentered pedagogies from moderate to high improves student learning and attitudes about biology. p. 1 One study [26] compared
physics student learning outcomes in four research-based pedagogical approaches,
finding that the SCALE-UP (Student-Centered Active Learning Environment with
Upside-down Pedagogies) pedagogy [4] coincided with higher student learning gains
than other commonly used active learning pedagogies. Another study [28] related the
extent to which different active learning activities were used during biology classes to
student learning outcomes and found that spending more time on group worksheets
was positively correlated with student learning. Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e.Where the paper says it
Where the paper says it
[3] Bazett T, Clough CL (2020) Course coordination as an avenue to departmental culture change. p. 7 We trained the
model for both cases and found significantly higher prediction accuracy when using
CG; therefore, CG was retained in our analysis and CQ was removed. 3 shows the prevalence of values for each COPUS code in the dataset as well
as their relationships against the change in concept inventory score means (gain), the
change in concept inventory score standard deviations, and effect sizes.Where the paper says it
[6] Burkholder E, Walsh C, Holmes NG (2020) Examination of quantitative methods for analyzing data from concept inventories. Physical Review Physics Education Research 16(1):010141. https://doi.org/ 10.1103/PhysRevPhysEducRes.
[7] Chi MTH (2009) Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science 1(1):73–105. https: //doi.org/ 10.1111/j.1756-8765.2008.01005.x
[8] Chi MTH, Wylie R (2014) The icap framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist 49(4):219–243. https://doi.org/ 10.1080/00461520.2014.965823
[7] Chi MTH (2009) Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science 1(1):73–105. https: //doi.org/ 10.1111/j.1756-8765.2008.01005.x
[9] Chin C, Osborne J (2008) Students’ questions: A potential resource for teaching and learning science. Studies in Science Education 44(1):1–39. https://doi.org/ 10.1080/030572607018281
[26] Sundstrom M, Gambrell J, Green C, et al (2026) Relative benefits of different active learning methods to conceptual physics learning. Nature Physics pp 1–6. https://doi.org/ 10.1038/s41567-026-03307-2
[28] Weir LK, Barker MK, McDonnell LM, et al (2019) Small changes, big gains: A curriculum-wide study of teaching practices and student learning in undergraduate biology. PLOS ONE 14(8):e0220900. https://doi.org/ 10.1371/journal.pone
[13] Dancy M, Henderson C, Apkarian N, et al (2024) Physics instructors’ knowledge and use of active learning has increased over the last decade but most still lecture too much. Physical Review Physics Education Research 20(1):010119. p. 2 Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e.Where the paper says it
[24] Smith MK, Jones FHM, Gilbert SL, et al (2013) The classroom observation protocol for undergraduate stem (copus): A new instrument to characterize university stem classroom practices. CBE–Life Sciences Education 12(4):618–627.
[25] Stains M, Harshman J, Barker MK, et al (2018) Anatomy of stem teaching in american universities: A snapshot from a large-scale observation study. Science 359(6383):1468–1470.
[26] Sundstrom M, Gambrell J, Green C, et al (2026) Relative benefits of different active learning methods to conceptual physics learning. Nature Physics pp 1–6.
[27] Theobald EJ, Hill MJ, Tran E, et al (2020) Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences 117(12):6476–6483.
[18] Nissen JM, Talbot RM, Nasim Thompson A, et al (2018) Comparison of normalized gain and cohen’s d for analyzing gains on concept inventories. p. 2 , [26]], we use effect sizes between pre-test and post-test scores to quantify student
learning. We particularly use Cohen’s d, a standardized mean difference between the
two sets of scores, as this measure can be applied across different concept inventories
and student populations [6, 18, 26]. Concept inventories are commonly used by discipline-based education researchers
to measure student learning. Concept inventories are, usually, research-validated, multiple choice assessments that measure students’ conceptual understanding of core
concepts in a given topic (e.Where the paper says it
Where the paper says it
[22] Seabold S, Perktold J (2010) Statsmodels: Econometric and statistical modeling with python. p. 5 Future work should consider using
continuous coding instead of two-minute intervals to capture more accurate fractions
of class time spent on different activities. Notably, this work focuses on the statistical accuracy of model predictions and
not the theoretical underpinnings or causality of the relationships represented by
the model. Modeling
In this section, we describe our development of a predictive model that maps four
COPUS codes – Lec (lecture), WG (group worksheets), CG (clicker question group
work), and SQ (student questions) – to effect sizes. Source code is available at https:
//doi.Where the paper says it
Where the paper says it
[3] Bazett T, Clough CL (2021) Course coordination as an avenue to departmental culture change. PRIMUS 31(3-5):467–482. https://doi.org/ 10.1080/10511970
[6] Burkholder E, Walsh C, Holmes NG (2020) Examination of quantitative methods for analyzing data from concept inventories. Physical Review Physics Education Research 16(1):010141. https://doi.org/ 10.1103/PhysRevPhysEducRes.
[7] Chi MTH (2009) Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science 1(1):73–105. https: //doi.org/ 10.1111/j.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Predictive Modeling for High Impact Active Learning Classrooms: The improved AI system can predict student conceptual learning by mapping classroom time spent on lecture, group worksheets, clicker questions, and student questions to concept inventory effect sizes.
-
Identification of Optimal Combinations: The system can identify a specific
optimal
combination of active learning strategies that should be tested in controlled studies, such as spending 10-20% of class time on group worksheets, 20-40% on group clicker questions, and two or more student questions per hour. -
Strategy Validation: The system can validate the importance of different activities by determining that
in-class group worksheets are strong predictors of student learning,
aligning with the ICAP framework's assertion thatconstructive and interactive activities lead to the highest learning gains.
Abstract
Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning classrooms; however, the term "active learning" lacks definition and little research has examined which types and combinations of associated strategies are most effective. This study uses data from 69 courses, 3 scientific disciplines, and 24 institutions to create a predictive model that maps classroom time spent on different activities to student conceptual learning. We find that a model using four classroom variables -- time spent on lecture, group worksheets, group clicker questions, and student questions -- can make predictions that account for 49% of the variance in effect sizes, as measured by R squared. We identify one type of class that is predicted to have exceptional student learning gains (effect sizes greater than 2): those that spend 10-20% of class time on group worksheets, 20-40% of class time on group clicker questions, and 10% or more of class time on student questions. We also find that classes that do not utilize group worksheets have learning outcomes comparable to fully lecture classes. These results provide testable recommendations for future controlled studies to investigate effective active learning implementation in undergraduate physics courses.
Related papers
- Raytracing Black Holes with AI Accelerators
- FlowMLLab: An open-source framework for reproducible computational-fluid-dynamics and scientific-machine-learning experiments
- Students' Validation Criteria for AI-Generated Physics Solutions
- Thermal fluctuations and electronic readout using a moving-coil movement
- Balancing broomsticks to a t!
- A Reproducible Inertia Dynamometer for Vocational Schools: Design, Fabrication, and Pedagogy