GenAI-Assisted Writing and Academic Performance: Evidence from the Social and Behavioral Sciences

summary

The gist

This paper estimates the effect of Generative AI (GenAI) adoption on scientific productivity and quality in the social and behavioral sciences because GenAI tools may help lower some structural…

In short

The study examined how using Generative AI (GenAI) tools like ChatGPT affects academic output in social and behavioral sciences. Researchers found that GenAI adoption leads to a significant increase in published papers, especially for early-career researchers and those from non-English speaking countries. This suggests GenAI lowers publication barriers and boosts research capacity.

Key concepts

GenAI-assisted writers
Authors identified by tracking specific language markers in their paper titles and abstracts that increased sharply after ChatGPT's release. This classification measures how much an author relies on AI tools when writing their academic work.
Difference-in-differences design
A statistical method used to compare the changes in outcomes (like publications) over time between a group that adopted GenAI and a similar group that did not. It helps isolate the specific impact of GenAI adoption from other external factors.
Publication output
The measure of research productivity, specifically the number of peer-reviewed papers an author publishes. The study found this metric increased substantially for authors who used GenAI tools compared to a baseline period.
SCImago Journal Rank (SJR)
A metric used to gauge the quality or impact of academic journals. The study tracked how journal placement improved for GenAI writers, showing modest gains in journal ranking based on this measure.

Terminology used across episodes

This episode discusses

The paper

GenAI-Assisted Writing and Academic Performance: Evidence from the Social and Behavioral Sciences · Read on arXiv

Dragan Filimonovic, Christian Rutzera, Conny Wunscha

University of Basel · CESifo · DIW Berlin · IZA

This paper studies how Generative AI (GenAI)-assisted writing is associated with peer-reviewed publication output and journal placement in the social and behavioral sciences. We select lexical markers whose frequency rose sharply after ChatGPT's release in late 2022 and classify authors as GenAI-assisted writers when the share of these markers in their papers' titles and abstracts rises relative to their own pre-release baseline. Using matched author-level panel data and a difference-in-differences design, we estimate a 72% increase in publication output among GenAI-assisted writers relative to matched comparison authors in 2025. Across alternative specifications, the estimates range from roughly 52% to 88%. Mean journal placement based on the SCImago Journal Rank (SJR) improves by roughly 1%, but this modest increase is less consistent across different specifications. Within authors' annual publication portfolios, the highest placement improves by 5%, whereas the lowest declines by 3%. However, only the improvement at the top persists once the annual number of publications is held constant. The increase in output is accompanied by an expansion of collaboration networks and a rise in single-authored publications. Productivity gains are largest among early-career researchers and authors from countries linguistically distant from English, groups likely to face greater writing-related barriers to publication. The results suggest that GenAI-assisted writing may lower these barriers and increase researchers' capacity to bring more projects to publication.

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: We're looking at the paper: "GenAI-Assisted Writing and Academic Performance".

Warren: Here's the gist of the paper. This paper estimates the effect of Generative AI (GenAI) adoption on scientific productivity and quality in the social and behavioral sciences because GenAI tools may help lower some structural barriers in academic publishing and promote more inclusive participation in research.

Ruth: Main Findings GenAI adoption is associated with sizable increases in research productivity, measured by the number of published papers.

Warren: It also leads to moderate gains in publication quality, based on journal impact factors.

Ruth: These effects are most pronounced among early-career researchers, authors working in technically complex subfields, and those from non-English-speaking countries.

Paper summary: Ruth: So we're looking at this paper by Filimonovic, Rutzera, and Wunscha titled "GenAI-Assisted Writing and Academic Performance: Evidence from the Social and Behavioral Sciences." Basically they're estimating how using GenAI tools affects research output and quality in the social sciences.

Warren: They use matched author-level panel data and a difference-in-differences design to see if there's a real impact of this technology adoption after ChatGPT came out late in two thousand twenty-two <ref:2510.02408#pg1,matched author-level panel data and a difference-in-differences design>.

Iris: The paper identifies authors as GenAI-assisted writers by looking at how frequently certain words appeared in their titles and abstracts compared to their baseline before the release.

Ruth: They found that GenAI adoption is linked to a sizable increase in research productivity, specifically when we measure that by the number of published papers. It also suggests a moderate gain in publication quality based on journal impact factors.

Warren: The most notable effects they see are concentrated among early-career researchers and those working in technically complex subfields, as well as authors coming from non-English speaking countries.

Iris: They also found a small but statistically significant increase in the average impact factor of publications for those authors who use GenAI tools. This suggests that GenAI tools might improve the efficiency of scientific writing without a noticeable drop in research quality.

Ruth: So, what does this mean for us? It points toward the idea that these AI tools could lower some of the structural barriers in academic publishing and encourage more inclusive participation in research.

Warren: The paper also noted that productivity gains are most pronounced among early-career researchers and authors from countries linguistically distant from English, which are groups likely to face bigger writing challenges when trying to get published.

Iris: They did find that within authors' annual publication portfolios, the highest placement improved by five percent, while the lowest placement declined by three percent. But they said that this improvement at the top only lasts once you hold the total number of publications per year constant.

Ruth: That's interesting because it suggests there are some uneven effects across different parts of a researcher's career or output level. It shows that the benefits aren't uniform across every single author in their portfolio.

Warren: They also mentioned that this increase in output is paired with an expansion of collaboration networks and a rise in single-authored publications, which is something to consider for how this changes the research landscape overall.

Iris: The study’s approach relies on identifying behavioral indicators, like shifts in language markers, as evidence of adoption. A potential limitation they flag is that their analysis focuses specifically on the social and behavioral sciences domain.

Ruth: That makes sense because they are targeting a specific area where writing skills and communication style are clearly important factors in getting work published. It’s about understanding how these tools interact with those specific academic norms.

Warren: This paper, "GenAI-Assisted Writing and Academic Performance: Evidence from the Social and Behavioral Sciences," gives us benchmark estimates on how GenAI adoption affects scientific productivity and quality right now.

Iris: The authors conclude that they point to important policy implications, specifically the need to promote equitable access to these AI tools, especially in non-English speaking environments where the potential benefits seem strongest.

Ruth: So what's the bigger picture for research funding or institutions? It suggests a push for making sure everyone has access and considering integrating GenAI subscriptions into project funding schemes.

Warren: And they also stress that ethical considerations have to guide how this technology is integrated into academic practice, which is a really important point to keep in mind as we move forward.

Conclusion: Ruth: So, to wrap up, this paper is looking at how using AI for writing affects research output and quality in social science papers after ChatGPT came out.

Warren: Right, so the core finding is that these authors get a noticeable bump in how many papers they publish and some improvement in the quality of those journals.

Iris: I think the key thing to remember is that these gains are happening most strongly for early-career researchers and people from countries that don't speak English well.

Ruth: Exactly, it points toward AI potentially lowering some of those structural hurdles in getting research published and making sure more people can participate.

Warren: But there’s a bit of nuance here because they found that the top authors see a bigger gain in placement, but only if you keep the total number of papers you publish per year the same.

Iris: And they also mentioned that this productivity boost comes with changes in how researchers work together and sometimes more people writing their own papers.

Ruth: It suggests that GenAI might be helping bring more projects to publication, especially for groups facing bigger writing barriers right now.

Warren: So if you're thinking about how research funding works, this gives us a benchmark estimate on what we can expect from these tools in the near future.

Iris: And it really forces us to think about how institutions should handle access and support for these kinds of new technologies.

More episodes

← Home