Tracking High-Impact Researchers Through Highly Cited Tails

Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.

Kai: We're looking at the paper: "Tracking High-Impact Researchers Through Highly Cited Tails".

Mira: Here's the gist of the paper. Tracking High-Impact Researchers Through Highly Cited Tails identifies researchers with persistent participation in influential research by constructing an indicator based on authorship of top 10% cited papers in All Science Journal Code (ASJC) bins and applying it to publications affiliated with Irish institutions over a typical funding cycle Pruning redundant reasoning tokens The core finding is that researchers authoring strictly more than one ASJC top-decile paper in a given year exhibit persistent participation in the highly cited tail of the literature in subsequent years, thus the simple track record indicator possesses predictive value This suggests that research excellence as judged by qualitative peer review and influence on the research literature identify substantially different cohorts of researchers The analysis demonstrates that literature influence may be improved through greater use of data in track record pre-screening prior to qualitative peer review How the training signal is built The methodology begins with the observation that field weighted citations for publications in a given year are skewed, being approximately lognormally distributed The indicator is constructed by restricting attention to papers with either a first or corresponding author from the jurisdiction The process involves fixing the country and publication types, and then restricting attention to publication years in the range 2022–2025 Papers are scored based on their fractional score derived from their citation count relative to the top 10% in each ASJC through a citation cutoff Where the gains come from The persistence of these researchers is shown by tracking cohorts over four years, where the typical or median high scoring researcher continues to author top 10% papers up to three years before and after the year used to define the cohort The analysis also shows that reappearance in the ranking acts as a simple but powerful filter, and requiring appearance in all three years nearly doubles the probability that an active researcher records a non-zero score in 2025 compared with requiring appearance in only one year Overlap with qualitative peer review The study finds that funding awarded through qualitative peer review need not overlap with influential research in the tails of citation distributions The median FFP awardee exhibits a score of zero throughout the observation period, providing little evidence that the typical awardee enhances Ireland’s international research reputation through influence on the scientific literature However, bibliometric indicators can make predictions based on historical data and these predictions ensure that cohorts identified this way already outperform qualitative peer review awardees when it comes to influence in the literature The paper concludes that bibliometrics tracking influence in the field and qualitative peer review select substantially different cohorts of researchers The results suggest that a bibliometric pre-screening stage based on prior research performance should identify a cohort with higher subsequent influence on the field than peer review alone The typical or median researcher in the cohort succeeds to co-author a paper that is in the top 10% of field weighted citations in a subfield, leading to a non-zero median score This suggests that repeated appearances in the highly cited tail are associated with sustained future participation in the highly cited literature The paper also notes that influential papers constitute one of the most visible and numerous mechanisms through which universities build international research reputation The analysis is intended to provide a scalable mechanism for identifying researchers that have generated academic impact, thereby introducing an ex post feedback loop that is largely absent from contemporary research funding systems The final author score is easily interpreted as the number of “whole” papers a researcher co-authors in the top 10% of field weighted citations across all subfields, where purely collaborative roles are not penalised The paper plans on extending the analysis to mine patents, collaborations with industry, policy citations through Overton and potentially even altmetrics to identify researchers with a track record of economic, policy and societal impacts in publicly available databases These design choices produce a researcherlevel indicator intended to capture persistent participation in influential research while remaining robust to differences in field citation practices, career length and publication volume The paper is released into the public domain alongside this paper for reasons of transparency, reproducibility and public scrutiny The analysis shows that authoring papers in the tails of citation distributions is a persistent researcher characteristic provided researchers author strictly more than one paper annually Even though we apply our indicator to ex post research evaluation where one may want a longer window, for ex ante research assessment, a short time window leading up to funding application is preferable The typical or median researcher in the cohort succeeds to co-author a paper that is in the top 10% of field weighted citations in a subfield, leading to a non-zero median score Against a backdrop where up to 4,000 domestic researchers co-author a top 10% paper each year, we find that the median Research Ireland FFP or ERC Starting and Consolidator grant awardee selected through qualitative peer review does not appear The paper is intended to provide a scalable mechanism for identifying researchers that have generated academic impact, thereby introducing an ex post feedback loop that is largely absent from contemporary research funding systems The results are most relevant to objectives 1), 4) and 6): To build research capacity, expertise and reputation through funding excellent scientific research, To retain and improve Ireland’s reputation as a place to carry out excellent research, and To allow Ireland-based researchers to win top-tier international prizes The ERC concerns itself with gate opening It is plausible that the herd traverses the gate later, but the rareness of sleeping beauty papers suggests that this is unlikely to happen in the majority of cases The paper takes an important step providing greater transparency on the typical awardee’s performance Any publicly observable indicator of research impact can, in principle, be incorporated into a similar framework The paper is intended to provide a scalable mechanism for identifying researchers that have generated academic impact, thereby introducing an ex post feedback loop that is largely absent from contemporary research funding systems The final author score is easily interpreted as the number of “whole” papers a researcher co-authors in the top 10% of field weighted citations across all subfields, where purely collaborative roles are not penalised The paper is intended to provide a scalable mechanism for identifying researchers that have generated academic impact, thereby introducing an ex post feedback loop that is largely absent from contemporary research funding systems The final author score is easily interpreted as the number of “whole” papers a researcher co-authors in the top 10% of field weighted citations across all subfields, where purely collaborative roles are not penalised The paper is intended to provide a scalable mechanism for identifying researchers that have generated academic impact, thereby introducing an ex post feedback loop that is largely absent from contemporary research funding systems The final author score is easily interpreted as the number of “whole” papers a researcher co-authors in the top 10% of field weighted citations across all subfields, where purely collaborative roles are not penalised The paper is intended to provide a scalable mechanism for identifying researchers that have generated academic impact, thereby introducing an ex post feedback loop that is largely absent from contemporary research funding systems The final author score is easily interpreted as the number of “whole” papers a researcher co-authors in the top 10% of field weighted citations across all subfields, where purely collaborative roles are not penalised The paper is intended to provide a scalable mechanism for identifying researchers that have generated academic impact, thereby introducing an ex post feedback loop that is largely absent from contemporary research funding systems <ref:2

Title and authors: Kai: So, let’s talk about the title of this paper, "Tracking High-Impact Researchers Through Highly Cited Tails". It sounds very focused on following the trail of influential work.

Mira: I think it really boils down to using citation data—specifically focusing on those papers in the top ten percent in certain journal categories—to create a metric for research influence <ref:2610.11891#pg1>. It’s trying to find a pattern in how researchers stay relevant over time >

Lev: From an engineering standpoint, that sounds like you're trying to track which experimental setups or methodologies keep getting cited consistently, rather than just one lucky result >

Kai: Exactly. They are using this approach to build a researcher-level indicator and applying it specifically to publications from Irish institutions over a typical funding cycle >

Mira: What’s interesting is the way they frame it as an ex ante assessment tool. They aren't trying to judge a single grant application; they’re looking at historical performance to predict future impact >

Lev: That shifts the focus entirely. It moves away from just assessing the quality of one proposal and towards assessing a researcher’s long-term trajectory in contributing to the field >

Kai: Right. So, it’s about creating a scalable mechanism for identifying researchers that have generated academic impact, which is something that isn't really inherent in contemporary funding systems >

The paper's summary: Mira: Moving into the actual summary of "Tracking High-Impact Researchers Through Highly Cited Tails", they explain how they build this indicator. It starts with the observation that field weighted citations are approximately lognormally distributed, which means most publications gradually get forgotten >

Kai: So, because of that skewness, they restrict attention to papers with a first or corresponding author from the jurisdiction and fix the country and publication types in the range two thousand twenty-two to two thousand twenty-five >

Lev: That’s like you’re setting up a very specific filter for your data collection. You’re not looking at everything, just what matters in that timeframe and location >

Mira: Then they score papers based on their fractional score derived from their citation count relative to the top ten percent in each ASJC through a citation cutoff > <ref:2610.11891#pg1>

Kai: So they're using a citation cutoff to define that top ten percent within each journal category, and then scoring the papers based on how close they are to that threshold >

Lev: That makes sense. It’s operationalizing the 'top decile' idea into a measurable number you can actually put in an Excel sheet >

Mira: And then they show that researchers authoring strictly more than one ASJC top-decile paper annually exhibit persistent participation in the highly cited tail of the literature in subsequent years >

Kai: That’s the key finding, right? The simple track record indicator possesses predictive value because of that persistence >

The paper's improvements: Lev: So what are some of the suggested improvements they make to this method, beyond just building the initial indicator? They seem to be focusing on making it more robust.

Mira: One major improvement is tracking cohorts over four years. They show that the typical or median high scoring researcher continues to author top ten percent papers up to three years before and after the year used to define the cohort > <ref:2610.11891#pg1>

Kai: And they also emphasize that reappearance in the ranking acts as a simple but powerful filter. Requiring appearance in all three years nearly doubles the probability that an active researcher records a non-zero score in two thousand twenty-five compared with requiring appearance in only one year >

Lev: That’s building redundancy into the tracking mechanism. It’s about making sure you're not just catching a single outlier year, but genuine sustained activity >

Mira: They also show that they can use this approach to address gaps in quantitative ex ante assessment and ex post evaluation in the Irish context by providing this scalable mechanism for identifying researchers with academic impact >

Kai: And they mention that the final author score is easily interpreted as the number of whole papers a researcher co-authors in the top ten percent of field weighted citations across all subfields where purely collaborative roles are not penalized > <ref:2610.11891#pg1>

Conclusion: Kai: So to wrap up, this paper suggests that bibliometrics tracking influence and qualitative peer review select substantially different cohorts of researchers. It’s a tool for ex ante research assessment based on prior performance >

Mira: The main takeaway is that a bibliometric pre-screening stage should identify a cohort with higher subsequent influence than peer review alone because they show persistent participation in the highly cited literature >

Lev: So, to put it simply, the median researcher in this cohort succeeds to co-author a paper that is in the top ten percent of field weighted citations in a subfield, leading to a non-zero median score > <ref:2610.11891#pg1>

Kai: It’s about repeated appearances in the highly cited tail being associated with sustained future participation in the highly cited literature >

Mira: They are also trying to provide a scalable mechanism for identifying researchers that have generated academic impact, which is largely absent from contemporary research funding systems >

Lev: So, it seems like they’re creating an ex post feedback loop that isn't really present in how we fund research right now >