MapHub: Daily Relevance Heatmaps of New arXiv Preprints for a Research Community
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Astrophysics. The day in astrophysics: observations, models and what the data forces.
Vera: We're looking at the paper: "MapHub: Daily Relevance Heatmaps of New arXiv Preprints for a Research Community".
Jocelyn: Here's the gist of the paper. MapHub assesses each day’s new arXiv submissions for every participant in a research community with a large language model reading each paper against portfolios built for the participants System Design and Methodology MapHub combines three components: Every new paper is scored for every participant and shown in one heatmap, one row per paper and one column per participant; the profiles are portfolios that participants can check and correct themselves; the scores are public and traceable A public prompt can be used to draft each participant’s research portfolio from a publication record, and the participant checks it Claude Opus 5.5 (at low effort; Anthropic 2026) reads each new paper’s title, authors, and abstract against the portfolios and returns a relevance score from 100 (“the participant wrote the paper, or would”) through 60 (“would cite it”) and 30 (“it may broaden their view”) to 0, a fit to the portfolio rather than a judgment of quality Since most cells score low, a first pass scores chunks of about 50 papers for all participants at once against short outlines of the portfolios, and only papers scoring 10 or more are rescored against a full portfolio On two test listings (748 papers), this cost about 36% of a single full pass and missed none of the 56 cells that pass scored 40 or more Relevance Scoring Metrics The relevance scores range from 0 to 100, showing whose work each paper touches as a heatmap In the authors’ blind ratings, the top 10% of papers by MapHub’s scores held about 70% of the raters’ own top 5%, pooled, against about a third for a text-matching baseline About 6% of a day’s astro-ph papers reach 30 for each participant, and one every three days reaches 60 Empirical Testing and Performance Seventy listings scored with version 0.8’s portfolios and prompt held 11,932 new submissions, scored for one individual, two groups, and two collaborations Of a day’s 87 astro-ph papers, 6.3% ± 3.5% (mean ± standard deviation) reach 30 for each participant and 0.4% reach 60 Even the individual column, spanning multiple subfields, sees only 7 ± 3 astro-ph papers a day at 30 or more: minutes of reading Reading in MapHub’s order finds the rater’s top 5% quickly (Figure 1b): MapHub’s top 5% of papers hold about a third of it, and its top 10% about 70%, against 33% for a term-frequency baseline (TF-IDF6 of title and abstract against the portfolio) reading the same share MapHub’s lead is 39 points at 10% read (95% interval: 11–60; ahead in every bootstrap draw) Discussion and Limitations A department’s heatmap could help an Astro Coffee-style discussion7 of the whole listing: Papers touching several columns are natural topics, and their highest-scoring participants natural people to ask Portfolios drawn from publication records may miss interests not yet published: The groups’ portfolios name machine learning only as a tool for their science, so MapHub may miss AI papers their members want Scores stay comparable across days, since every paper is read against the portfolio on one rubric, and rescoring changed them by 1.7 points on average MapHub is still under development; the test has limits The first author built the tool and rated three of the five columns; ratings are not a gold standard, since each paper colors the next Data Availability and Acknowledgments MapHub’s code, documentation, and tables are on GitHub K.C.
Vera: was supported by the Eric and Wendy Schmidt AI in Science Postdoctoral Fellowship, a program of Schmidt Sciences C.A.
Jocelyn: acknowledges support from the Leinweber Institute for Theoretical Physics at the University of Michigan and DOE grant DE-SC009193 This work was, in part, enabled by the Claude Team Plan for Scientists The prompts, code, and tables are public, scores are labeled as AI-generated, and each table’s commit records what produced it, so the output can be checked The paper is arXiv:2611.07174v1
astro-ph.IM: 4 Nov 2026 Keywords Keywords: Astroinformatics (78) — Astronomical research (91) — Open source software (1866) References Anthropic.
Vera: 2026, System Card: Claude Opus 5.5, https://www.anthropic.com/claude-opus-5-5 Cao, K.
Paper summary: Vera: So looking at the whole thing, MapHub seems designed to provide a daily relevance heatmap for every participant in a research community by reading new arXiv submissions against personalized portfolios. The authors are pointing out that this system could facilitate discussions within departments by showing which papers touch multiple columns, and it flags potential gaps where interests might be missed because the portfolios only come from existing publication records.
Jocelyn: It also highlights that the relevance scores are relatively stable day to day because of how every paper is scored against the established portfolio, even with rescoring adjustments. And they admit that MapHub is still under development and has limitations, like the fact that the first author built the tool and rated three of five columns, meaning those initial ratings aren't a perfect gold standard because each paper influences the next one.
Subrahmanyan: The implication for cosmology or structure formation might be seeing how quickly certain areas of research are being touched across different subfields in real time. But they also flag that since the portfolios only come from published work, MapHub might miss papers where members want to explore topics not yet published by those groups.
Vera: So, in short, it's a tool to visualize relevance between new science and established research interests for researchers at a community level. It’s not saying the AI is perfect yet, but it shows how you can use this kind of system to organize and discuss incoming literature.
Conclusion: Vera: So this paper, MapHub, is basically tracking how new arXiv papers are relevant to different research groups every single day through these heatmaps.
Jocelyn: Yeah, it’s this system where you get a visual map of what kinds of science is happening and who cares about it.
Subrahmanyan: It connects the daily influx of new research directly to established portfolios that researchers have built over time.
Vera: It lets you see if a paper touches several different areas at once, which can spark some good conversations within a department.
Jocelyn: And it shows you where those interests might be missing because the portfolios are only based on what's already been published.
Subrahmanyan: The authors point out that the AI tool helps create these personal research summaries from publication records, but they admit it might miss new ideas that haven't been published yet.
Vera: That’s a key point, because it means you have to be careful not to only look at what's already in the literature.
Jocelyn: And they mention that the scores stay pretty stable day to day because every paper is checked against the same set of interests.
Subrahmanyan: But they also say it’s still being developed, and since one person built it, there are some limitations on how good that initial rating system actually is.
Vera: It sounds like a useful way to organize the noise of new science into something that’s easier to discuss as a community.
Michigan Institute for Data and AI in Society · Department of Physics · Leinweber Institute for Theoretical Physics · Department of Astronomy
astro-ph.IM, cs.DL, cs.IR
Submitted: 2026-11-04
Updated: 2026-11-07
Comments: 3 pages, 1 figure, matches version accepted for publication
Code: https://github.com/kailicao/maphub-public
Project page: https://kailicao.github.io/maphub-public
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 75/100
The gist: MapHub assesses each day’s new arXiv submissions for every participant in a research community with a large language model reading each paper against portfolios built for the participants System…
Key concepts
- MapHub
- A system that analyzes new arXiv preprints daily for every researcher in a community. It uses an LLM to score each paper based on how relevant it is to the participant's established research portfolio, visualized as a heatmap.
- Relevance Scoring
- A metric from 0 to 100 indicating the fit between a new paper and a researcher's profile. Scores range from 100 (the paper was written by or would be cited by the participant) down to 30 or lower, showing increasing levels of connection.
- Portfolios
- Personal collections of publications that participants can check and correct. These portfolios are used as the basis for scoring new papers, allowing researchers to define their research focus and interests.
- Heatmap Visualization
- A visual representation where rows represent new papers and columns represent individual participants. The cells show the relevance score, creating a map that illustrates which papers touch which researcher's area of expertise.
Terminology
Summary
MapHub assesses each day’s new arXiv submissions for every participant in a research community with a large language model reading each paper against portfolios built for the participants
System Design and Methodology
MapHub combines three components: Every new paper is scored for every participant and shown in one heatmap, one row per paper and one column per participant; the profiles are portfolios that participants can check and correct themselves; the scores are public and traceable A public prompt can be used to draft each participant’s research portfolio from a publication record, and the participant checks it Claude Opus 5.5 (at low effort; Anthropic 2026) reads each new paper’s title, authors, and abstract against the portfolios and returns a relevance score from 100 (“the participant wrote the paper, or would”) through 60 (“would cite it”) and 30 (“it may broaden their view”) to 0, a fit to the portfolio rather than a judgment of quality Since most cells score low, a first pass scores chunks of about 50 papers for all participants at once against short outlines of the portfolios, and only papers scoring 10 or more are rescored against a full portfolio On two test listings (748 papers), this cost about 36% of a single full pass and missed none of the 56 cells that pass scored 40 or more
Relevance Scoring Metrics
The relevance scores range from 0 to 100, showing whose work each paper touches as a heatmap In the authors’ blind ratings, the top 10% of papers by MapHub’s scores held about 70% of the raters’ own top 5%, pooled, against about a third for a text-matching baseline About 6% of a day’s astro-ph papers reach 30 for each participant, and one every three days reaches 60
Empirical Testing and Performance
Seventy listings scored with version 0.8’s portfolios and prompt held 11,932 new submissions, scored for one individual, two groups, and two collaborations Of a day’s 87 astro-ph papers, 6. p. 13% ± 3.5% (mean ± standard deviation) reach 30 for each participant and 0. p. 14% reach 60 Even the individual column, spanning multiple subfields, sees only 7 ± 3 astro-ph papers a day at 30 or more: minutes of reading Reading in MapHub’s order finds the rater’s top 5% quickly (Figure 1b): MapHub’s top 5% of papers hold about a third of it, and its top 10% about 70%, against 33% for a term-frequency baseline (TF-IDF6 of title and abstract against the portfolio) reading the same share MapHub’s lead is 39 points at 10% read (95% interval: 11–60; ahead in every bootstrap draw)
Discussion and Limitations
A department’s heatmap could help an Astro Coffee-style discussion7 of the whole listing: Papers touching several columns are natural topics, and their highest-scoring participants natural people to ask Portfolios drawn from publication records may miss interests not yet published: The groups’ portfolios name machine learning only as a tool for their science, so MapHub may miss AI papers their members want Scores stay comparable across days, since every paper is read against the portfolio on one rubric, and rescoring changed them by 1.7 points on average MapHub is still under development; the test has limits The first author built the tool and rated three of the five columns; ratings are not a gold standard, since each paper colors the next
Data Availability and Acknowledgments
MapHub’s code, documentation, and tables are on GitHub K.C. was supported by the Eric and Wendy Schmidt AI in Science Postdoctoral Fellowship, a program of Schmidt Sciences C. p. 2A. acknowledges support from the Leinweber Institute for Theoretical Physics at the University of Michigan and DOE grant DE-SC009193 This work was, in part, enabled by the Claude Team Plan for Scientists The prompts, code, and tables are public, scores are labeled as AI-generated, and each table’s commit records what produced it, so the output can be checked The paper is arXiv:2611. p. 207174v1 [astro-ph. p. 0IM] 4 Nov 2026
Keywords
Keywords: Astroinformatics (78) — Astronomical research (91) — Open source software (1866)
References
Anthropic. 2026, System Card: Claude Opus 5. p. 25, https://www. p. 2anthropic.com/claude-opus-5-5 Cao, K. p. 2 2026, kailicao/maphub-public: MapHub: code and tables of the research note’s rating study (October 2026), Zenodo, doi: 10. p. 25281/ZENODO. p. 223539894 Flicke, M. p. 2, Angrabeit, G. p. 2, Iyengar, M. p. 2, et al. 2025, arXiv e-prints, arXiv:2504. p. 208385, doi: 10. p. 248550/arXiv. p. 2 p. 2 p. 2 p. 22504.08385 Haque, A. p. 2-u., & Ginsparg, P. p. 0 2009, Journal of the American Society for Information Science and Technology, 60, 2203, doi: 10. p. 21002/asi.21166 Henneken, E. p. 2, Kurtz, M. p. 2 J., Eichhorn, G. p. 2, et al. 2007, in Astronomical Society of the Pacific Conference Series, Vol. p. 2 377, Library and Information Services in Astronomy V, ed. p. 2 S. Ricketts, C. p. 2 Birdie, & E. p. 2 Isaksson (ASP), 106, doi: 10. p. 248550/arXiv.cs/0608027 Kerzendorf, W. p. 2 E. 2019, Journal of Astrophysics and Astronomy, 40, 23, doi: 10. p. 21007/s12036-019-9590-5 Ting, Y. p. 2-S. 2026, arXiv e-prints, arXiv:2603. p. 229771, doi: 10. p. 248550/arXiv.2603.29771 Wang, F. p. 2, Tan, S., Guo, Z., et al. 2026, arXiv e-prints, arXiv:2606. p. 207454, doi: 10. p. 248550/arXiv.2606.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Automated research portfolio generation using a public prompt allows participants to draft their research portfolios from publication records, which are then checked by themselves for accuracy against a model-generated draft of about 1500 to 3000 words.
-
Relevance scoring based on a large language model reading each new paper against participant portfolios generates scores from 1 to 100, categorized as
the participant wrote the paper, or would
(68%) throughwould cite it
(33%) orit may broaden their view
(15%), providing a quantifiable relevance metric rather than a qualitative judgment of quality. -
MapHub can be configured to assess new submissions to specific listings, such as astro-ph and cs.AI, allowing researchers to tailor the relevance heatmap analysis for their particular field of interest.
-
The system provides insight into reading efficiency by comparing MapHub's top 5% and top 10% of papers against a text-frequency baseline (TF-IDF), showing that
MapHub’s lead is 39 points at 10% read
compared to the baseline, indicating superior discovery in focused reading. -
The heatmap visualization allows for an Astro Coffee style discussion by identifying
Papers touching several columns are natural topics, and their highest-scoring participants natural people to ask.
-
Portfolios can be augmented by adding interests not yet reflected in publication records, such as machine learning, which
may miss AI papers their members want,
while the individual's column specifically flags AI papers.
Sources
- Scholar Inbox: Personalized Paper Recommendations for Scientists
- myADS-arXiv - a Tailor-Made, Open Access, Virtual Journal
- Your Outie Is a Wonderful Astronomer: Macrodata Refinement of the Astro-ph ArXiv Feed at Phermon Industries
- PaperFlow: Profiling, Recommending, and Adapting Across Daily Paper Streams
Related papers
- On-sky demonstration of a vector Zernike wavefront sensor in a cascaded adaptive optics system
- Euclid Data Release 1 (DR1): Calibration strategy and observations
- The General Antiparticle Spectrometer (GAPS) Antarctic Balloon Payload
- Detection and excision of radio-frequency interference in radio arrays
- Unified matched-filter and excess-power pipeline for gravitational-wave detector characterization
- Ushering in sub-arcsecond angular resolution observatories in the far-infrared: The case for a double-Fourier interferometer