PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction

arXiv:2512.10888 · cs.CV · Submitted 2025-12-11 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction".

Jane: Here's the gist of the paper. Table extraction (TE) is a key challenge in document understanding PubTables-v2 unifies TE across various levels of surrounding context and is the first benchmark for multi-page TE Our evaluations reveal that while current frontier models strongly outperform small models on the most complex task (full-document multi-page TE), this gap can be closed or even reversed on narrower tasks (cropped table extraction) with targeted training.

Tom: Dataset Creation and Scope PubTables-v2 is a new large-scale dataset that unifies TE across three levels of context: cropped tables, single pages, and full documents It is the first large-scale benchmark for page-level and document-level table extraction, containing 548,414 tables annotated in their full-page context with bounding boxes and hierarchical connections to captions and footers.

Jane: The dataset includes 9,492 multi-page tables spanning up to 13 pages, including over 200 tables that span at least 5 pages and 17 tables that span 10 pages or more.

Tom: Annotation Quality Control The creation of PubTables-v2 involved strict quality control procedures to ensure high-quality annotations.

Paper summary: Tom: So we're looking at this paper now, "PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction." What they are saying is that table extraction in documents is a really hard problem right now.

Jane: It sounds like they've created a new, big dataset to tackle that difficulty by unifying how you look at tables across different levels of context, from just one page to the whole document.

Tom: Exactly. The core idea is that traditional methods first find tables and then figure out their structure, but the focus here is on models that can extract tables directly within the full page or even the entire document context without that extra step.

Lu: They're claiming this dataset is the first benchmark for multi-page table extraction, which is a pretty big deal because most of what we've done has been focused on single pages or just smaller sets of tables.

Meng: So they built this thing to give us something better to train models against, but how much data are we actually talking about? Is it enough to make a real difference for production systems?

Tom: They say the dataset contains five hundred forty-eight thousand four hundred fourteen tables annotated in their full-page context, and it includes nine thousand four hundred ninety-two multi-page tables spanning up to thirteen pages <ref:2512.10888#pg2,in their full-page context>.

Jane: That's a lot of material, and they’ve done some serious work on quality control to make sure the annotations are high quality. They say the mean normalized edit distance between cell text in a PDF and HTML has to be less than a small threshold of zero point zero two for PubTables-v2.

Lu: And they have this really strict rule, which is that if any table originally annotated by the article’s author doesn't pass that quality control step after alignment, the whole article gets discarded from the dataset. That’s how they keep things clean.

Meng: So it's not just throwing random data together; they’re filtering it based on how accurate the original annotations were. That adds a layer of rigor to what we get to work with, which is good for us when we think about deployment issues at scale.

Tom: And they’ve also shown that while frontier models, like those big multimodal large language models, are much better than smaller models on the hardest task—the full-document multi-page extraction—that advantage shrinks significantly when the task gets narrower, like just cropping a table.

Jane: That's an interesting finding because it suggests that even smaller models can actually get competitive or even better performance on those more specific, cropped table extraction tasks if they are trained specifically on PubTables-v2.

Paper summary: Lu: They also developed the first data-driven approach for predicting when tables continue across pages, which they say shows that model merging significantly improves document-level table extraction performance for models that only see single-page context.

Meng: If it helps a single-page focused model handle multi-page things by using context from other pages, then maybe we can actually deploy lighter models instead of always needing the massive multimodal ones for every task.

Tom: So what they are really emphasizing is this gap closing—the idea that targeted training on this unified data can reverse the performance advantage frontier models have on complex tasks if you narrow the focus down to specific extraction jobs.

Jane: And they introduce new evaluation metrics, like GriTS and TEDS, which are extensions of standard ones designed to handle the most general case where you might have multiple predicted and ground truth tables that don't match up at all.

Lu: They use a Hungarian algorithm to find the best one-to-one matches between predicted and ground truth tables to maximize those metrics, and then they aggregate the true positive score over all samples to compute them as pseudo F1 scores.

Meng: That sounds like a lot of computation for evaluation, but if it correctly measures performance when things don't line up perfectly, that’s a more realistic measure than just looking at one simple metric.

Tom: So the implication is that we need this unified benchmark to properly test if these new approaches are actually better than the existing ones across all these different contexts.

Jane: And beyond just testing models, they point out a real limitation: standard metrics like TEDS and GriTS assume a known one-to-one correspondence between predicted and ground truth tables, which breaks down at the page and document level evaluation.

Lu: They also mention that inferring hierarchical or other relationships between page elements is still an open area of study, even though they have this dataset connecting tables with their captions and footers across pages.

Meng: So even with all this data, it sounds like figuring out how tables relate to each other structurally across multiple documents is still a challenge that hasn't been fully solved yet.

Tom: That makes sense because the paper highlights that multi-page document parsing is still an open area of study, especially when you think about visual question answering where you need evidence from multiple pages.

Paper summary: Jane: So what this paper really changes for people who just listen to the show is that it’s giving us a concrete resource—this dataset—to test the next generation of models in a way that covers everything from single tables on one page to tables spread across ten pages.

Lu: It shifts the focus from just building bigger models to finding better ways for AI to handle context and structure across large documents coherently.

Meng: For deployment, if we can leverage these results, it might mean we don't always need those super expensive multimodal models running constantly; maybe a smaller model trained specifically on this data will suffice for many of our use cases.

Tom: So the title itself, "PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction," really just points to the fact that we needed a better way to train AI to read tables in their real world—on pages and across documents.

Jane: And the authors are Brandon Smock, Valerie Faucon-Morin, Max Sokolov, Libin Liang, Tayyibah Khanam, Amrit Ramesh, and Maury Courtland Kensho Technologies. It’s a big team coming together on this effort.

Lu: Their work connects document layout annotations from PubLayNet with the TSR annotations from PubTables-1M to create a dataset for page-contextualized table extraction, but they admit that version only contains thirty-five thousand tables in total and doesn't keep the train and test split of PubTables-1M <ref:2512.10888#pg2>.

Meng: That lack of a consistent train and test split is something I worry about when I think about how we’d actually deploy this for real traffic, because you don't want to overfit on a specific subset.

Tom: They address that by focusing on the fact that the gap between frontier models and smaller models can be closed or reversed depending on whether you are looking at the most complex task or a narrower one, which is what makes this benchmark so valuable.

Jane: So, in simple terms, it’s about providing a single place to measure how well AI understands tables when they are messy—either on one page or spread out across many pages.

Lu: The implication is that for document understanding, we need datasets that go beyond just isolated table extraction and start connecting those tables to the surrounding document structure more deeply.

Meng: If this data helps us build more robust systems, it means less failure when we try to extract information from complex reports or legal documents where tables are everywhere.

Tom: We need to keep watching how these results play out as other models start training on this new PubTables-v2 data set.

Conclusion: Tom: So we've been looking at how this new dataset, PubTables-v2, tackles table extraction across whole pages and documents.

Jane: Right, it’s a big move to unify all those different levels of context into one place for training.

Lu: The authors are Brandon Smock and his team; they built this by combining data from other projects to create something much larger than what we had before.

Meng: It’s a lot of work building that scale, but the point is they have five hundred thousand tables annotated in full context.

Tom: That scale is impressive, but I want to know what this actually means for the models we're shipping right now.

Jane: Well, it suggests that if you train a model on this unified data, it should get better at handling tables spread across many pages.

Lu: Exactly, they show that even when you narrow the task down to just cropping a table from one page, smaller models can actually compete well if they see this kind of comprehensive data.

Meng: I'm still skeptical about using a dataset like this for real traffic because we don't have a consistent way to split it into train and test sets for deployment.

Tom: That’s the practical hurdle, Meng, but the paper focuses on how performance changes depending on whether you look at the whole document versus just one page.

Jane: And they introduce new ways to measure success, using things like GriTS and TEDS to handle cases where predicted tables don't exactly match what we have in our ground truth.

Lu: They use a Hungarian algorithm to find the best possible pairing between predicted tables and true ones, which helps them get a more honest score for document-level extraction.

Meng: So the implication is that if we want better systems for legal documents or long reports, this unified benchmark gives us a clearer target to train against.

Tom: It shifts our focus from just building bigger models to finding smarter ways for AI to handle context and structure coherently across large documents.

Jane: And they’ve shown that using model merging can help single-page focused models actually predict when a table continues onto the next page.

Lu: So it’s about moving beyond just isolated extraction and starting to connect those tables to the surrounding document structure more deeply.

Meng: If we can build systems that handle this complexity, it means less failure when we try to extract information from really messy documents.

Tom: We need to keep watching how these results play out as other models start training on this PubTables-v2 data set.

Brandon Smock, Valerie Faucon-Morin, Max Sokolov, Libin Liang, Tayyibah Khanam, Amrit Ramesh, Maury Courtland

Kensho Technologies

cs.CV

Submitted: 2025-12-11

Updated: 2026-11-03

Comments: Accepted to WACV 2027

Code: https://github.com/kensho-technologies/grits

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: Table extraction (TE) is a key challenge in document understanding PubTables-v2 unifies TE across various levels of surrounding context and is the first benchmark for multi-page TE Our evaluations…

Key concepts

PubTables-v2
A new large dataset designed to test table extraction capabilities across different contexts: cropped tables, single pages, and full documents. It is the first major benchmark for page-level and document-level table extraction.
Context Levels
The dataset organizes tables into three context levels: cropped tables (small sections), single pages (one page at a time), and full documents (the entire text). This allows researchers to test models on varying levels of surrounding information.
Evaluation Metrics
The paper extends standard metrics like GriTS and TEDS to handle complex scenarios where multiple predicted and ground truth tables exist without direct matches. They use the Hungarian algorithm to find optimal one-to-one pairings before calculating these pseudo-F1 scores.
Model Performance Gap
Frontier models show a significant advantage on full-document extraction, but this gap shrinks when the task becomes more specific, such as extracting only cropped tables. Smaller models can achieve comparable performance on these narrower tasks with targeted training.

Terminology

Summary

Table extraction (TE) is a key challenge in document understanding PubTables-v2 unifies TE across various levels of surrounding context and is the first benchmark for multi-page TE Our evaluations reveal that while current frontier models strongly outperform small models on the most complex task (full-document multi-page TE), this gap can be closed or even reversed on narrower tasks (cropped table extraction) with targeted training. p. 0

Where the paper says it

Our evaluations reveal that while current frontier models strongly outperform (+0. 354 GriTSCon) small models on the most complex task (full-document multi-page TE), this gap can be closed or even reversed (−0.

Dataset Creation and Scope

PubTables-v2 is a new large-scale dataset that unifies TE across three levels of context: cropped tables, single pages, and full documents It is the first large-scale benchmark for page-level and document-level table extraction, containing 548,414 tables annotated in their full-page context with bounding boxes and hierarchical connections to captions and footers. The dataset includes 9,492 multi-page tables spanning up to 13 pages, including over 200 tables that span at least 5 pages and 17 tables that span 10 pages or more. p. 2

Where the paper says it

It is the first large-scale benchmark for page-level and document-level table extraction, containing 548,414 tables annotated in their full-page context with bounding boxes and hierarchical connections to captions and footers. – We introduce the first large dataset of multi-page tables, with 9,492 instances spanning up to 13 pages, including over 200 tables that span at least 5 pages and 17 tables that span 10 pages or more.

Annotation Quality Control

The creation of PubTables-v2 involved strict quality control procedures to ensure high-quality annotations. Specifically, the mean normalized edit distance between cells' text in the PDF and HTML must be less than a small threshold, α = 0.02 for PubTables-v2. p. 0

Where the paper says it

PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction Brandon Smock, Valerie Faucon-Morin, Max Sokolov, Libin Liang, Tayyibah Khanam, Amrit Ramesh, and Maury Courtland Kensho Technologies {brandon. smock,maury.

Furthermore, for tables in the Single Pages and Full Documents collections, if any table originally annotated by the article’s author does not pass this quality control step following alignment, the entire article is discarded from the dataset. p. 5
Where the paper says it

For tables in PubTables-v2 with full page context (i. the Single Pages and Full Documents collections), if any table originally annotated by the article’s author does not pass this quality control step following alignment, the entire article is discarded from the dataset.

Evaluation Methodology

The paper introduces extensions of standard TSR metrics—GriTS, TEDS, and exact match accuracy—to handle the most general case for page and document-level TE where there can be multiple predicted and ground truth tables with no correspondence between them. This is achieved by using the Hungarian algorithm to determine the one-to-one match between ground truth and predicted tables that maximizes each metric. p. 9

Where the paper says it

We consider the most general case, where both are sets—with no ordering and no correspondence given between the tables in each—and propose generalizations of GriTS, TEDS, and exact match accuracy to handle this case. The generalizations of TEDS and GriTS both use the Hungarian algorithm to determine the one-to-one match between the ground truth and predicted tables that maximizes each metric.

The new definition aggregates the true positive score over all samples, then computes GriTS and TEDS as a pseudo-F1 score. p. 9
Where the paper says it

The generalizations of TEDS and GriTS both use the Hungarian algorithm to determine the one-to-one match between the ground truth and predicted tables that maximizes each metric. We aggregate the result as an F1 score in order to penalize poor precision in addition to recall for a given predicted set of tables.

Model Performance Insights

The evaluation across a broad range of model types reveals that frontier MLLMs’ ability to leverage full-document context gives them a decisive advantage on complex multi-page extraction, yet this advantage disappears as the task narrows, with small models trained on PubTables-v2 capable of achieving competitive or even superior performance. The authors also developed the first data-driven approach to cross-page table continuation prediction, showing that model-based merging significantly improves document-level TE performance for single-page-context models. p. 3

Where the paper says it

Dataset Name Total Long Wide FinTabNet 113k 7,604 477 PubTables-1M 948k 72,570 31,987 PubTables-v2 136k 100,781 37,106 tion but smaller models capable of achieving competitive or even superior performance on narrower tasks. – We develop the first data-driven approach to cross-page table continuation prediction and show that model-based merging of tables across pages significantly improves document-level TE performance for single-page-context models.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Development of a new large-scale benchmark PubTables-v2 which unifies TE across various levels of surrounding context and, notably, is the first benchmark for multi-page TE. This allows for more robust evaluation of end-to-end document parsing by testing models on cropped tables, single pages, and full documents.

  2. Improvement of model performance via data-driven approaches to cross-page table continuation prediction. The paper proposes model-based merging of tables across pages significantly improves document-level TE performance for single-page-context models.

  3. Refinement of evaluation metrics for generalization across different extraction scenarios. The authors propose generalizations of GriTS, TEDS, and exact match accuracy to handle the most general case for page and documentlevel TE, where there can be multiple predicted and ground truth tables with no correspondence between them.

  4. Creation of a specialized training strategy for narrow-task models. Fine-tuning TATR on the PubTables-v2 Cropped Tables subset showed the impact that further training has on improving model performance for long and wide tables.

  5. Enhancement of visual cue recognition for multi-page table continuation. Training image classification models (like ViT-B-16) can predict continuations with near perfect accuracy when trained on PubTables-v2, which can then be used to improve page-by-page document TE by using the ViT model to predict when to merge tables.

Abstract

Table extraction (TE) is a key challenge in document understanding. Traditional approaches detect tables first, then recognize their structure. Recently, interest has surged in developing methods, such as vision-language models (VLMs), to extract tables directly in their full page or document context. However, a lack of annotated data has made progress difficult to demonstrate. To address this, we create a new large-scale dataset, PubTables-v2. PubTables-v2 unifies TE across various levels of surrounding context and, notably, is the first benchmark for multi-page TE. Our evaluations reveal that while current frontier models strongly outperform (+0.354 GriTS Con) small models on the most complex task (full-document multi-page TE), this gap can be closed or even reversed (-0.056 GriTS Con) on narrower tasks (cropped table extraction) with targeted training. Data is available at https://huggingface.co/datasets/kensho/PubTables-v2.

Sources

Related papers