PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction
summary
The gist
Table extraction (TE) is a key challenge in document understanding PubTables-v2 unifies TE across various levels of surrounding context and is the first benchmark for multi-page TE Our evaluations…
In short
PubTables-v2 is a large dataset unifying table extraction across cropped tables, single pages, and full documents. It contains over 548,000 annotated tables in their full context, including multi-page ones. Evaluations show that while large models excel at complex multi-page tasks, smaller models can perform competitively on narrower tasks like cropped table extraction.
Key concepts
- PubTables-v2
- A new large dataset designed to test table extraction capabilities across different contexts: cropped tables, single pages, and full documents. It is the first major benchmark for page-level and document-level table extraction.
- Context Levels
- The dataset organizes tables into three context levels: cropped tables (small sections), single pages (one page at a time), and full documents (the entire text). This allows researchers to test models on varying levels of surrounding information.
- Evaluation Metrics
- The paper extends standard metrics like GriTS and TEDS to handle complex scenarios where multiple predicted and ground truth tables exist without direct matches. They use the Hungarian algorithm to find optimal one-to-one pairings before calculating these pseudo-F1 scores.
- Model Performance Gap
- Frontier models show a significant advantage on full-document extraction, but this gap shrinks when the task becomes more specific, such as extracting only cropped tables. Smaller models can achieve comparable performance on these narrower tasks with targeted training.
Terminology used across episodes
This episode discusses
- PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction · Paper Radio
- Qwen2.5-VL Technical Report
- PaddleOCR 3.0 Technical Report
- CTE: A Dataset for Contextualized Table Extraction
- dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model
- SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion
- PaliGemma 2: A Family of Versatile VLMs for Transfer
- Gemini: A Family of Highly Capable Multimodal Models
- Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence
- DeepSeek-OCR: Contexts Optical Compression
- DeepSeek-OCR 2: Visual Causal Flow
The paper
PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction · Read on arXiv
Brandon Smock, Valerie Faucon-Morin, Max Sokolov, Libin Liang, Tayyibah Khanam, Amrit Ramesh, Maury Courtland
Kensho Technologies
Table extraction (TE) is a key challenge in document understanding. Traditional approaches detect tables first, then recognize their structure. Recently, interest has surged in developing methods, such as vision-language models (VLMs), to extract tables directly in their full page or document context. However, a lack of annotated data has made progress difficult to demonstrate. To address this, we create a new large-scale dataset, PubTables-v2. PubTables-v2 unifies TE across various levels of surrounding context and, notably, is the first benchmark for multi-page TE. Our evaluations reveal that while current frontier models strongly outperform (+0.354 GriTS Con) small models on the most complex task (full-document multi-page TE), this gap can be closed or even reversed (-0.056 GriTS Con) on narrower tasks (cropped table extraction) with targeted training. Data is available at https://huggingface.co/datasets/kensho/PubTables-v2.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction".
Jane: Here's the gist of the paper. Table extraction (TE) is a key challenge in document understanding PubTables-v2 unifies TE across various levels of surrounding context and is the first benchmark for multi-page TE Our evaluations reveal that while current frontier models strongly outperform small models on the most complex task (full-document multi-page TE), this gap can be closed or even reversed on narrower tasks (cropped table extraction) with targeted training.
Tom: Dataset Creation and Scope PubTables-v2 is a new large-scale dataset that unifies TE across three levels of context: cropped tables, single pages, and full documents It is the first large-scale benchmark for page-level and document-level table extraction, containing 548,414 tables annotated in their full-page context with bounding boxes and hierarchical connections to captions and footers.
Jane: The dataset includes 9,492 multi-page tables spanning up to 13 pages, including over 200 tables that span at least 5 pages and 17 tables that span 10 pages or more.
Tom: Annotation Quality Control The creation of PubTables-v2 involved strict quality control procedures to ensure high-quality annotations.
Paper summary: Tom: So we're looking at this paper now, "PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction." What they are saying is that table extraction in documents is a really hard problem right now.
Jane: It sounds like they've created a new, big dataset to tackle that difficulty by unifying how you look at tables across different levels of context, from just one page to the whole document.
Tom: Exactly. The core idea is that traditional methods first find tables and then figure out their structure, but the focus here is on models that can extract tables directly within the full page or even the entire document context without that extra step.
Lu: They're claiming this dataset is the first benchmark for multi-page table extraction, which is a pretty big deal because most of what we've done has been focused on single pages or just smaller sets of tables.
Meng: So they built this thing to give us something better to train models against, but how much data are we actually talking about? Is it enough to make a real difference for production systems?
Tom: They say the dataset contains five hundred forty-eight thousand four hundred fourteen tables annotated in their full-page context, and it includes nine thousand four hundred ninety-two multi-page tables spanning up to thirteen pages <ref:2512.10888#pg2,in their full-page context>.
Jane: That's a lot of material, and they’ve done some serious work on quality control to make sure the annotations are high quality. They say the mean normalized edit distance between cell text in a PDF and HTML has to be less than a small threshold of zero point zero two for PubTables-v2.
Lu: And they have this really strict rule, which is that if any table originally annotated by the article’s author doesn't pass that quality control step after alignment, the whole article gets discarded from the dataset. That’s how they keep things clean.
Meng: So it's not just throwing random data together; they’re filtering it based on how accurate the original annotations were. That adds a layer of rigor to what we get to work with, which is good for us when we think about deployment issues at scale.
Tom: And they’ve also shown that while frontier models, like those big multimodal large language models, are much better than smaller models on the hardest task—the full-document multi-page extraction—that advantage shrinks significantly when the task gets narrower, like just cropping a table.
Jane: That's an interesting finding because it suggests that even smaller models can actually get competitive or even better performance on those more specific, cropped table extraction tasks if they are trained specifically on PubTables-v2.
Paper summary: Lu: They also developed the first data-driven approach for predicting when tables continue across pages, which they say shows that model merging significantly improves document-level table extraction performance for models that only see single-page context.
Meng: If it helps a single-page focused model handle multi-page things by using context from other pages, then maybe we can actually deploy lighter models instead of always needing the massive multimodal ones for every task.
Tom: So what they are really emphasizing is this gap closing—the idea that targeted training on this unified data can reverse the performance advantage frontier models have on complex tasks if you narrow the focus down to specific extraction jobs.
Jane: And they introduce new evaluation metrics, like GriTS and TEDS, which are extensions of standard ones designed to handle the most general case where you might have multiple predicted and ground truth tables that don't match up at all.
Lu: They use a Hungarian algorithm to find the best one-to-one matches between predicted and ground truth tables to maximize those metrics, and then they aggregate the true positive score over all samples to compute them as pseudo F1 scores.
Meng: That sounds like a lot of computation for evaluation, but if it correctly measures performance when things don't line up perfectly, that’s a more realistic measure than just looking at one simple metric.
Tom: So the implication is that we need this unified benchmark to properly test if these new approaches are actually better than the existing ones across all these different contexts.
Jane: And beyond just testing models, they point out a real limitation: standard metrics like TEDS and GriTS assume a known one-to-one correspondence between predicted and ground truth tables, which breaks down at the page and document level evaluation.
Lu: They also mention that inferring hierarchical or other relationships between page elements is still an open area of study, even though they have this dataset connecting tables with their captions and footers across pages.
Meng: So even with all this data, it sounds like figuring out how tables relate to each other structurally across multiple documents is still a challenge that hasn't been fully solved yet.
Tom: That makes sense because the paper highlights that multi-page document parsing is still an open area of study, especially when you think about visual question answering where you need evidence from multiple pages.
Paper summary: Jane: So what this paper really changes for people who just listen to the show is that it’s giving us a concrete resource—this dataset—to test the next generation of models in a way that covers everything from single tables on one page to tables spread across ten pages.
Lu: It shifts the focus from just building bigger models to finding better ways for AI to handle context and structure across large documents coherently.
Meng: For deployment, if we can leverage these results, it might mean we don't always need those super expensive multimodal models running constantly; maybe a smaller model trained specifically on this data will suffice for many of our use cases.
Tom: So the title itself, "PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction," really just points to the fact that we needed a better way to train AI to read tables in their real world—on pages and across documents.
Jane: And the authors are Brandon Smock, Valerie Faucon-Morin, Max Sokolov, Libin Liang, Tayyibah Khanam, Amrit Ramesh, and Maury Courtland Kensho Technologies. It’s a big team coming together on this effort.
Lu: Their work connects document layout annotations from PubLayNet with the TSR annotations from PubTables-1M to create a dataset for page-contextualized table extraction, but they admit that version only contains thirty-five thousand tables in total and doesn't keep the train and test split of PubTables-1M <ref:2512.10888#pg2>.
Meng: That lack of a consistent train and test split is something I worry about when I think about how we’d actually deploy this for real traffic, because you don't want to overfit on a specific subset.
Tom: They address that by focusing on the fact that the gap between frontier models and smaller models can be closed or reversed depending on whether you are looking at the most complex task or a narrower one, which is what makes this benchmark so valuable.
Jane: So, in simple terms, it’s about providing a single place to measure how well AI understands tables when they are messy—either on one page or spread out across many pages.
Lu: The implication is that for document understanding, we need datasets that go beyond just isolated table extraction and start connecting those tables to the surrounding document structure more deeply.
Meng: If this data helps us build more robust systems, it means less failure when we try to extract information from complex reports or legal documents where tables are everywhere.
Tom: We need to keep watching how these results play out as other models start training on this new PubTables-v2 data set.
Conclusion: Tom: So we've been looking at how this new dataset, PubTables-v2, tackles table extraction across whole pages and documents.
Jane: Right, it’s a big move to unify all those different levels of context into one place for training.
Lu: The authors are Brandon Smock and his team; they built this by combining data from other projects to create something much larger than what we had before.
Meng: It’s a lot of work building that scale, but the point is they have five hundred thousand tables annotated in full context.
Tom: That scale is impressive, but I want to know what this actually means for the models we're shipping right now.
Jane: Well, it suggests that if you train a model on this unified data, it should get better at handling tables spread across many pages.
Lu: Exactly, they show that even when you narrow the task down to just cropping a table from one page, smaller models can actually compete well if they see this kind of comprehensive data.
Meng: I'm still skeptical about using a dataset like this for real traffic because we don't have a consistent way to split it into train and test sets for deployment.
Tom: That’s the practical hurdle, Meng, but the paper focuses on how performance changes depending on whether you look at the whole document versus just one page.
Jane: And they introduce new ways to measure success, using things like GriTS and TEDS to handle cases where predicted tables don't exactly match what we have in our ground truth.
Lu: They use a Hungarian algorithm to find the best possible pairing between predicted tables and true ones, which helps them get a more honest score for document-level extraction.
Meng: So the implication is that if we want better systems for legal documents or long reports, this unified benchmark gives us a clearer target to train against.
Tom: It shifts our focus from just building bigger models to finding smarter ways for AI to handle context and structure coherently across large documents.
Jane: And they’ve shown that using model merging can help single-page focused models actually predict when a table continues onto the next page.
Lu: So it’s about moving beyond just isolated extraction and starting to connect those tables to the surrounding document structure more deeply.
Meng: If we can build systems that handle this complexity, it means less failure when we try to extract information from really messy documents.
Tom: We need to keep watching how these results play out as other models start training on this PubTables-v2 data set.
More episodes
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI