POTATR: A Lightweight Image-to-Graph Model for Page-Level Table Extraction
summary
The gist
Page-level table extraction requires contextually aware models that are both accurate and efficient, and this paper introduces POTATR…
In short
POTATR is a lightweight image-to-graph model designed for page-level table extraction, extending the Table Transformer. It frames extraction as paralleldecoded spatial graph prediction rather than text generation. POTATR detects tables, structures, captions, and footers with hierarchical relationships between elements while remaining highly efficient.
Key concepts
- Image-to-Graph Model
- This model takes an image (like a page) and converts it into a graph structure representing the layout of elements such as tables. Instead of just recognizing text, it understands how different visual components relate to each other spatially, allowing for more accurate extraction of complex table structures.
- Table Transformer Extension
- POTATR builds upon the existing Table Transformer (TATR) architecture but adapts it for page-level tasks. It adds specific output classes and a relation head to handle the complexity of entire pages, enabling it to identify tables alongside other page elements like captions and footers.
- Paralleldecoded Spatial Graph Prediction
- Instead of generating text sequentially, POTATR predicts the entire graph structure at once. This approach allows the model to process spatial relationships across a page simultaneously, which is key to its fast inference time and ability to handle complex document layouts effectively.
- Spatially Grounded Recognition
- Every detected element—tables, captions, footers—is tied directly to its location on the image. This means the model doesn't just identify a table; it knows exactly where it is on the page, which improves accuracy and allows for hierarchical understanding of document components.
Terminology used across episodes
This episode discusses
- POTATR: A Lightweight Image-to-Graph Model for Page-Level Table Extraction · Paper Radio
- Qwen2.5-VL Technical Report
- PaddleOCR 3.0 Technical Report
- dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model
- SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion
- olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models
- PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction · Paper Radio
- Gemini: A Family of Highly Capable Multimodal Models
- Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence
- DeepSeek-OCR: Contexts Optical Compression
- DeepSeek-OCR 2: Visual Causal Flow
The paper
POTATR: A Lightweight Image-to-Graph Model for Page-Level Table Extraction · Read on arXiv
Brandon Smock, Libin Liang, Max Sokolov, Amrit Ramesh, Valerie Faucon-Morin, Tayyibah Khanam, Maury Courtland
Kensho Technologies
Large-scale document processing requires contextually aware table extraction (TE) that is both accurate and efficient. Yet current approaches require billions of parameters, hundreds of autoregressive steps, or costly API inference. Motivated by this, we introduce the Page-Object Table Transformer (POTATR), a lightweight 29M parameter image-to-graph model that extends the Table Transformer (TATR) for contextualized page-level TE. POTATR outperforms all models tested on the PubTables-v2 Single Pages benchmark -- including frontier MLLMs -- achieving GriTS Con of 0.964 while running over 130 times faster at 300 times lower cost. Further, POTATR's output is spatially grounded: every recognized element has a bounding box, enabling visual verification and geometric text assignment. As a result, POTATR performs unified page-level TE while composing with other models, enabling extension to scanned documents via external OCR and to full-document TE via techniques like cross-page merging. Code and models are available at https://github.com/kensho-technologies/POTATR.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "POTATR: A Lightweight Image-to-Graph Model for Page-Level Table Extraction".
Jane: Here's the gist of the paper. Page-level table extraction requires contextually aware models that are both accurate and efficient, and this paper introduces POTATR, a lightweight image-to-graph model that extends the Table Transformer for page-level table extraction <ref:2606.09788#pg2> Model Introduction POTATR is a lightweight 29M parameter image-to-graph model that extends the Table Transformer (TATR) for contextualized page-level TE <ref:2606.09788#pg3>.
Tom: Rather than autoregressive text generation, POTATR frames page-level TE as paralleldecoded spatial graph prediction <ref:2606.09788#pg4>.
Jane: It extends TATR to the page level, adding page-specific output classes and a relation head for graph prediction <ref:2606.09788#pg5>.
Tom: POTATR detects tables, their structures, captions, and footers with hierarchical relationships between them, and every element spatially grounded <ref:2606.09788#pg6>.
Paper summary: Tom: So we're looking at a paper called "POTATR: A Lightweight Image-to-Graph Model for Page-Level Table Extraction." Basically, the authors are tackling the huge problem of needing accurate table extraction across entire pages, which is usually very expensive and slow right now.
Jane: That sounds like it could be really useful for people dealing with massive amounts of document data, or even just finding tables in photos.
Lu: The core idea here is they’re moving away from the old way of doing things where you generate text token by token, which is what most current models do when they try to extract tables. Instead, POTATR frames page-level table extraction as a paralleldecoded spatial graph prediction <ref:2606.09788#pg4>.
Meng: A spatial graph prediction sounds like it’s trying to figure out the layout and relationships between elements all at once, rather than one step after another. But what's the actual model? How big is this thing?
Tom: It’s a lightweight twenty-nine million parameter image-to-graph model <ref:2606.09788#pg1>. That small size is key because they claim it can outperform other models on the PubTables-v2 Single Pages benchmark, achieving a GriTSCon of zero point nine six four <ref:2606.09788#pg1>.
Jane: A GriTSCon of zero point nine six four is quite high for this kind of task, and they also report it runs over one hundred thirty times faster at roughly three hundred times lower cost <ref:2606.09788#pg2>. That speed and cost reduction is a big deal for deployment.
Lu: What makes it different from previous attempts is that POTATR extends the Table Transformer, or TATR, which already does good work on individual tables but only in isolation <ref:2606.09788#pg3>. They take those pre-trained weights and add the necessary parts for full-page extraction.
Meng: So they’re not training a whole new massive model from scratch, just extending an existing one? That makes sense for efficiency, but does extending it really solve the structural validity issues that autoregressive models have?
Tom: They claim their modular design separates structure detection from text recognition, which means if one part fails, you can improve it separately. And they also add page-specific output classes and a relation head for graph prediction <ref:2606.09788#pg5>.
Jane: So it’s not just finding the tables; it’s trying to figure out how the tables relate to each other, like a caption or a footer attached to them. They detect those hierarchical relationships between elements and everything is spatially grounded <ref:2606.09788#pg6>.
Lalam: From my perspective, what I see is that this spatial grounding means every piece of data has a specific location on the image, which makes it verifiable visually. When you have that kind of visual verification, you can build trust in the output more easily than with just raw text generation <ref:2606.09788#pg1>.
Meng: But what about deployment? If this model is so fast, how does that translate to real-world usage when we’re processing millions of pages? Does it break down when you move from single pages to full documents with multi-page tables?
Paper summary: Lu: That’s the limitation they admit in the future work section. They say optimizing POTATR for full documents with multi-page tables is outside the scope of this paper <ref:2606.09788#pg1>. However, they did look at that gap and their cross-page merging approach achieves a GriTSCon of zero point eight two seven, which is a forty-seven point four percent reduction in error compared to processing pages without merging <ref:2606.09788#pg1>.
Tom: That’s an interesting result because they achieved that improvement without having to retrain the page-level model itself. They just composed it with a lightweight classifier, a ViT B sixteen model, and used a simple vertical concatenation rule <ref:2606.09788#pg1>.
Jane: So what does this mean for us? It means we have a much more efficient way to get table extraction than the huge language models that were dominating the space before. It lets us use these smaller, specialized models with external tools like OCR for scanned documents <ref:2606.09788#pg1>.
Lalam: I'm fairly sure of my answer here; this model is designed to be a powerful component you can plug into other systems without needing the entire multi-modal behemoth just to get table data out reliably <ref:2606.09788#pg1>.
Meng: From an engineering standpoint, the constant inference time of around twenty-one milliseconds regardless of how many cells are on a page is very attractive for throughput on table-heavy documents <ref:2606.09788#pg3>. It’s over fifty-two thousand pages per hour on a single A10G GPU <ref:2606.09788#pg1>.
Tom: That throughput advantage is what really sells it for production environments where you need to process a lot of data quickly and cheaply. It’s significantly faster than the fastest frontier multi-modal model, Gemini three point one Pro, which runs at about three hundred ninety-three pages per hour <ref:2606.09788#pg1>.
Jane: So for the person on the ground, whether they are a researcher or just someone trying to automate data processing, this suggests we can get much better extraction results while drastically cutting down on computational overhead. It makes complex tasks accessible without requiring a massive infrastructure budget <ref:2606.09788#pg1>.
Lu: The authors really focused on making it an extension of TATR, leveraging its existing success with cropped tables, but adding the necessary components for the page level context <ref:2606.09788#pg3>. It shows you don't always have to reinvent the wheel when you can adapt a strong foundation.
Tom: So to summarize, this paper introduces POTATR, which is a twenty-nine million parameter model that uses spatial graph prediction instead of text generation for page-level table extraction <ref:2606.09788#pg1>. It achieves high accuracy on the benchmark while being extremely fast and cheap to run <ref:2606.09788#pg1>.
Jane: And it’s useful because it lets us build systems that can handle page-level tables with visual verification, even if they aren't perfect for full document workflows yet <ref:2606.09788#pg1>.
Lalam: Exactly. It’s a tool for contextualized extraction that prioritizes efficiency and spatial accuracy over just raw text output <ref:2606.09788#pg1>.
Conclusion: Tom: So, we’re wrapping up on POTATR, which is this lightweight image-to-graph model for extracting tables across entire pages.
Jane: It’s basically a small model that tries to understand the whole page structure—tables, captions, footers—all at once by predicting a spatial graph.
Lu: The authors extend something called the Table Transformer, which is good for single tables but not yet for whole pages. They frame page-level table extraction as this paralleldecoded spatial graph prediction.
Meng: So it’s twenty-nine million parameters, and they trained it in two stages to handle different kinds of data distribution.
Tom: The results on the PubTables-v2 Single Pages benchmark show a GriTSCon score of zero point nine six four for POTATR, and this is super fast, over three hundred times faster than some other models at a much lower cost.
Jane: That speed and cost reduction is what really matters for putting these kinds of tools into real-world systems. It suggests that we can get better extraction results without needing huge computational power.
Lu: The architectural advantage they point out is that it separates structure detection from text recognition, which means if one part struggles, you can fix it independently.
Meng: But they also mention a limitation here: optimizing this for full documents with tables on multiple pages is beyond the scope of this work.
Tom: They did look at that gap and showed that by merging pages together with a simple classifier, they could reduce the error by almost fifty percent without retraining the main model.
Jane: That’s interesting because it shows how composition can help bridge that gap between single page extraction and multi-page document understanding.
Lu: The authors emphasize that POTATR's modular design works well with other small, specialized models, which is a big plus for building complex systems.
Meng: So the paper ends with this idea that these smaller, focused models can be powerful components in a larger pipeline.
More episodes
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI