PaDoc: Layout-Grounded Parallel Decoding for Document Parsing

The paper addresses a fundamental inefficiency in end-to-end document parsers. As the authors state: "End-to-end document parsers provide a unified interface, but serialize page layouts and regional contents into one autoregressive sequence. This formulation forces independent regions onto a decoding path whose length grows with the total content, whereas crop-based two-stage parsers expose region-level parallelism at the cost of repeated visual prefills and fragmented page context."

The core problem is illustrated with a flat output sequence: "a causal decoder must complete yk before generating bk+1, irrespective of whether the two regions are spatially separated and semantically independent. The resulting critical path therefore scales with the sum of all regional content lengths and becomes particularly costly for pages containing many independently recognizable regions. Meanwhile, layout-first systems expose region-level parallelism, but their per-region crops require repeated visual encoding and restrict access to page-level context. General accelerators create speculative or textual branches that are not necessarily grounded in visual document instances and may be excluded from the final output."

PaDoc is a layout-grounded document parser implemented within a single MLLM that treats the predicted layout as a branching structure over a shared page representation. The key insight is a region-specific conditional-independence assumption under which the content Yk of region k is independent of other regions' contents conditioned on the page image X and its layout Bk.

Formal assumptions and factorization. The paper states two conditional-independence assumptions using conditional mutual information:

• Assumption 1 (Region-specific content sufficiency): I(Yk; Bk, Y<k X, Bk) = 0

• Assumption 2 (Crop sufficiency): I(Yk; X, Bk Xk) = 0

From Assumption 1, the authors derive the prefix-conditioned form: I(Yk; B>k, Y<k X, B≤k) = 0, which yields the factorization p⋆(b1:N, y1:N x) = Π p⋆(bk x, b<k) Π p⋆(yk x, b≤k). This "enables parallel decoding: after the layout prefix B≤k has been generated, the next layout region and the current region content are both conditionally available... Since neither variable conditions on the other in the model factorization, they can be decoded concurrently."

The paper derives the theoretical speedup: a sequential decoder has depth Dseq = l(B1:N) + Σl(Yk), while under ideal parallel scheduling, PaDoc has critical-path decoding depth DPaDoc = max l(B≤k) + l(Yk), reducing the sequential dependency path from the total length of all layout and content sequences to the longest path in the layout-content dependency graph.

Training paradigm. PaDoc uses ancestor attention with ancestor sets Anc(Bk) = X, B<k and Anc(Yk) = X, B≤k. The authors develop a packed variable-length implementation avoids a dense mask while retaining the standard next-token objective. Specifically, "the layout stream forms one causal sequence, while the query tokens of content branch Yk attend to the virtual key/value sequence [X, B≤k, Yk]... This tree-varlen realization avoids quadratic mask storage and retains standard causal FlashAttention kernels."

Training uses a standard next-token objective with teacher forcing: L(θ) = −Σ log pB θ(b⋆k x, b⋆<k) − Σ log pY θ(y⋆k x, b⋆≤k). The authors explain that "a standard autoregressive head supervises one next-token target at each position, whereas a fork in the document tree has multiple continuations. Each layout node Bk forks into the next layout node Bk+1 and its content branch Yk; we retain the former as the main stream and prepend a branch-start token to the latter."

Inference. At inference, masked parallel decoding uses region-closing tokens to instantiate branch-isolated content streams under the same ancestor visibility. The evaluated serving uses "a request-level vLLM realization rather than physically materializing the fused ancestor mask. The layout stream and every content stream execute as independent generation requests in a shared serving runtime... token-identical multimodal and layout prefixes allow automatic prefix caching to reuse complete key–value blocks that remain resident. Only the unreused suffix and branch initialization require additional prefill, and continuous batching overlaps live content requests with the continuing layout request."

Benchmark. Quality evaluation uses all 1,651 pages of OmniDocBench v1.6... which spans diverse document types, languages, layouts, and visual conditions. Serving efficiency uses a fixed 384-page subset obtained by systematic uniform sampling and shared by all systems and concurrency settings.

Training data. The authors curated approximately 11 million document-parsing samples from multiple open-source datasets, public academic and electronic documents, and in-house collections. The SFT subset comprised approximately 0.5 million examples selected via CLIP-similarity-guided sampling across clusters to preserve visual diversity and long-tail cases.

Training recipe. PaDoc is initialized from Qwen3-VL-2B... extending its vocabulary with specific control tokens. Training proceeds with continual pre-training (CPT) for one epoch on 11 million examples followed by supervised fine-tuning (SFT) on this 0.5-million-example subset, using "a global batch size of 256, and a learning-rate schedule comprising 500 warmup steps followed by cosine decay from a peak of 5 × 10−6. Training uses bf16 and FSDP across 128 A800 GPUs with a maximum sequence length of 16,384."

Comparison systems. Two-stage systems include PaddleOCR-VL 1.5, MinerU2.5-Pro, Dolphin-v2, and GLM-OCR. End-to-end systems include dots.ocr, Qianfan-OCR, MonkeyOCRv2-B-Parsing, DeepSeek-OCR-2, and HunyuanOCR 1.5. A same-backbone Sequential SFT baseline follows an identical training configuration and serializes all layout-content records into a single causal sequence.

Layout quality. "Table 1 shows that PaDoc achieves top-tier layout accuracy among end-to-end systems, with an Overall IoU/F1 of 81.4/91.1 and the highest Overall precision (93.3). Its advantage is particularly pronounced for structured regions, where it leads all four table metrics, including 92.7 IoU and 97.0 F1. It also ties the best image F1 (87.0) and obtains the best full-page F1 and recall (88.1 and 87.3) within the end-to-end group."

End-to-end parsing quality. "Table 2 likewise places PaDoc in the top tier with an Overall score of 94.24, exceeding Qianfan-OCR by 0.34 points and the strongest evaluated general VLM by 0.62 points. It attains the best end-to-end Text Edit (0.038) and Formula CDM (95.59), while delivering strong Table TEDS/TEDS-S (90.94/93.39)."

Inference efficiency. "Across every tested concurrency, PaDoc has the highest valid-page throughput and lowest P95 latency among end-to-end parsers. Relative to the same-backbone Sequential SFT baseline, it improves throughput by 67.4–118% and reduces P95 latency by 39.2–54.9%. Its throughput reaches 1.722 pages per second per GPU at 64 concurrent requests and remains near this level at higher concurrency. The paper also reports that Despite using 2.1B parameters, PaDoc consistently surpasses the 1.0B HunyuanOCR 1.5 in both efficiency metrics and substantially outpaces the 0.7B MonkeyOCRv2-B-Parsing, and approaches compact 1.0–1.3B two-stage systems, indicating that the proposed parallelism remains effective at a larger model scale."

Controlled analyses. In a same-backbone comparison, PaDoc improves Overall F1 by 1.460 points over Sequential SFT, with the largest gain on formulas (2.231 points), while preserving aggregate end-to-end quality (Overall change of only −0.345 points). The median number of autoregressive forward steps decreases from 725.0 for a sequentialized response to 297.5 along the PaDoc critical path, a median per-page reduction of 58.4% with an interquartile range of 47.7–68.2%. Median paired wall-clock speedups range from 1.453× to 1.937× across concurrency levels. A training-backend ablation shows tree-varlen is 1.163–1.185× faster than dense SDPA across all cluster sizes at 16K sequence length and 1.253× on 16 nodes at 8K, while Flex Attention does not provide a favorable trade-off in this setting.

The authors list three contributions: "(1) We derive a prefix-conditioned factorization of document parsing that exposes region-level parallelism while preserving a shared page-image prefix, thereby reducing the logical decoding depth to the longest layout-content path. (2) We develop a single-model training formulation based on ancestor attention and an efficient packed tree-varlen realization, retaining standard next-token training without auxiliary prediction heads or losses. (3) We implement layout-grounded parallel serving with shared-prefix KV reuse and demonstrate a favorable quality–efficiency trade-off against representative end-to-end and two-stage document parsers."

The paper concludes: "We presented PaDoc, a layout-grounded document parser that factorizes regional content generation under a shared page-image and layout prefix, exposing region-level parallelism without repeated visual prefills. Packed variable-length ancestor attention trains this factorization with a standard next-token objective, while concurrent serving reuses shared prefixes. On one A800 GPU, PaDoc is the fastest end-to-end parser at every tested concurrency level, improving throughput by 67.4–118% and reducing P95 latency by 39.2–54.9% relative to the same-backbone Sequential SFT baseline. Despite its 2.1B size, it also surpasses the 1.0B HunyuanOCR-1.5, demonstrating that layout-grounded parallelism can substantially reduce serving cost without modifying the MLLM architecture."

1. Parallelize decoding along the predicted layout tree

• Improved system: after generating layout boxes, decode each region’s content concurrently instead of serially.

• Benefit: critical-path decoding depth drops from total content length to the longest single layout-content path; on document pages this reduces autoregressive steps by a median of 58% and wall-clock latency by 39–55%.

2. Use a region-specific conditional-independence factorization

• Improved system: factorizes page output as p(layout) × p(region content page image + layout) rather than one long flat sequence.

• Benefit: independent regions no longer wait for each other, enabling parallel branch decoding while preserving page-level visual context.

3. Train with ancestor attention instead of dense causal masks

• Improved system: each content branch attends only to the page image, prior layout tokens, and its own region tokens; layout stream remains causal.

• Benefit: standard next-token objective works without auxiliary heads or losses; packed variable-length implementation avoids quadratic mask memory and trains 1.16–1.25× faster than dense SDPA.

4. Serve branches as independent requests with shared-prefix KV reuse

• Improved system: layout stream and per-region content streams run as concurrent requests in a vLLM-style runtime; token-identical image/layout prefixes reuse cached KV blocks.

• Benefit: only branch-specific suffixes and branch-start tokens require new prefill; continuous batching overlaps layout and content generation, increasing valid-page throughput by 67–118% vs. sequential baselines.

5. Decode layout and first content branch simultaneously after each layout token

• Improved system: once B≤k is generated, the next layout Bk+1 and current region content Yk are both conditionally available and decoded concurrently.

• Benefit: reduces sequential dependency to max layout prefix + region content rather than the sum of all regions.

6. Maintain or improve parsing quality while gaining speed

• Improved system: same-backbone layout F1 improves by 1.46 points, especially on formulas (+2.23), with negligible end-to-end quality change and top-tier table, image, and formula metrics.

• Benefit: users get faster document parsing without sacrificing accuracy, enabling cheaper large-scale ingestion of academic papers, forms, invoices, and mixed-layout pages.

7. Make the parallel mechanism architecture-agnostic

• Improved system: works inside a single existing MLLM (e.g., Qwen3-VL-2B) with only added control tokens, no new model architecture.

• Benefit: can be applied to other multimodal LLMs to convert sequential document parsers into layout-parallel parsers while reusing existing weights and serving stacks.

End-to-end document parsers provide a unified interface, but serialize page layouts and regional contents into one autoregressive sequence. This formulation forces independent regions onto a decoding path whose length grows with the total content, whereas crop-based two-stage parsers expose region-level parallelism at the cost of repeated visual prefills and fragmented page context. To retain full-page context while removing dependencies, we propose PaDoc, a layout-grounded parser that treats the predicted layout as a branching structure over a shared page representation. Under a region-sufficiency assumption, we derive a prefix-conditioned factorization in which the layout stream and regional content branches advance concurrently, reducing the decoding depth to the longest layout-content path. We realize this factorization within a single MLLM: packed variable-length ancestor attention preserves the visibility under standard next-token training, while masked parallel decoding creates branches that the evaluated vLLM backend serves as concurrent requests with cache-resident shared-prefix reuse. On OmniDocBench Full, PaDoc attains an Overall layout F1 of 91.1 and, among end-to-end parsers, a top-tier Overall score of 94.24 together with the best Text Edit (0.038) and Formula CDM (95.59). On a 384-page subset and one A800 GPU, it is the fastest end-to-end parser at five concurrency levels, improving valid-page throughput by 67.4-118% and reducing P95 latency by 39.2-54.9% relative to a same-backbone Sequential SFT baseline. Code is available at https://github.com/Longin-Yu/Padoc