M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models

Abstract

In long, multi-page industrial documents, retrieval-augmented generation (RAG) depends heavily on whether chunk boundaries follow the document’s true structure. Existing text-centric chunkers and generative hierarchy parsers often miss cross-page parent–child relations, figure/table–caption bindings, and boundary cues, which leads to fragmented or redundant chunks and degrades both retrieval and answer quality. We propose M3DocDep, an LVLM-based pipeline that first recovers block-level dependencies and then constructs chunks along the recovered document tree. The pipeline uses SharedDet as a common DP+OCR preprocessing layer, extracts multi-modal block embeddings with boundary-aware SoftROI pooling, scores candidate parent–child edges with a biaffine head, decodes a globally valid dependency tree with MST constraints, and builds tree-guided chunks annotated with section paths and page ranges. Under a shared-block evaluation protocol, M3DocDep improves STEDS by +28.5–39.6% on DHP benchmarks, retrieval nDCG by +1.1–15.3%, and QA ANLS by +4.5–15.3% on corpus-level RAG benchmarks. These results show that recovering document dependencies before chunking yields more coherent retrieval units for long, multi-page multi-modal documents.

Key Contribution

LVLM-based dependency chunking that reconstructs cross-page structure for long-document retrieval and QA.

Method

M3DocDep treats chunking as a dependency-recovery problem: it first reconstructs a global document dependency tree with a vision-language model, then assembles chunks from coherent section subtrees of that tree.

• (a) SharedDet (DP + OCR). Document parsing and OCR run once per document and produce a stable set of layout blocks (titles, headers, text, tables, figures, captions) in a unified coordinate frame. These global document blocks are shared by all downstream models, which keeps component-wise comparisons fair.
• (b) LVLM-based multi-modal block embedding. Each page passes through a frozen vision-language model to produce page multi-modal tokens. A SoftROI embedder pools the tokens inside each block with boundary-aware weighting that is robust to box jitter and OCR noise.
• (c) Global document dependency parsing. A biaffine head scores parent candidates for each block, with a header-centric prior that restricts candidates to plausible attachments. An MST decoder turns the edge scores into a globally valid dependency tree (single root, single parent, acyclic), including cross-page links.
• (d) Structure-aware dependency chunking. The tree is traversed by section subtree, figures and tables are kept with their captions, and every chunk is annotated with its section path, page range, and block IDs.

Qualitative Example

A five-page physics paper on photon trajectories around a Kerr black hole: Fig. 1 sits on page 2, and its section content is spread over pages 1 to 3.

The project page includes an interactive comparison of the chunks produced by each baseline for this document.

Results

Document hierarchical parsing on HRDS, HRDH, and DocHieNet (%). Tree-aware models and M3DocDep use the same ground-truth layout blocks.

Retrieval by chunking method (%), macro-averaged over top-k ∈ {1, 2, 3, 4} and over BGE, E5, BM25, and MM-Embed retrievers.

QA performance by chunking method (%), averaged over LLaVA-OneVision-1.5, InternVL-3.5, and Qwen2.5-VL readers. R-L = ROUGE-L, MTR = METEOR.

• M3DocDep improves STEDS by +28.5–39.6% on DHP benchmarks, retrieval nDCG by +1.1–15.3%, and QA ANLS by +4.5–15.3% on corpus-level RAG benchmarks.
• Disabling cross-page edges or replacing MST decoding with local argmax reduces macro-averaged F1/STEDS by 7.15/9.26 and 5.19/6.70 points, respectively.

Condensed ablation (%), macro-averaged over HRDS, HRDH, and DocHieNet.

BibTeX

@InProceedings{Shin_2026_CVPR,
    author    = {Shin, Joongmin and Park, Jeongbae and Seo, Jaehyung and Lim, Heuiseok},
    title     = {M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2026},
    pages     = {16603-16613}
}