MultiDocFusion: Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documents

Abstract

RAG-based QA has emerged as a powerful method for processing long industrial documents. However, conventional text chunking approaches often neglect the complex structures of long industrial documents, causing information loss and reduced answer quality. To address this, we introduce MultiDocFusion, a multimodal chunking pipeline that integrates: (i) detection of document regions using vision-based document parsing, (ii) text extraction from these regions via OCR, (iii) reconstruction of document structure into a hierarchical tree using large language model (LLM)-based document section hierarchical parsing (DSHP-LLM), and (iv) construction of hierarchical chunks through DFS-based Grouping. Extensive experiments across industrial benchmarks demonstrate that MultiDocFusion improves retrieval precision by 8–15% and ANLS QA scores by 2–3% compared to baselines, emphasizing the critical role of explicitly leveraging document hierarchy for multimodal document-based QA. These significant performance gains underscore the necessity of structure-aware chunking in enhancing the fidelity of RAG-based QA systems.

Key Contribution

A hierarchical multimodal chunking pipeline that preserves layout and improves evidence composition in industrial RAG.

Method

MultiDocFusion (Multimodal Document Structure Fusion) combines the visual layout and the hierarchical section structure of long industrial documents to build better retrieval units. It works on PDFs, scanned images, and documents with complex layouts, and supports corpus-level multi-document RAG. The pipeline has four stages:

• (a) Document parsing (DP). Vision models detect titles, section headers, text blocks, tables, and figures on each page and assign each segment a type and bounding box, producing a page-by-page layout structure.
• (b) OCR. Each segment is sent to an OCR engine, and the recognized text is linked back to its bounding box to form an annotated layout.
• (c) DSHP-LLM. An LLM instruction-tuned with LoRA on public document-hierarchy datasets reads the list of candidate section headers and assigns each header a parent (e.g., ID:3 Parent:2), forming a header tree. It then scans the spatially sorted list of all segments and attaches tables, figures, and text blocks to the current header, which yields a full document hierarchical tree.
• (d) DFS-based grouping. A depth-first traversal of the tree aggregates each node's text with its descendants and splits a chunk when it exceeds a maximum length. Each chunk keeps its hierarchy as Markdown headers, so a subsection is retrieved together with its enclosing sections.

Results

Section-header hierarchy parsing on DocHieNet and HRDH. ↳ marks DSHP-LLM applied to the backbone above; bold marks an improvement over that backbone.

Retrieval by chunking method, averaged over top-k = 1 to 4. Bold marks the best score in each column.

QA performance by chunking method, averaged over Llama-3.2-3B, Mistral-8B, and Qwen-2.5-7B readers. Bold marks the best score in each column.

• MultiDocFusion improves retrieval precision by 8–15% and ANLS by 2–3% over the chunking baselines.
• Instruction tuning with DSHP-LLM raises HRDH F1 from 0.2962–0.4389 for the base backbones to 0.6330–0.9321.

BibTeX

@inproceedings{shin-etal-2025-multidocfusion,
    title = "{M}ulti{D}oc{F}usion : Hierarchical and Multimodal Chunking Pipeline for Enhanced {RAG} on Long Industrial Documents",
    author = "Shin, Joongmin  and
      Park, Chanjun  and
      Park, Jeongbae  and
      Seo, Jaehyung  and
      Lim, Heuiseok",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.emnlp-main.1062/",
    doi = "10.18653/v1/2025.emnlp-main.1062",
    pages = "20985--21004",
    ISBN = "979-8-89176-332-6"
}