HiKEY: Hierarchical Multimodal Retrieval for Open-Domain Document Question Answering

Abstract

Retrieval-augmented generation (RAG) for document-based open-domain question answering (ODQA) over large industrial corpora faces two core bottlenecks: routing to the correct document and combining scattered evidence. Flat text chunks and page-level images often fail to (i) identify the right document among thousands of candidates and (ii) connect multimodal evidence, such as tables and figures, within a fixed token budget. We propose HiKEY, a hierarchical tree-based multimodal retrieval framework that treats document hierarchy as a first-class retrieval signal. Rather than simply chunking text, HiKEY uses Document Hierarchical Parsing (DHP) to reconstruct a logical heterogeneous graph with explicit parent–child relations. At query time, HiKEY follows a hierarchical coarse-to-fine process: it first performs global routing with hierarchical indexes to prune the corpus, and then ranks sections with a multimodal fusion strategy that selects the most discriminative evidence. It finally builds a token-efficient evidence subgraph through hybrid structural–semantic packing. Experiments on ODQA benchmarks show that HiKEY outperforms page- and chunk-based baselines, improving retrieval recall by up to 12.9 points and end-to-end QA by up to 6.8 points.

Key Contribution

Hierarchical retrieval for multimodal document QA with structured evidence assembly.

Method

HiKEY follows how people search long documents: it first narrows the scope using the document structure, then reads the relevant sections together with their tables and figures. The framework has an offline phase and an online phase.

• Offline: hierarchy-aware graph and index construction. The Document Hierarchical Parsing (DHP) module from M3DocDep recovers a logical hierarchy tree from page-level layout blocks. Each text, table, or figure block becomes a node with parent-child edges and an ancestor section path (e.g., Title > Section 5 > 5.3). HiKEY then builds two kinds of index cards: a Doc_card (title and section paths) for global routing, and a Sec_card that stores the text, table, and figure units of each section. Tables and figures without captions are enriched with the logically preceding text node found by traversing the tree.
• Online Stage 1: hierarchical document routing. Doc_cards are ranked with a hybrid lexical and dense score, and only the top candidate documents are kept.
• Online Stage 2: hierarchical section MaxSim scoring. Within the candidate documents, each unit is scored with a type-specific encoder (hybrid text scoring for text; visual similarity combined with the upper context for tables and figures). A section takes the maximum score over its units, and the final score combines document-level and section-level evidence.
• Hierarchical subgraph assembly. For each selected section, HiKEY inserts the anchor unit with its governing headers (ancestry-aware packing). It then fills the remaining token budget with tables and figures under the same parent section and, if budget remains, semantically similar units from other sections (hybrid evidence expansion).

Results

Document-level retrieval on M3DocVQA and FRAMES, averaged over K = 1, ..., 10.

End-to-end QA on M3DocVQA and FRAMES.

• HiKEY outperforms page- and chunk-based baselines on both benchmarks, improving retrieval recall by up to 12.9 points and end-to-end QA by up to 6.8 points.
• In the ablation below, field-separated hierarchy indexing raises averaged R@10 from 81.2 (body only) to 88.6, and coarse-to-fine routing outperforms document-only (87.7) and section-only (76.1) routing.

Ablation of Stage-1 hierarchy indexing and routing, and of Stage-2 retrieval scope and multimodal fusion signals (averaged R@10).

BibTeX

@inproceedings{shin-etal-2026-hikey,
    title = "{H}i{KEY}: Hierarchical Multimodal Retrieval for Open-Domain Document Question Answering",
    author = "Shin, Joongmin  and
      Shim, Gyuho  and
      Park, Jeongbae  and
      Seo, Jaehyung  and
      Lim, Heuiseok",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.acl-long.818/",
    doi = "10.18653/v1/2026.acl-long.818",
    pages = "17967--17987",
    ISBN = "979-8-89176-390-6"
}