CoReTab: Improving Multimodal Table Understanding with Code-driven Reasoning

Read full text
Authors: Van-Quang Nguyen, Takayuki Okatani
Cited by: 0

Abstract

Existing datasets for multimodal table understanding, such as MMTab, primarily provide short factual answers without explicit multi-step reasoning supervision. Models trained on these datasets often generate brief responses that offers insufficient accuracy and limited interpretability into how these models arrive at the final answer. We introduce CoReTab, a code-driven reasoning framework that produces scalable, interpretable, and automatically verifiable annotations by coupling multi-step reasoning with executable Python code. Using the CoReTab framework, we curate a dataset of 115K verified samples averaging 529 tokens per response and fine-tune open-source MLLMs through a three-stage pipeline. We evaluate the resulting model trained on CoReTab across 17 MMTab benchmarks spanning table question answering, fact verification, and table structure understanding. Our model achieves significant gains of +6.2%, +5.7%, and +25.6%, respectively, over MMTab-trained baselines, while producing transparent and verifiable reasoning traces. These results establish CoReTab as a robust and generalizable supervision framework for improving multi-step reasoning in multimodal table understanding.

Fields of Study & Subjects

Multimodal Machine Learning Applications Topic Modeling Handwritten Text Recognition Techniques

Publication details

Journal / source
arXiv (Cornell University)
Publication date
27 January 2026
Document type
Preprint