NCRAgent: A Neuro-Inspired Multiagent Framework for Transparent and Explainable Medical VQA

Medical Visual Question Answering (MedVQA) systems are increasingly important for helping clinicians interpret and diagnose medical images. However, many existing MedVQA solutions provide limited support for ambiguous queries and lack sufficient clarity in how reports are generated, weakening their usefulness for clinical decision-making. To address these issues, we propose a multi-agent system inspired by human neurological processes to improve accuracy, transparency, and explainability. Our framework includes three specialized agents: (1) a Query Rewrite Agent (inspired by the dorsolateral prefrontal cortex) that decomposes complex queries into structured sub-questions, (2) a Visual Analysis Agent (modeled after the inferotemporal cortex) that interprets medical images, and (3) a Report Generation Agent (drawing on the hippocampus and prefrontal cortex) that synthesizes final answers. The system exposes its internal reasoning steps by dividing tasks across multiple agents, ensuring a transparent decision-making process. Specifically, we break down the original query into smaller, interactive sub-questions. On the other hand, our report generation process incorporates positive and negative keywords, further enhancing the explainability of the final output. Related code and prompts will be available athttps://github.com/lv3mmray/NCRAgent.