KIS-Based Multi-LLM Verification Protocol: Role-Specialized Heterogeneous AI Debate for Inductive Theory Generation: Design and Case Studies

AbstractWe present a role-specialized heterogeneous multi-LLM verification protocol (KIS-MAI) built on the Knowledge Innovation System (KIS) framework [1][2][3]. Unlike existing Multi-Agent Debate (MAD) and AI Jury approaches that operate within a single LLM, KIS-MAI assigns distinct epistemic roles to five heterogeneous AI systems (Claude, ChatGPT, Gemini, Grok, Perplexity) under a human facilitator. The protocol operates in two modes: Single-Round Verification (Phase0->1->2) and Extended Verification with AI cross-critique (Phase0->1->2->3->4). We demonstrate that AY-KIS analogical reasoning combined with iterative SourceCheck generates inductively-derived hypotheses comparable to mid-range theory, validated through heterogeneous AI debate. Two case studies are presented: (1) beauty pageant evaluation strategy (individual level, Single-Round), reaching Burt's Structural Holes theory from a casual question through four spiral rounds; (2) Japan's digital productivity paradox (societal level, Extended), reaching institutional complementarity theory through AY-KIS spiraling and AI cross-critique. Results suggest that heterogeneous LLM role-specialization with human facilitation shows promising potential for hypothesis validation and refinement, while the 'precision vs. insight' tradeoff observed in cross-critique represents a pattern warranting further investigation in AI-assisted theory generation. Keywords: Multi-LLM verification, role-specialized AI debate, KIS, AY module, inductive theory generation, heterogeneous AI, human-AI collaboration