Structural Effects of the Knowledge Innovation System on AI Judgment Patterns: — 3 Models × 4 Conditions × 5 Questions × 30 Repetitions —

Read full text Summarise with VIRAT
Authors: Hiroyasu Hasegawa
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Abstract This study experimentally examines the structural effects of the Knowledge Innovation System (KIS) on the judgment patterns of three generative AI models (ChatGPT, Claude, and Gemini), analyzing 1,800 judgments across four experimental conditions, five questions, and 30 repetitions each, from both quantitative and qualitative perspectives. Key findings are as follows: (1) KIS introduction significantly altered judgment distributions (chi-squared test, p < 10^-28); (2) independent analysis by evaluator revealed a strong KIS pure effect in Gemini evaluations (r = 0.88, p < .001); (3) KIS x Step interaction diverged into three patterns — independent additive, Step-excessive, and prerequisite types — depending on the internal structure of the question; and (4) judgment consistency and depth of reasoning structure were confirmed to be independent dimensions, with high consistency not necessarily indicating high-quality judgment. KIS functions as a "judgment process structuring device" rather than an "answer-generating device," and the results demonstrate that design choices adapted to the variable structure of the question are necessary.

Fields of Study & Subjects

Artificial Intelligence Applications Topic Modeling Information Retrieval and Search Behavior

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
26 March 2026
Language
EN
Document type
Preprint