CategoRisk Supplementary Materials

Read full text Summarise with VIRAT
Authors: Nimrod Mike
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This repository contains the supplementary materials for the paper "Socrates Asks, Themis Decides: Conversational Elicitation and Structured Legal Reasoning for EU AI Act Risk Classification" (Nimrod Mike, Corvinus University of Budapest). Contents include: (1) a 50-scenario benchmark for EU AI Act risk classification with expert-validated ground truths, difficulty ratings, legal basis documentation, and contested-point reasoning for ambiguous cases; (2) verbatim system prompts for both the Socrates elicitation agent and the Themis classification agent under both evaluation conditions (baseline unstructured prompting and IRAC+CoR structured prompting ablation); (3) full evaluation results for both conditions in JSON format; and (4) the Node.js evaluation harness scripts used to conduct LLM-as-user simulation testing. The benchmark evaluates a hybrid dual-agent LLM architecture against a 50-scenario ground-truth dataset, achieving 76% overall accuracy (Cohen's κ = 0.684) with a documented systematic safety-conservative over-classification bias concentrated in the minimally-defined minimal risk tier of the EU AI Act.

Fields of Study & Subjects

Artificial Intelligence in Law Ethics and Social Impacts of AI Digitalization Law and Regulation

Publication details

Publication date
1 January 2026
Language
EN
Document type
Preprint