Behavior Discovery and Attribution for Explainable Reinforcement Learning

Read full text Summarise with VIRAT
Authors: Rishav Rishav
Cited by: 0
References: 69

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Building trust in reinforcement learning (RL) agents requires understanding why they make certain decisions, especially in high-stakes applications like robotics, healthcare, and finance. As RL systems are increasingly deployed in real-world settings, the needfor interpretable explanations of agent behavior becomes critical for debugging, verification, and human oversight. Existing explainability methods often focus on single states or entire trajectories, either providing only local, step-wise insights or attributing decisions to coarse, episode-level summaries. Both approaches miss the recurring strategies and temporally extended patterns that actually drive agent behavior across multiple decisions. State-level explanations fail to capture the broader context of why an agent follows a particular strategy, while trajectory-level explanations cannot pinpoint which specific behavioral pattern is responsible for a given action. This thesis addresses this gap by proposing a fully offline, reward-free framework for behavior discovery and segmentation, enabling the attribution of actions to meaningful and interpretable behavior segments that capture recurring patterns appearing across multiple trajectories. The approach consists of three stages: (1) a transformer-based Vector Quantized Variational Autoencoder (VQ-VAE) that discretizes state-action sequences into a learned codebook of latent representations, (2) a graph-based spectral clustering algorithm that segments these discrete codes into coherent behavior clusters based on both transition frequency and latent similarity, and (3) behavior cloning models trained on each cluster that enable fine-grained attribution of individual actions to discovered behaviors. I evaluate the framework on four diverse offline RL environments spanning continuous control (HalfCheetah, Pen manipulation), discrete navigation (MiniGrid), and visual domains (Seaquest). Quantitative evaluations demonstrate that the approach discovers semantically meaningful behaviors and outperforms trajectory-level baselines in Average Fidelity Score, measuring how well attributed behaviors predict agent actions. A human study with 15 participants shows that users prefer behavior-based explanations over trajectory retrieval methods and can consistently interpret the discovered clusters. Ablation studies validate the importance of the hybrid graph weighting scheme and the choice of codebook size. The code is publicly available.

Fields of Study & Subjects

Explainable Artificial Intelligence (XAI) Reinforcement Learning in Robotics AI-based Problem Solving and Planning

Publication details

Publication date
20 March 2026
Language
EN
Document type
Thesis