An Improved Method for fMRI-based Human Visual Reconstruction Using Diffusion Models

We introduce a plug-and-play framework for reconstructing natural images from functional Magnetic Resonance Imaging (fMRI) using a frozen diffusion prior. The approach decouples appearance and semantics: early-visual activity is linearly decoded to a Variational Autoencoder (VAE) latent that preserves structure, while higher-level activity is decoded to a text embedding that guides content. A lightweight diagonal ridge calibration aligns predicted semantics with the model’s conditioning space. Reconstruction runs in image-to-image mode by stochastically encoding from the predicted appearance latent and denoising with U-Net conditioned on the calibrated semantics—without fine-tuning the generative model. On Natural Scenes Dataset (NSD), the method improves semantic alignment and perceptual fidelity over strong baselines, offering a simple and reproducible pathway to brain-guided image synthesis.