Study on estimation of light reflection for AR try-on systems for accessories

This paper proposes an AR-based virtual try-on method for metallic and gemstone accessories that reproduces realistic reflections under real-world illumination. To estimate environment lighting, a small mirror hemisphere is mounted on the wrist and captured using a smartphone camera. The hemisphere image enables to detect highlight regions, estimate light source directions, colors, and intensities, and generate a high-resolution HDR environment map. The estimated lighting parameters are applied to physically based rendering of a 3D ring model for AR visualization. The method consists of two input stages: an initial input image, in which illumination is estimated from the mirror hemisphere, and subsequent input images captured under the same environment. For the subsequent inputs, only hand posture changes, allowing the ring pose and scale to be re-estimated while reusing the illumination parameters from the first image. Experimental results show that the detected highlight centroids align with those in the ground-truth hemisphere image, achieving an average angular error of about 3° across three scenes. Color and brightness consistency are evaluated using the weighted mean RGB values around each highlight centroid, suggesting that the estimated illumination parameters are sufficiently accurate for AR-based visualization. We confirmed that once illumination is estimated from the initial input image, stable AR try-on results can be achieved for subsequent inputs by estimating only the ring pose and scale, without reestimating lighting for each frame. These results suggest that a wrist-mounted mirror hemisphere is an effective light probe for realistic AR jewelry visualization under everyday lighting conditions.