When Hands Meet Physics in Virtual Reality: Effects of Interaction Fidelity on User Experience

Physics governs everyday interaction, yet in Virtual Reality (VR) the fidelity of such interactions can diverge from reality. We investigate how Physical Fidelity (virtual object behavior) and Action Fidelity (virtual hand behavior) of physics-driven interaction shape user experience. In a within-subject study (n = 34), participants performed gamified tasks under three conditions: No-Physics (lower Physical and Action Fidelity), Object-Physics (higher Physical, lower Action Fidelity), and Full-Physics (higher Physical and Action Fidelity). Results show that higher Physical Fidelity reduces task efficiency and increases overall workload, with the No-Physics condition outperforming the others in these metrics. When combined with higher Action Fidelity, although efficiency gets even worse in some cases, the Full-Physics condition enhances body ownership and interaction quality. The hybrid Object-Physics condition consistently ranks lowest across all qualitative measures. Interpreting these results through the Interaction Fidelity Model, we offer design implications for VR applications.