Side-channel Inference of User Activities in AR/VR Using GPU Profiling

The myriad of sensors and powerful processors in AR/VR devices enable users to become deeply immersed in digital content, whether in fully virtual environments or blended physical-digital scenes.However, recent research has shown that these capabilities also introduce novel side-channel vulnerabilities in these devices.These works include virtual keystroke inference using WiFi signals [5] or embedded 2D infrared sensors [63], as well as avatar typing observation [96], head movement analysis [78], or a combination of computer vision methods and motion sensors [41].Additionally, other studies have explored the inference of visual and audio activities during device charging [40], and the reconstruction of high-quality vital signals and speech content, employing embedded motion sensors in AR/VR headsets [11], [97].A recent work [98] proposed side-channel attacks that recover gestures, voice commands, keystrokes, and detect bystanders using a concurrent malicious app on AR/VR devices.These attacks exploit memory allocation APIs and performance counters exposed through SDKs provided by game engines such as Unity and Unreal.Although the aforementioned work [98] uses CPU and GPU frame rates obtained via SDKs as attack vectors, it faces three key limitations.First, it requires a concurrently running standalone background app to access these performance counters.However, devices like the Meta Quest restrict concurrent app execution, apart from a few Meta-approved apps (e.g., Messenger and Meta Chat), rendering the attack infeasible.Second, their approach relies on high-resolution profiling (60 Hz), which makes it detectable and easily prevented by lowering the resolution of the profiling tool.Lastly, their work focuses on a small set of attack attributes and performance counters without modeling the detailed relationship between 3D object rendering and GPU usage.Hence, limited sensitive data, such as built-in voice commands and simple gestures, can be captured.In this paper, we revisit the threat landscape of GPU-based side-channel attacks in immersive environments by focusing on Meta devices, which expose GPU metrics with unusually high and fine-grained correlation to application behavior.We demonstrate that even low-resolution (1Hz) profiling of these metrics can reveal fine-grained information and leak sensitive application activity.To this end, we introduce OVRWATCHER which leverages the built-in GPU profiling tool [16] to fingerprint user activity with over 98% accuracy, without relying on concurrent app execution or additional SDKs.We show that even profiling a single GPU metric is sufficient to leak Abstract-Over the past decade, AR/VR devices have drastically changed how we interact with the digital world.Users often share sensitive information, such as their location, browsing history, and even financial data, within third-party apps installed on these devices, assuming a secure environment protected from malicious actors.Recent research has revealed that malicious apps can exploit such capabilities and monitor benign apps to track user activities, leveraging fine-grained profiling tools, such as performance counter APIs.However, app-to-app monitoring is not feasible on all AR/VR devices (e.g., Meta Quest), as a concurrent standalone app execution is disabled.In this paper, we present OVRWATCHER, a novel side-channel primitive for AR/VR devices that infers user activities by monitoring low-resolution (1Hz) GPU usage via a background script, unlike prior work that relies on high-resolution profiling.OVRWATCHER captures correlations between GPU metrics and 3D object interactions under varying speeds, distances, and rendering scenarios, without requiring concurrent app execution, access to application data, or additional SDK installations.We demonstrate the efficacy o f OVRWATCHER in fingerprinting both standalone AR/VR and WebXR applications.OVRWATCHER also distinguishes virtual objects, such as products in immersive shopping apps selected by real users and the number of participants in virtual meetings, thereby revealing users' product preferences and potentially exposing confidential information from those meetings.OVRWATCHER achieves over 99% accuracy in app fingerprinting a nd o ver 9 8% a ccuracy in object-level inference.