Product Truth in the Age of Agentic Commerce: A Multi-Platform Evaluation of AI Shopping System Accuracy, Completeness, and Regulatory Reliability
Abstract
AI-powered shopping assistants such as ChatGPT, Gemini, Perplexity, Claude, and Copilot are increasingly marketed as end-to-end "agentic commerce" systems, yet they depend on web-scraped content, merchant feeds, and partner integrations that may be incomplete, volatile, or misaligned with regulatory obligations and product truth. This paper introduces product truth—a SKU-level representation that is factually accurate, complete on key attributes, and aligned with relevant regulatory regimes—and evaluates whether leading AI shopping platforms can reliably deliver it. We test five major platforms on 100 product scenarios spanning ten categories: five regulation-sensitive (cosmetics, cleaning, batteries, aerosols, paints) and five lower-regulation controls (apparel, home, food, kitchen, books). Across 2,500 evaluated steps using a chained five-step protocol, full success is achieved in 28.2% of cases, with 36.5% partial and 35.2% failure. Notably, attribute completeness is substantially weaker in regulated categories (μ = 0.46/2) than controls (μ = 1.18/2), and availability lookup fails in 52.7% of cases. Platform performance varies dramatically—from ChatGPT (43.4% success) to Perplexity (13.6%)—with checkout feasibility ranging from functional (Gemini: 0.78/2) to almost non-existent (Claude: 0.02/2). We argue that while AI shopping systems have improved, persistent gaps in availability verification, regulated-product completeness, and checkout execution mean they cannot yet reliably serve as stewards of product truth.
Fields of Study & Subjects
AI in Service Interactions
Ethics and Social Impacts of AI
Information Retrieval and Search Behavior
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 28 January 2026