From Benchmarks to Business Impact: Deploying IBM Generalist Agent in Enterprise Production

Agents are rapidly advancing in automating digital work, but enterprises face a harder challenge: moving beyond prototypes to deployed systems that deliver measurable business value. This path is complicated by fragmented frameworks, slow development, and the absence of standardized evaluation practices. Generalist agents have emerged as a promising direction, excelling on academic benchmarks and offering flexibility across task types, applications, and modalities. Yet, evidence of their deployment in production enterprise settings is still limited. This paper reports IBM’s experience developing and deploying the \textbf{Computer Using Generalist Agent (CUGA)}. CUGA adopts a hierarchical planner--executor architecture with strong analytical foundations, achieving state-of-the-art performance on AppWorld and WebArena. Beyond benchmarks, it was deployed in the Business-Process-Outsourcing talent acquisition domain, meeting enterprise requirements for scalability, auditability, safety, and governance. To support evaluation, we introduce \textbf{BPO-TA}, a 26-task benchmark spanning 13 analytics endpoints. In deployment, CUGA matched the accuracy of specialized agents while reducing development time by 91.9\% and cost by 52.3\%. Our contribution is twofold: demonstrating generalist agents operating at enterprise scale, and distilling technical and organizational lessons from this deployment. We outline requirements and next steps for advancing research-grade architectures like CUGA into robust, enterprise-ready systems.