Lessons Learned from the M365 AI Copilot Trial

AI copilots such as Microsoft 365 Copilot are increasingly being adopted in workplaces, but real-world evaluations of their impact remain limited. As part of a government-wide initiative, Australia’s national science agency, Commonwealth Scientific and Industrial Research Organisation, conducted a six-month trial of Microsoft 365 Copilot involving 300 users across scientific and engineering research contexts. Expectations outpaced reality: before the trial ∼80% anticipated productivity gains; after real use ∼55% reported them. Pre- and post-trial surveys and interviews showed clear efficiency gains for routine, well-defined work, while complex, domain-specific tasks still required careful human review. We offer evidence-based lessons for users and organisations on using copilots effectively, integrating them into daily workflows, and applying the right safeguards.