Towards a secure on-prem AI lab: Adventures with Apple Silicon

At PHF Science, we handle sensitive information across multiple domains, which means commercially hosted AI services aren’t always an option. As such, we’ve been exploring ways to securely host large language models (LLMs) on-prem. While we have access to GPUs on premises via our high-performance compute cluster, we wanted to see whether Apple Silicon - particularly with its large, unified memory - could offer a viable alternative at a fraction of the cost of equivalent GPU configurations.This talk will share how we built an AI lab using Apple M3 hardware, Ollama for local model hosting in our server room, OpenWebUI integrated with Entra ID for secure staff authentication, and VSCode integration via the Continue plugin. So far, we’ve successfully run models from IBM’s Granite (3.2B) through to Qwen, DeepSeek, and Moonshot variants exceeding 1 trillion parameters. The goal is to provide a secure, flexible option for all staff - not just one team - enabling experimentation and everyday research tasks while maintaining data security and sovereignty.We will share our setup, lessons from deployment and pen testing, and why we believe on-shore AI infrastructure matters—not just for PHF Science, but for Aotearoa. From resilience to climate, to reducing dependence on offshore data centres, and creating opportunities for local capability, this approach feels promising. We’re excited to discuss what “AI readiness” could look like when balancing capability, cost, and trust.ABOUT THE AUTHORRichard Dean is a senior data scientist at the New Zealand Institute of Public Health and Forensic Science – a New Zealand Government-owned research organisation dedicated to the health, wellbeing and safety of our communities. His role explores practical applications of emerging technologies for research and decision making. With a background in computer science and over 20 years’ experience working with Public Health data teams in the UK and NZ. He is particularly interested in making advanced tools accessible while respecting data sovereignty and institutional requirements, and believes that honest conversations about technology experiments - including failures - are crucial for building resilient research infrastructure in Aotearoa. -----For more information about the eResearch NZ / eRangahau Aotearoa conference, visit: https://eresearchnz.co.nz/