Zero Trust Data Engineering: A Reference Architecture for Serverless, FedRAMP-High Healthcare Pipelines

The collision of Big Data Velocity and Federal Compliance creates a significant friction point in modern healthcare informatics. As research institutions migrate petabyte-scale genomic and clinical datasets to the cloud, traditional perimeter-based security models-relying on bastion hosts and persistent clusters-have proven insufficient. These legacy architectures often violate the core tenet of Zero Trust: "never trust, always verify," and create a substantial attack surface where a single compromised credential can expose an entire data lake. This paper proposes a paradigm shift from Identity-Per-User to Identity-Per-Transaction. We present a reference architecture for a Serverless Zero-Trust Data Pipeline that utilizes ephemeral compute resources (AWS Lambda and Glue) to process Protected Health Information (PHI). By implementing a Clean Room ingestion pattern and a Just-In-Time (JIT) identity broker, we enforce permissions that are scoped strictly to the lifecycle of a single batch processing event. We further introduce a streaming de-identification layer utilizing probabilistic entity detection to tokenize PHI before it reaches persistent storage. Our evaluation within a production FedRAMP High environment demonstrates that this architecture reduces the theoretical blast radius of a compromised identity by over 99.9% while maintaining linear throughput scalability, proving that rigorous federal compliance need not come at the cost of cloud agility.