Taming Multidimensional Analytics Costs: An Empirical Study of HLL and KLL Sketches in Cloud Data Warehouses
Abstract
Analytics dashboards that track unique users and percentiles across multiple dimensions create a costly problem for data teams: each query requires scanning the entire dataset, leading to spiraling costs and slow performance. We present an empirical study of a novel architectural pattern that stores compact probabilistic summaries (HyperLogLog (HLL) and Karnin-Lang-Liberty (KLL) sketches) once, then merges them to answer any dimensional query without rescanning data. Testing six configurations on Stack Overflow data (12.46 GB), we achieve 93-96% reduction in data scanning with less than 1% error. Our data-driven guidelines recommend HLL precision 20 and KLL precision 10,000 for production deployments. This approach transforms expensive multi-dimensional analytics into cost-effective operations suitable for real-time dashboards across cloud data warehouses including BigQuery, Snowflake, Redshift, and Presto.
Fields of Study & Subjects
Advanced Database Systems and Queries
Big Data and Business Intelligence
Cloud Computing and Resource Management
Publication details
- Journal / source
- IEEE Software
- Volume
- 43
- Issue
- 3
- Pages
- 53–61
- Publication date
- 3 February 2026