MLego: Fast and Scalable Analytic Queries through Model Materialization and Reuse

With massive texts on social media, there is an increasing desire to leverage first-hand information about local events in texts to extract insights and identify hot topics. A straightforward approach is to leverage topic modeling techniques and return hot topics via the range given by a user. However, constructing topics from massive texts is time-consuming, making it challenging to apply in real-time query scenarios. Despite recent efforts to accelerate the model training process using distributed training techniques and rapid sampling methods, the high training expenses restrict its practical use. In this paper, we study how to efficiently conduct an analytic query in an interactive manner. We propose MLego, an analytic query framework by leveraging the concepts from data management - materialization and reuse. Our solution builds the approximate model by merging materialized models. Additionally, we optimize the workflow of MLego by leveraging the hierarchical plan searching process to reduce the query cost. The extensive experiments on different datasets show the efficiency and scalability of our method.