Row Pattern Recognition for Pandas: Bringing MATCH_RECOGNIZE to Python DataFrames
Abstract
Row pattern recognition (RPR) enables data scientists to detect complex sequences such as trends, repetitions, and the absence of events. Modern SQL engines such as Oracle, Snowflake, Flink, and Trino implement RPR through the MATCH_RECOGNIZE clause that allows users to define patterns in regular expressions over ordered rows in a table. However, Python users widely use pandas DataFrame library that lacks native support for MATCH_RECOGNIZE. This gap forces data scientists to construct complex, error-prone pipelines using multiple groupby, window, and join operations if they want to process patterns within Python/ pandas workflows. We provide an SQL-in-Pandas engine that executes match_recognize() queries directly from an SQL query on a pandas DataFrame. The system supports SQL:2016 RPR semantics, including PARTITION BY, ORDER BY, PATTERN, DEFINE, and MEASURES for offline analytics in Python. Our approach uses an NFA-based matcher to process patterns over partitioned, ordered data, supporting both one-row-per-match and all-rows-per-match output modes, and returns results as a DataFrame. Using synthetic financial datasets, we show a reduction in code size compared to native pandas, while maintaining correctness parity with established SQL engines like Trino and Oracle.
Fields of Study & Subjects
Advanced Database Systems and Queries
Data Mining Algorithms and Applications
Scientific Computing and Data Management
Publication details
- Journal / source
- IEEE Mediterranean Electrotechnical Conference (MELECON)
- Pages
- 1–6
- Publication date
- 2 February 2026
- Language
- EN