Row Pattern Recognition for Pandas: Bringing MATCH_RECOGNIZE to Python DataFrames

View at publisher
Authors: Monier Lawande, Mohamed Elhelw, Ahmed Awad
Cited by: 0
References: 11

Abstract

Row pattern recognition (RPR) enables data scientists to detect complex sequences such as trends, repetitions, and the absence of events. Modern SQL engines such as Oracle, Snowflake, Flink, and Trino implement RPR through the MATCH_RECOGNIZE clause that allows users to define patterns in regular expressions over ordered rows in a table. However, Python users widely use pandas DataFrame library that lacks native support for MATCH_RECOGNIZE. This gap forces data scientists to construct complex, error-prone pipelines using multiple groupby, window, and join operations if they want to process patterns within Python/ pandas workflows. We provide an SQL-in-Pandas engine that executes match_recognize() queries directly from an SQL query on a pandas DataFrame. The system supports SQL:2016 RPR semantics, including PARTITION BY, ORDER BY, PATTERN, DEFINE, and MEASURES for offline analytics in Python. Our approach uses an NFA-based matcher to process patterns over partitioned, ordered data, supporting both one-row-per-match and all-rows-per-match output modes, and returns results as a DataFrame. Using synthetic financial datasets, we show a reduction in code size compared to native pandas, while maintaining correctness parity with established SQL engines like Trino and Oracle.

Fields of Study & Subjects

Advanced Database Systems and Queries Data Mining Algorithms and Applications Scientific Computing and Data Management

Publication details

Journal / source
IEEE Mediterranean Electrotechnical Conference (MELECON)
Pages
1–6
Publication date
2 February 2026
Language
EN