A KKT linearized cross-market coordinated trading optimization model in the context of big data

In the context of the deep integration of big data and artificial intelligence, the explosive growth of multisource heterogeneous data has brought new opportunities and challenges to the collaborative optimization of power and carbon markets. Addressing the complex structural and nonlinear constraint challenges in optimization problems driven by massive market data, this study innovatively proposes a Karush-Kuhn-Tucker (KKT) condition linearization collaborative trading model based on big data analysis. By deeply mining the spatiotemporal correlation features in power spot market transaction logs and carbon quota verification data, a two-layer optimization architecture incorporating dynamic carbon emission factors has been constructed. This method employs a distributed computing framework to process terabyte-scale market data streams, introduces adaptive relaxation factors and penalty coefficients to achieve intelligent relaxation of constraint conditions, and transforms the original model's difficult-to-solve problem into a parallelizable mixed-integer linear programming problem. Tests based on historical data from the IEEE-30 and Guangdong carbon trading systems show that the model significantly outperforms traditional methods in terms of accuracy (94.7%), recall rate (92.5%), and F1 score (93.6%). Additionally, GPU acceleration reduces the solution time to under 15 seconds, improving computational efficiency by 46% compared to the Stackelberg Game-Based Optimization Model (SGO). The research findings provide a scalable intelligent optimization paradigm for real-time collaborative decision-making across multiple markets in a big data environment. The "data-model-decision" closed-loop framework it constructs has been successfully applied to the electricity-carbon joint clearing system.