Towards Stable Training of Complex-Valued Physics-Informed Neural Networks: A Holomorphic Initialization Approach

This work introduces a new initialization scheme for complex-valued layers in physics-informed neural networks that use holomorphic activation functions. The proposed method is derived empirically by estimating the activation and gradient gains specific to complex-valued tanh and sigmoid functions through Monte Carlo simulations. These estimates are then used to formulate variance-preserving initialization rules. The effectiveness of these formulas is evaluated on several second-order complex-valued ordinary differential equations derived from the Helmholtz equation, a fundamental model in wave theory and theoretical physics. Comparative experiments show that complex-valued neural solvers initialized with the proposed method outperform traditional real-valued physics-informed neural networks in terms of both accuracy and training dynamics.