Outage–Throughput Performance of RSMA‐Enabled GEO Multibeam Satellite–Terrestrial Downlink Systems

ABSTRACT This paper presents an analytical framework for evaluating the outage–throughput performance of rate‐splitting multiple access (RSMA) in GEO multibeam satellite–terrestrial downlink systems operating over shadowed–Rician fading channels. Closed‐form expressions are derived for the outage probability and system throughput under the baseline assumption of perfect successive interference cancellation (SIC), accompanied by asymptotic analysis revealing the diversity gain achieved with multiple satellite antennas. The proposed model is developed for a general ‐user GEO multibeam downlink and specializes to the two‐user configuration for numerical illustration, while a separate discussion is provided on how the outage, throughput, and computational complexity scale with . In addition, an explicit design for the common message precoding vector is specified as a weighted combination of user channels, enabling a quantitative assessment of its impact on link reliability and spectral efficiency. To better match practical satellite receivers, an imperfect SIC model is introduced, where a residual interference factor captures decoding errors and leakage, and its effect on outage probability, throughput, and robustness is analyzed. A simple yet effective power allocation strategy is also discussed: We formulate a throughput‐maximization problem with outage constraints and then motivate the adopted power‐splitting coefficients through offline search and sensitivity analysis. Monte Carlo simulations validate the analytical results and demonstrate that RSMA achieves significant gains over NOMA, especially under moderate and heavy shadowing conditions.