Comparative RSM-based study of nano, hybrid, and ternary nanofluid heat transfer with Cattaneo–Christov flux in porous media

Abstract This study conducts a comparative analysis of nano, hybrid, and ternary nanofluids in an Oldroyd-B fluid model within a Darcy–Forchheimer porous medium, focusing on heat transfer efficiency. It examines the effects of thermal radiation, heat source/sink, viscous dissipation, relaxation, and retardation time under different thermophysical conditions. Nanofluids are widely used in medicine, electronic cooling, heat exchangers, and renewable energy systems. However, the comparative performance of nano, hybrid, and ternary nanofluids in an Oldroyd-B framework remains unexplored. This study fills this gap by evaluating TiO 2 , Fe 3 O 4 , and CoFe 3 O 4 nanoparticles and optimizing their thermal efficiency using numerical and statistical modeling. The governing nonlinear partial differential equations are transformed into ordinary differential equations using similarity variables and solved with the BVP4C method. Response surface methodology (RSM) is applied for parameter optimization, while multilinear regression analysis provides further insights. Graphical and tabulated results show that nano nanofluids achieve the highest heat transfer, followed by hybrid, while ternary nanofluids perform the lowest due to increased resistance. Thermal radiation improves heat transfer, whereas the Forchheimer number reduces it. The obtained findings can guide the selection of nanofluids for high-performance heat exchangers, magnetohydrodynamic (MHD) cooling systems, solar-thermal collectors, nuclear and industrial cooling, and biomedical thermal regulation technologies. The integration of RSM further provides a practical optimization framework for engineering design and performance improvement in energy-efficient systems. These findings offer valuable insights into optimizing nanofluid-based thermal systems, aiding advancements in heat exchangers, MHD cooling, and energy-efficient technologies.