Adaptive Optimization in Dynamic Machine Learning Systems: From Federated Networks to Multi-Modal Retrieval

Modern distributed and augmented artificial intelligence systems face critical challenges when deployed in dynamic, real-world environments characterized by shifting data distributions, fluctuating network topologies, and complex reasoning requirements. This thesis investigates methodologies to enhance the robustness, efficiency, and adaptability of these systems across three distinct projects. The first project addresses resource and temporal dynamics in wireless edge networks by proposing a Dynamic Multi-Model Asynchronous Federated Learning (DMA-FL) framework, which optimizes device scheduling and resource allocation to balance multi-task machine learning performance with energy consumption. The second project tackles spatial and population dynamics within federated learning by developing a novel model initialization algorithm that accelerates convergence when devices dynamically join or leave the network, successfully mitigating the effects of distribution shifts. Finally, the third project addresses utility misalignment in Multi-Modal Retrieval-Augmented Generation (MM-RAG) systems by introducing a Generator-in-the-Loop Alignment Framework. This approach fine-tunes multimodal retrievers to prioritize reasoning utility over semantic similarity, effectively bridging the semantic-utility gap. Collectively, this research provides theoretical foundations and practical algorithms to optimize distributed model training and augmented reasoning, ensuring high performance and resource efficiency in dynamic AI deployments.