Resilient Distributed Collaborative Machine Learning

Abstract

The rapid proliferation of distributed data and improved computing has spurred the development of learning algorithms that can operate collaboratively across multiple entities while preserving privacy and provides scalability with faster learning. However, distributed learning systems are inherently vulnerable to heterogeneity, unreliable communication, and adversarial behaviors that can compromise their reliability. This dissertation presents a unified framework for designing resilient distributed machine learning algorithms that ensure robust and trustworthy learning across a variety of collaborative paradigms. The research spans four major directions: federated learning, peer-to-peer (P2P) learning, graph machine learning (GML), and multi-agent reinforcement learning (MARL). In the federated learning setting, new aggregation techniques, based on trimming, clipping, and adaptive weighting, are developed to defend against Byzantine or malicious participants while maintaining model accuracy. The P2P learning framework removes the need for central coordination by enabling decentralized, adaptive aggregation among agents through peer-to-peer communication. This idea is extended to P2P Graph Machine Learning, where multiple agents collaboratively learn on graph-structured data by sharing structural representations while keeping node features private, achieving improved robustness under heterogeneous and cross-domain conditions. Finally, the principles of resilient aggregation and adaptive collaboration are applied to MARL, enabling multiple reinforcement learning agents to share policies across distinct environments for enhanced generalization and stability. Across all paradigms, the proposed methods are analyzed theoretically and validated empirically on diverse benchmark datasets, including vision, text, and graph-structured data. The results demonstrate that adaptive and resilient aggregation substantially improves resilience, convergence and improved accuracy in the presence of data heterogeneity and adversarial perturbations. Overall, this work advances the state of distributed collaborative intelligence by providing a principled foundation for secure, scalable, and adaptive learning in networked environments.