Comparative Federated Algorithms for Solving Non- IID Data Challenges

Authors

  • Alireza Asl Nemati * Department of Computer Engineering, Imam Khomeini International University, Qazvin, Iran.
  • Mohammad Hassan Sadreddini Department of Computer Engineering, Imam Khomeini International University, Qazvin, Iran.
  • Mohammad Mahdizade Department of Computer Engineering, Imam Khomeini International University, Qazvin, Iran.

https://doi.org/10.48314/tsc.v1i1.35

Abstract

This study compares three Federated Learning (FL) algorithms—FedAvg, FedProx, and MOON—by evaluating their performance in both IID and non-IID settings. We found that FedAvg performs best in IID scenarios, offering quick convergence and high accuracy. However, in non-IID settings, MOON stood out as the top performer, thanks to its contrastive learning method, providing better stability and accuracy across heterogeneous data. FedProx showed improvements over FedAvg in handling non-IID data but was less effective than MOON. Our findings suggest that for environments with IID data, FedAvg is ideal, while MOON is more suitable for non-IID cases. We also highlight the need for further research into personalized FL, regularization techniques, and the integration of multimodal data.

Keywords:

Federated learning, FedAvg, FedProx, MOON, Data heterogeneity

References

  1. [1] AbdulRahman, S., Tout, H., Ould-Slimane, H., Mourad, A., Talhi, C., & Guizani, M. (2020). A survey on federated learning: The journey from centralized to distributed on-site learning and beyond. IEEE internet of things journal, 8(7), 5476–5497. https://doi.org/10.1109/JIOT.2020.3030072
  2. [2] Huang, C., Huang, J., & Liu, X. (2022). Cross-silo federated learning: Challenges and opportunities. https://arxiv.org/abs/2206.12949
  3. [3] Li, Q., Diao, Y., Chen, Q., & He, B. (2022). Federated learning on non-iid data silos: An experimental study. 2022 IEEE 38th international conference on data engineering (ICDE) (pp. 965–978). IEEE. https://doi.org/10.1109/ICDE53745.2022.00077
  4. [4] Gao, D., Yao, X., & Yang, Q. (2022). A survey on heterogeneous federated learning. https://arxiv.org/abs/2210.04505
  5. [5] Che, L., Wang, J., Zhou, Y., & Ma, F. (2023). Multimodal federated learning: A survey. Sensors, 23(15), 6986. https://doi.org/10.3390/s23156986
  6. [6] Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2020). Federated optimization in heterogeneous networks. Proceedings of machine learning and Systems (Vol. 2, pp. 429–450). arXiv https://doi.org/10.48550/arXiv.1812.06127
  7. [7] McMahan, B., Moore, E., Ramage, D., Hampson, S., & Y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. Artificial Intelligence and Statistics (pp. 1273–1282). PMLR. https://proceedings.mlr.press/v54/mcmahan17a?ref=https://githubhelp.com
  8. [8] Li, Q., He, B., & Song, D. (2021). Model-contrastive federated learning. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10713–10722). IEEE. https://B2n.ir/uf2079
  9. [9] McMahan, H. B., Moore, E., Ramage, D., & Y Arcas, B. A. (2016). Federated learning of deep networks using model averaging. https://arxiv.org/pdf/1602.05629v1/1000
  10. [10] Yurdem, B., Kuzlu, M., Gullu, M. K., Catak, F. O., & Tabassum, M. (2024). Federated learning: Overview, strategies, applications, tools, and future directions. Heliyon, 10(19), e38137. https://doi.org/10.1016/j.heliyon.2024.e38137
  11. [11] Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018). Federated learning with non-iid data. https://arxiv.org/abs/1806.00582

Downloads

Published

2025-02-15

Issue

Vol. 1 No. 1 (2025): Transactions on Soft Computing (TSC)

Section

Articles

How to Cite

Comparative Federated Algorithms for Solving Non- IID Data Challenges. (2025). Transactions on Soft Computing , 1(1), 27-35. https://doi.org/10.48314/tsc.v1i1.35