Energy-Aware Decentralized Learning with Intermittent Model Training
Abstract
SkipTrain is a novel Decentralized Learning (DL) algorithm, which minimizes energy consumption in decentralized learning by strategically skipping some training rounds and substituting them with synchronization rounds. These training-silent periods, besides saving energy, also allow models to better mix and produce models with superior accuracy than typical DL algorithms. Our empirical evaluations with 256 nodes demonstrate that SkipTrain reduces energy consumption by 50% and increases model accuracy by up to 12% compared to D-PSGD, the conventional DL algorithm.
Cite
BibTeX
@inproceedings{sharma2024energy,
title={Energy-Aware Decentralized Learning with Intermittent Model Training},
author={De Vos, Martijn and Dhasade, Akash and Dini, Paolo and Guerra, Elia and Kermarrec, Anne-Marie and Miozzo, Marco and Pires, Rafael and Sharma, Rishi},
booktitle={2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)},
pages={1172--1174},
year={2024},
organization={IEEE}
}