- Title
- A New Look and Convergence Rate of Federated Multitask Learning With Laplacian Regularization
- Creator
- Dinh, Canh T.; Vu, Tung T.; Tran, Nguyen H.; Dao, Minh N.; Zhang, Hongyu
- Relation
- IEEE Transactions on Neural Networks and Learning Systems Vol. 35, Issue 6, p. 8075-8085
- Publisher Link
- http://dx.doi.org/10.1109/TNNLS.2022.3224252
- Publisher
- Institute of Electrical and Electronics Engineers (IEEE)
- Resource Type
- journal article
- Date
- 2024
- Description
- Non-independent and identically distributed (non-IID) data distribution among clients is considered as the key factor that degrades the performance of federated learning (FL). Several approaches to handle non-IID data, such as personalized FL and federated multitask learning (FMTL), are of great interest to research communities. In this work, first, we formulate the FMTL problem using Laplacian regularization to explicitly leverage the relationships among the models of clients for multitask learning. Then, we introduce a new view of the FMTL problem, which, for the first time, shows that the formulated FMTL problem can be used for conventional FL and personalized FL. We also propose two algorithms and decentrali-zed () to solve the formulated FMTL problem in communication-centralized and decentralized schemes, respectively. Theoretically, we prove that the convergence rates of both algorithms achieve linear speedup for strongly convex and sublinear speedup of order 1/2 for nonconvex objectives. Experimentally, we show that our algorithms outperform the conventional algorithm FedAvg, FedProx, SCAFFOLD, and AFL in FL settings, MOCHA in FMTL settings, as well as pFedMe and Per-FedAvg in personalized FL settings.
- Subject
- federated learning (FL); federated multitask learning (FMTL); Laplacian regularization; personalized learning
- Identifier
- http://hdl.handle.net/1959.13/1505255
- Identifier
- uon:55654
- Identifier
- ISSN:2162-237X
- Rights
- © 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
- Language
- eng
- Full Text
- Hits: 1852
- Visitors: 1821
- Downloads: 30
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | ATTACHMENT02 | Author final version | 1 MB | Adobe Acrobat PDF | View Details Download |