2.1 Federated Learning 现有处理non-IID问题的方法分为提升本地训练效果和提升聚合效果,本文是前一种,因为对比损失函数在本地计算 其他研究方向有个性化联邦学习,尝试为每个客户端学习一个模型,本文属于经典联邦学习,学习单个全局最佳模型 2.2 Contrastive Learning 对比学习的核心原理是拉近相同样本的表征让模
今天要介绍的就是其中一篇论文《Model-Contrastive Federated Learning》 一、Motivation 联邦学习的关键挑战是客户端之间数据的异质性(Non-IID),尽管已有很多方法(例如FedProx,SCAFFOLD)来解决这个问题,但是他们在图像数据集上的效果欠佳(见实验Table1)。 传统的对比学习是data-level的,本文改进了FedAvg的本地模型训练阶段...
We study the problem of learning knowledge graph embeddings for a set of federated knowledge graphs, where their raw triples are not allowed to be collected together. We propose a federated learning framework FedEC. In our framework, a local training procedure is responsible for learning knowledge...
【摘要】 摘要 联邦学习使多方能够在不交流本地数据的情况下协作训练机器学习模型。 联邦学习的一个关键挑战是处理各方本地数据分布的异质性。 尽管已经提出了许多研究来应对这一挑战,但我们发现它们无法在具有深度学习模型... 摘要 联邦学习使多方能够在不交流本地数据的情况下协作训练机器学习模型。 联邦学习的一个...
论文信息 论文标题:Federated Learning from Pre-Trained Models: A Contrastive Learning Approach论文作者:Yue Tan, Guodong Long, Jie Ma, Lu Liu, Tianyi Zhou, Jing Jia
model-contrastive federated learning 对比联邦学习(Contrastive Federated Learning)是基于联邦学习(Federated Learning)的一种机器学习技术,它受到了联合训练(Joint Training)的启发,通过在客户机训练数据集上进行联合训练,在客户机和服务器端应用搭配特征(Contrasting Features)以及对抗损失(Contrastive Loss)以提高模型的精度...
However, federated learning suffers from a nonindependent homogeneous distribution of user data, which can lead to a user-side learned job representation with its own bias. Therefore, we introduce the idea of contrastive learning to guide user-side training to alleviate user-side bias and improve ...
《Communication-Efficient Learning of Deep Networks from Decentralized Data》 Federated Averaging Algorithm(FedAVG) 联邦优化(Federated Optimization) 联邦优化:隐含在联邦学习中的优化问题 关键属性: 1、用户数据非独立同分布 2、用户数据量不平衡 3、用户分布规模非常大 ...
Federated learning allows multiple clients to collaborate to train high-performance deep learning models while keeping the training data locally. However, when the local data of all clients are not independent and identically distributed (i.e., non-IID), it is challenging to implement this form ...
Federated learning enables multiple parties to collaboratively train a machine learning model without communicating their local data. A key challenge in federated learning is to handle the heterogeneity of local data distribution across parties. Although many studies have been proposed to address this ...