传统的对比学习是data-level的,本文改进了FedAvg的本地模型训练阶段,提出了model-level的联邦对比学习(Model-Contrastive Federated Learning) 作者从NT-Xent loss中获得灵感,提出了model-contrastive loss。model-contrastive loss可以从两方面影响本地模型 1. 本地模型能够学到接近于全局模型的representation 2. 本地模型...
2.1 Federated Learning 现有处理non-IID问题的方法分为提升本地训练效果和提升聚合效果,本文是前一种,因为对比损失函数在本地计算 其他研究方向有个性化联邦学习,尝试为每个客户端学习一个模型,本文属于经典联邦学习,学习单个全局最佳模型 2.2 Contrastive Learning 对比学习的核心原理是拉近相同样本的表征让模型学习到更好...
请注意,FedProx 还有一个超参数 µ 来控制其近端项的权重(即 LF edP rox = ‘F edAvg + µ‘prox)。 对于 FedProx,我们从 f0:001 调整 µ; 0:01; 0:1; 1g(范围也在之前的论文[28]中使用)并报告最佳结果。 用于 CIFAR-10、CIFAR-100 和 Tiny-Imagenet 的 FedProx 的最佳 µ 分别为 0:01...
model-contrastive federated learning 对比联邦学习(Contrastive Federated Learning)是基于联邦学习(Federated Learning)的一种机器学习技术,它受到了联合训练(Joint Training)的启发,通过在客户机训练数据集上进行联合训练,在客户机和服务器端应用搭配特征(Contrasting Features)以及对抗损失(Contrastive Loss)以提高模型的精度...
model-contrastive federated learning解决思路 Model-Contrastive FederatedLearning解决思路是将用户在本地模型训练的参数作为主要特征,利用相典技术实现“跨用户”的参数更新训练。在每轮训练中,系统将所有的用户的模型参数从位置服务器拉取,然后计算出该模型的输入和输出,以及其在模型参数训练表中计算出的模型变化。之后...
In this paper, we propose MOON: model-contrastive federated learning. MOON is a simple and effective federated learning framework. The key idea of MOON is to utilize the similarity between model representations to correct the local training of individual parties, i.e., conducting contrastive ...
(MOON)in an edge-cloud environment,named FRM.Specifically,based on federated partial averaging,MOON warmup is added to correct the local training of individual edge servers and improve the model's effectiveness by calculating and back-propagating a model-contrastive loss,which represents the ...
Key words: federated learning, data heterogeneity, model heterogeneity, communication overhead, intellectual property (IP) protection Take away: We enhance the typical HtFL method FedProto with Trainable Global Prototypes (TGP) and Adaptive-margin-enhanced Contrastive Learning (ACL), making it more ver...
CVPR 2019之迁移学习解读(三)Contrastive Adaptation Network for Unsupervised Domain Adaptation Contrastive Adaptation Network for Unsupervised Domain Adaptation 简述: 无监督域自适应(UDA)对目标域数据进行预处理,而手工注释只在源域可用。以往的方法在忽略类信息的情况下,会使域间的差异最小化,从而导致不一致和泛化...
3. model-contrastive federated learning 3.1. problem statement N parties:P_1,...,P_N, partyP_ihas a local datasetD^i.goal is to learn a machine learning modelwover the datasetD(which is the combination of all the local datasets) with the help of a central sever, while the raw data...