Federated-Learning

A collection of research papers, tutorials , blogs and Frameworks on FL.

中文版联邦学习综述： 联邦学习综述

1.Contents

1. Contents

2. ResearchPapers

   2.1 Top-tier(ICML, NeurIPS, ICLR, CVPR, IJCAI, AAAI)

   • CVPR2021
   • ICML2020
   • ICML2019
   • NeurIPS2020
   • NeurIPS2016-2019
   • AAAI2021
   • AAAI2020
   • IJCAI2021
   • IJCAI2020
   • ICLR2021
   • KDD2020

   2.2 ByResearchArea

   • Survey
   • VerticalFL
   • Non-IID

3. Tutorials

   3.1 book

   3.2 IndustryReport

   3.3 blogs

   3.4 slide

4. Frameworks

5. Company & Application

---

2.Papers

2.1Top-tier

CVPR2021

• Multi-Institutional Collaborations for Improving Deep Learning-Based Magnetic Resonance Image Reconstruction Using Federated Learning. Pengfei Guo, Puyang Wang, Jinyuan Zhou, Shanshan Jiang, Vishal M. Patel
• Model-Contrastive Federated Learning. Qinbin Li, Bingsheng He, Dawn Song
• FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space. Quande Liu, Cheng Chen, Jing Qin, Qi Dou, Pheng-Ann Heng
• Soteria: Provable Defense Against Privacy Leakage in Federated Learning From Representation Perspective. Jingwei Sun, Ang Li, Binghui Wang, Huanrui Yang, Hai Li, Yiran Chen

ICML2020

• Federated Learning with Only Positive Labels;Google Research;2020;label deficiency in multi-class classification
• SCAFFOLD: Stochastic Controlled Averaging for Federated Learning;Google Research;2020;non-iid
• FedBoost: A Communication-Efficient Algorithm for Federated Learning;NYU & Google Research;2020;communication cost
• FetchSGD: Communication-Efficient Federated Learning with Sketching;UC Berkeley;2020;communication cost
• From Local SGD to Local Fixed-Point Methods for Federated Learning;KAUST;2020;communication cost

ICML2019

• Analyzing Federated Learning through an Adversarial Lens
• Bayesian Nonparametric Federated Learning of Neural Networks
• Agnostic Federated Learning

NeurIPS2020

• Personalized Federated Learning with Moreau Envelopes
• Lower Bounds and Optimal Algorithms for Personalized Federated Learning [KAUST]
• Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach [MIT]
• Federated Principal Component Analysis [Cambridge]
• FedSplit: an algorithmic framework for fast federated optimization [Berkeley]
• Minibatch vs Local SGD for Heterogeneous Distributed Learning [Toyota]
• Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms
• Throughput-Optimal Topology Design for Cross-Silo Federated Learning
• Distributed Distillation for On-Device Learning [Stanford]
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge [USC]
• Federated Accelerated Stochastic Gradient Descent [Github] [Stanford]
• Distributionally Robust Federated Averaging
• An Efficient Framework for Clustered Federated Learning [Berkeley]
• Robust Federated Learning: The Case of Affine Distribution Shifts [MIT]
• Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization [CMU]
• Federated Bayesian Optimization via Thompson Sampling [NUS] [MIT]
• Distributed Newton Can Communicate Less and Resist Byzantine Workers [Berkeley]
• Byzantine Resilient Distributed Multi-Task Learning
• A Scalable Approach for Privacy-Preserving Collaborative Machine Learning [USC]
• Inverting Gradients - How easy is it to break privacy in federated learning?
• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
• Election Coding for Distributed Learning: Protecting SignSGD against Byzantine Attacks
• Differentially-Private Federated Linear Bandits [MIT]

NeurIPS2016-2019

• Federated Optimization: Distributed Optimization Beyond the Datacenter NIPS 2016 workshop
• Practical Secure Aggregation for Federated Learning on User-Held Data NIPS 2016 workshop
• Differentially Private Federated Learning: A Client Level Perspective NIPS 2017 Workshop
• Federated Multi-Task Learning NIPS 2017
• Deep Leakage from Gradients NIPS 2019

AAAI2021

• Federated Multi-Armed Bandits
• Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning
• Provably Secure Federated Learning against Malicious Clients
• On the Convergence of Communication-Efficient Local SGD for Federated Learning
• Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating
• Communication-Aware Collaborative Learning
• Peer Collaborative Learning for Online Knowledge Distillation
• A Communication Efficient Collaborative Learning Framework for Distributed Features
• Defending Against Backdoors in Federated Learning with Robust Learning Rate
• FLAME: Differentially Private Federated Learning in the Shuffle Model
• Toward Understanding the Influence of Individual Clients in Federated Learning
• Personalized Cross-Silo Federated Learning on Non-IID Data
• Model-sharing Games: Analyzing Federated Learning Under Voluntary Participation
• Federated Block Coordinate Descent Scheme for Learning Global and Personalized Models
• Addressing Class Imbalance in Federated Learning

AAAI2020

• Federated Learning for Vision-­‐and-­‐Language Grounding Problems;2020
• Federated Latent Dirichlet Allocation: A Local Differential Privacy Based Framework;2020
• Federated Patient Hashing;2020
• Robust Federated Learning via Collaborative Machine Teaching;2020
• Practical Federated Gradient Boosting Decision Trees;2020

IJCAI2021

• Collaborative Fairness in Federated Learning [IJCAI 2021 Workshop Best Paper]
• FPGA-Based Hardware Accelerator of Homomorphic Encryption for Efficient Federated Learning [IJCAI 2021 Workshop Best Student Paper]
• Federated Learning with Diversified Preference for Humor Recognition [IJCAI 2021 Workshop Best Application Paper]
• Heterogeneous Data-Aware Federated Learning [IJCAI 2021 Workshop]
• Privacy-Preserving Technology to Help Millions of People: Federated Prediction Model for Stroke Prevention [IJCAI 2021 Workshop]
• FedMVT: Semi-supervised Vertical Federated Learning with MultiView Training [IJCAI 2021 Workshop]
• FOCUS: Dealing with Label Quality Disparity in Federated Learning [IJCAI 2021 Workshop]
• Fed-Focal Loss for imbalanced data classification in Federated Learning [IJCAI 2021 Workshop]
• Threats to Federated Learning: A Survey [IJCAI 2021 Workshop]
• Asymmetrical Vertical Federated Learning [IJCAI 2021 Workshop]
• Federated Learning System without Model Sharing through Integration of Dimensional Reduced Data Representations [IJCAI 2021 Workshop]
• Achieving Differential Privacy in Vertically Partitioned Multiparty Learning [IJCAI 2021 Workshop]
• Privacy Threats Against Federated Matrix Factorization [IJCAI 2021 Workshop]
• TF-SProD: Time Fading based Sensitive Pattern Hiding in Progressive Data [IJCAI 2021 Workshop]

IJCAI2020

• Federated Meta-Learning for Fraudulent Credit Card Detection
• A Multi-player Game for Studying Federated Learning Incentive Schemes

ICLR2021

• Federated Learning Based on Dynamic Regularization
• Adaptive Federated Optimization
• Federated Learning via Posterior Avgeraging: A New Perspective and Practical Algorithms
• Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning
• Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning
• FedBN: Federated Learning on Non-IID Features via Local Batch Normalization
• FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning
• FedMix: Approximation of Mixup under Mean Augmented Federated Learning
• HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients
• Personalized Federated Learning with First Order Model Optimization

KDD2020

• FedFast: Going Beyond Average for Faster Training of Federated Recommender Systems
• Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data
• FedCD: Improving Performance in non-IID Federated Learning [KDD20 Workshop]
• Resource-Constrained Federated Learning with Heterogeneous Labels and Models [KDD2020 Workshop]

2.2ByResearchArea

Survey

• Federated Machine Learning: Concept and Applications
• Federated Learning: Challenges, Methods, and Future Directions
• Advances and Open Problems in Federated Learning
• IBM Federated Learning: an Enterprise Framework White Paper V0.1
• Federated Learning White Paper V1.0
• Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• Survey of Personalization Techniques for Federated Learning. 2020-03-19
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• Threats to Federated Learning: A Survey
• An Introduction to Communication Efficient Edge Machine Learning
• Federated Learning for Wireless Communications: Motivation, Opportunities and Challenges
• A Review of Applications in Federated Learning
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• Federated Learning for 6G Communications: Challenges, Methods, and Future Directions
• Federated Learning for Wireless Communications: Motivation, Opportunities and Challenges
• 联邦学习算法综述，王健宗，孔令炜

VerticalFL

• "Communication-Efficient Multimodal Split Learning for mmWave Received Power Prediction,"Yusuke Koda, Jihong Park, Mehdi Bennis, Koji Yamamoto, Takayuki Nishio, and Masahiro Morikura, Mar. 2020.
• "Federated Hierarchical Hybrid Networks for Clickbait Detection,"Feng Liao, Hankz Hankui Zhuo, Xiaoling Huang, and Yu Zhang, Jun. 2019.
• "Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption,"Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Richard Nock, Giorgio Patrini, Guillaume Smith, and Brian Thorne, Nov. 2017.
• "FDML: A Collaborative Machine Learning Framework for Distributed Features," Yaochen Hu, Di Niu, Jianming Yang, and Shengping Zhou,May 2019.
• "Learning Privately over Distributed Features: An ADMM Sharing Approach,"Yaochen Hu, Peng Liu, Linglong Kong, and Di Niu, Jul. 2019.
• "SplitFed: When Federated Learning Meets Split Learning,"Chandra Thapa, M. A. P. Chamikara, and Seyit Camtepe, Apr. 2020.
• "Privacy Enhanced Multimodal Neural Representations for Emotion Recognition,"Mimansa Jaiswal, and Emily Mower Provost, Oct. 2019.
• "Privacy-preserving Neural Representations of Text,"Maximin Coavoux, Shashi Narayan, and Shay B. Cohen, Aug. 2018.
• "PrivyNet: A Flexible Framework for Privacy-Preserving Deep Neural Network Training,"Meng Li, Liangzhen Lai, Naveen Suda, Vikas Chandra, and David Z. Pan, Sep. 2017.
• "One Pixel Image and RF Signal Based Split Learning for mmWave Received Power Prediction," Yusuke Koda, Jihong Park, Mehdi Bennis, Koji Yamamoto, Takayuki Nishio, and Masahiro Morikura, Nov. 2019.
• "Communication-Efficient Multimodal Split Learning for mmWave Received Power Prediction," Yusuke Koda, Jihong Park, Mehdi Bennis, Koji Yamamoto, Takayuki Nishio, and Masahiro Morikura, Mar. 2020.
• "Optimization for Large-Scale Machine Learning with Distributed Features and Observations," Alexandros Nathan, and Diego Klabjan, Apr. 2017.
• "SecureBoost: A Lossless Federated Learning Framework," Kewei Cheng, Tao Fan, Yilun Jin, Yang Liu, Tianjian Chen, Qiang Yang, Jan. 2019.
• "Privacy-Preserving Backpropagation Neural Network Learning," Tingting Chen, Sheng Zhong, Oct. 2009.
• "Split learning for health: Distributed deep learning without sharing raw patient data,"Praneeth Vepakomma, Otkrist Gupta, Tristan Swedish, and Ramesh Raskar, Dec. 2018.

Non-IID

• The Non-IID Data Quagmire of Decentralized Machine Learning;Kevin Hsieh， Amar Phanishayee， Onur Mutlu， Phillip B. Gibbons；Microsoft Research；PMLR 2020
• Federated Learning with Non-IID Data;Yue Zhao，Vikas Chanra；San Jose, CA；arXiv 2018
• FedCD: Improving Performance in non-IID Federated Learning. 2020
• Survey of Personalization Techniques for Federated Learning
• Data selection for federated learning with relevant and irrelevant data at clients, Tiffany Tuor, Shiqiang Wang, Bong Jun Ko, Changchang Liu, Kin K. Leung; arXiv 2020
• Three approaches for personalization with applications to federated learning, Yishay Mansour, Mehryar Mohri, Jae Ro, Ananda Theertha Suresh, arXiv 2020
• Life Long Learning: FedFMC: Sequential Efficient Federated Learning on Non-iid Data. 2020
• Personalized Federated Learning with Moreau Envelopes. 2020
• Proxy Experience Replay: Federated Distillation for Distributed Reinforcement Learning. 2020
• Towards Flexible Device Participation in Federated Learning for Non-IID Data. 2020
• NeurIPS 2020 submission: An Efficient Framework for Clustered Federated Learning
• FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data
• Federated learning with hierarchical clustering of local updates to improve training on non-IID data
• Federated Learning with Only Positive Labels
• Adaptive Personalized Federated Learning
• Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge based Framework
• Three Approaches for Personalization with Applications to Federated Learning
• Personalized Federated Learning: A Meta-Learning Approach
• Federated Learning with Personalization Layers
• Federated Evaluation of On-device Personalization
• Overcoming Forgetting in Federated Learning on Non-IID Data
• Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints
• Improving Federated Learning Personalization via Model Agnostic Meta Learning
• Client Adaptation improves Federated Learning with Simulated Non-IID Clients
• Adaptive gradient-based meta-learning methods
• Salvaging federated learning by local adaptation
• Federated learning of a mixture of global and local models
• Federated multi-task learning

3.Tutorials

3.1book

• 杨强，刘洋，程勇，康焱，陈天健，于涵，《联邦学习》，电子工业出版社，2020年5月
• 彭南博，王虎等. 《联邦学习技术及实战》. 电子工业出版社. 2021年3月
• 杨强，黄安埠，刘洋，陈天健等. 《联邦学习实战》. 电子工业出版社. 2021年5月
• 王建宗，李泽远，何安珣. 《深入浅出联邦学习：原理与实践》. 机械工业出版社. 2021年5月

3.2IndustryReport

• 隐私机密计算蓝皮书
• 隐私计算白皮书（2021）
• 2021隐私计算行业研究报告
• 全球数字经济白皮书
• 2021全球联邦学习研究与应用趋势报告
• 腾讯隐私计算白皮书

3.3blogs

• 杨强，刘洋，陈天健，童咏昕，“联邦学习”，中国计算机学会通讯
• 中国信息通信研究院《隐私保护计算与合规应用研究报告 (2021年)
• Federated Learning Comic
• Federated Learning: Collaborative Machine Learning without Centralized Training Data
• An Introduction to Federated Learning
• Online Comic from Google AI on Federated Learning
• GDPR, Data Shotrage and AI , AAAI-19

3.4slide

• 《隐私保护机器学习》slide
• 面向隐私安全保密的联邦学习和迁移学习
• 联邦学习的研究和应用
• 用非对称联邦保护客户隐私
• 云原生联邦学习的开源框架
• 联邦学习与安全多方计算
• FATE：联邦学习技术落地与应用

4.Frameworks

框架名称	所属公司	框架简介	github star
PySyft	OpenMined	PySyft 是开源社区OpenMined推出的一个用于安全和私有深度学习的 Python 库。 它使用联邦学习、差分隐私和加密计算来解耦私人和敏感数据，可以在主要的深度学习框架中使用， 例如 TensorFlow 和PyTorch。PySyft 代表了在深度学习程序中启用可靠的隐私模型的首批尝试之一。	7800
FATE	微众银行	微众银行AI部门研发了FATE（Federated AI Technology Enabler）联邦学习开源项目，是首个开源的联邦学习工业级框架。 目前FATE开源社区已汇聚了700多家企业、300余所高校等科研机构的开发者，是国内最大的联邦学习开源社区。 FATE项目使用多方安全计算 (MPC) 以及同态加密 (HE) 技术构建底层安全计算协议，以此支持不同种类的机器学习的安全计算，包括逻辑回归、树算法、深度学习（人工神经网络）和迁移学习等。 FATE目前支持三种类型联邦学习算法：横向联邦学习、纵向联邦学习以及迁移学习。	3700
TFF-TensorFlow Federated	Google	TensorFlow Federated project (TFF) 由谷歌公司开发和维护，是一个为联邦机器学习和其他计算方法在去中心化数据集上进行实验的开源框架。| TFF 让开发者能在自己的模型和数据上模拟实验现有的联邦学习算法，以及其他新颖的算法。TFF 提供的建造块也能够应用于去中心化数据集上，来实现非学习化的计算，例如聚合分析。 TFF 的接口有两层构成：联邦层（FL）应用程序接口（API）和联邦核心（FC）API。TFF 使得开发者能够声明和表达联邦计算，从而能够将其部署于各类运行环境。TFF 中包含的是一个单机的实验运行过程模拟器。	1700
TF-Encrypted	阿里巴巴 OpenMined	TF Encrypted是TensorFlow中用于加密机器学习的框架。 利用KerasAPI的易用性，同时通过安全多方计算和同态加密对加密数据进行训练和预测。 TF Encrypted旨在使保护隐私的机器学习变得容易，而不需要密码学、分布式系统或高性能计算方面的专业知识。	963
CrypTen	facebook	CrypTen是一个基于PyTorch的隐私保护机器学习框架。它的目标是使机器学习实践者能够使用安全的计算技术。它目前将安全多方计算作为其安全计算后端，并为ML研究人员提供三个主要好处： 1、机器学习。该框架通过一个看起来和感觉都像PyTorch张量的CryptoSensor对象来展示协议。这允许用户使用类似于PyTorch中的自动微分和神经网络模块。 2、CrypTen是基于库的。它实现了一个与PyTorch一样的张量库。这使得从业者更容易调试、实验和探索ML模型。 3、该框架的构建考虑到了现实世界的挑战。CrypTen不会缩减或过度简化安全协议的实现。	973
FedML	USC	美国南加州大学USC联合MIT、Stanford、MSU、UW-Madison、UIUC以及腾讯、微众银行等众多高校与公司联合发布了FedML联邦学习开源框架。 FedML是一个开放的研究库和基准，支持分布式训练、移动设备训练、独立仿真三种计算范例，可促进新的联合学习算法的开发和公平的性能比较。	945
Fedlearner	字节跳动	字节跳动联邦学习平台 Fedlearner 基于字节跳动在推荐和广告领域积累的机器学习建模技术和个性化推荐算法， 可以支持多类联邦学习模式，已经在电商、金融、教育等行业多个落地场景实际应用。	709
PaddleFL	百度	PaddleFL是一个基于百度飞桨（PaddlePaddle）的开源联邦学习框架。 PaddleFL提供很多联邦学习策略及其在计算机视觉、自然语言处理、推荐算法等领域的应用，例如，横向联邦学习（联邦平均、差分隐私、安全聚合）和纵向联邦学习（带privc的逻辑回归，带ABY3的神经网络）。 研究人员可以用PaddleFL复制和比较不同的联邦学习算法。	325
Rosetta	矩阵元	Rosetta 是一个基于TensorFlow开发的隐私计算框架，它将陆续集成密码学、联邦学习和可信执行环境等主流的隐私计算技术。 Rosetta 旨在为人工智能快速提供隐私保护技术解决方案，不需要用户掌握任何密码学、联邦学习和硬件安全执行环境领域的专业知识。 Rosetta 在用户接口层复用了 TensorFlow 的对外 API 从而使得用户可以以最低的改造成本将隐私保护功能集成到现有的 TensorFlow 程序中。	401
federated-learning-lib	IBM	IBM federated learning是企业环境中用于联邦学习（FL）的Python框架。 为FL提供了一个基本结构，可以向其中添加高级功能。它不依赖于任何特定的机器学习框架，并支持不同的学习拓扑，例如，共享聚合器和协议。 它支持深度神经网络（DNN）以及经典的机器学习技术，如线性回归和k-均值。这包括监督和非监督方法以及强化学习。	257
federated	Google Research	联邦研究是与联邦学习和联邦分析相关的研究项目的集合。 该存储库中包含的许多项目都使用TensorFlow Federated（TFF），这是一个用于机器学习和其他分散数据计算的开源框架。	243
Flower	Adap	Flower是构建联邦学习系统的框架。flower的设计基于以下几个指导原则： 可定制：联邦学习系统因用例而异。Flower允许根据每个用例的需要进行多种不同的配置。 可扩展：Flower源于牛津大学的一个研究项目，因此它是在考虑AI研究的情况下建造的。可以扩展和覆盖许多组件以构建新的最先进的系统。 框架不可知：不同的机器学习框架有不同的优势。Flower可以与任何机器学习框架一起使用，例如，PyTorch、TensorFlow、PyTorch Lightning、MXNet、scikit learn、TFLite，甚至可以与原始NumPy一起用于喜欢手工计算梯度的用户 可以理解：flower是用可维护性来写的。鼓励社区阅读和贡献代码库。	615

5.Company&Application

公司	平台&地址
微众银行	联邦学习FATE
蚂蚁金服	摩斯多方安全计算平台
百度	百度点石隐私计算平台
腾讯	Angel PowerFL
洞见智慧	Insightone
华控清交	PrivPy 多方安全计算平台
富数科技	FMPC
矩阵元	rosetta
数犊科技	Tusita多方安全隐私计算平台
同盾科技	同盾智邦平台
致星科技	星云Clustar
趣链科技	BitMesh
蓝象智联	GAIA-Edge
锘崴科技	锘崴信隐私计算平台
华为云	可信智能计算服务 TICS
联易融	蜂密隐私计算平台
天冕科技	天冕联邦学习平台
度小满	貔貅隐私计算平台
星环科技	星环联邦学习软件
平安科技	蜂巢联邦智能平台
光之树	天机&云间
翼方健数	翼数坊XDP