Posts

TOWARDS FASTER AND BETTER FEDERATED LEARNING: A FEATURE FUSION APPROACH Abstract This paper mainly proposes a feature fusion approach to accelerate and improve the performance of federated learning. Introduction Nowadays many smart devices rely on pretrained models, which makes the machine’s inference capability lack personalization and flexibility. At the same time, smart terminals also generate a large amount of valid privacy-sensitive data, which can improve the personalization capability of these models. Federated learning, a distributed training algorithm that can train models directly on terminals, solves this problem. Among federated learning algorithms, those represented by FedAvg effectively alleviate the privacy issues in information exchange, but later research has also shown that federated learning still has problems such as: computational consumption, model accuracy. ...

Introduction Preamble Machine learning: suppose we use P to evaluate the performance of a computer program on a certain class of tasks T; if a program improves its performance on the tasks in T through experience E, then we say that, with respect to T and P, the program has learned from E. Basic Terminology Data-related: A dataset contains a series of records, and each record is an instance or a sample. That which reflects certain properties is an attribute or a feature. The value an attribute takes is called the attribute value, and the space spanned by the attributes is called the attribute space, the sample space, or the input space. Every point in this space is called a feature vector. ...

Model Evaluation and Selection Empirical Error and Overfitting Error rate: the overall proportion of misclassified samples Accuracy: the proportion of correctly classified samples Error: the difference between the actual output and the true output; on training samples it is the empirical error or training error, and on new samples it is the generalization error Overfitting: generalization performance degrades Underfitting: performs poorly even on the training samples Evaluation Methods A testing set is typically used to measure a model’s performance, with the testing error serving as an approximation of the generalization error ...

Abstract The goal of federated learning is to obtain a shared global model that all nodes can use. However, due to Non-IID data distributions, local models trained solely on local data often outperform the global model, which makes these nodes reluctant to participate in the federation process. This post introduces techniques currently used to personalize the global model so as to improve its performance on individual nodes. Introduction Federated learning is a distributed machine learning algorithm designed for Non-IID data. It enables multiple nodes to collaboratively train a shared global model without sharing their own data. At the beginning of each training round, the central server sends the current global model to every node, and each node trains the model on its local data. The central server then collects the model updates from all nodes and applies them to the global model. ...

A Survey on Personalization in Federated Learning Abstract The goal of federated learning is to obtain a shared global model that all nodes can use. However, due to Non-IID data distributions, local models trained solely on local data often outperform the global model, which makes these nodes reluctant to participate in the federation process. This post introduces techniques currently used to personalize the global model so as to improve its performance on individual nodes. ...