Researchers have studied how to improve Federated Learning (FL) in various areas, such as statistical and system heterogeneity, communication cost, and privacy. So far, most of the proposed solutions are either very tied to the application context or complex to be broadly reproduced in real-life applications involving humans. Developing modular solutions that can be leveraged by the vast majority of FL structures and are independent of the application people use is the new research direction opened by this paper. In this work, we propose a plugin (named FedPredict) to address three problems simultaneously: data heterogeneity, low performance of new/untrained and/or outdated clients, and communication cost. We do so mainly by combining global and local parameters (which brings generalization and personalization) in the inference step while adapting layer selection and matrix factorization techniques to reduce the downlink communication cost (server to client). Due to its simplicity, it can be applied to federated learning of different number of topologies. Results show that adding the proposed plugin to a given FL solution can significantly reduce the downlink communication cost by up to 83.3% and improve accuracy by up to 304% compared to the original solution.
@article{capanema2024novel,title={A Novel Prediction Technique for Federated Learning},author={Capanema, Claudio and {de Souza}, Allan M and {da Costa}, Joahannes B D and Silva, Fabricio A and Villas, Leandro A and Loureiro, Antonio A},journal={IEEE Transactions on Emerging Topics in Computing},volume={},pages={16},year={2024},doi={10.1109/TETC.2024.3471458},publisher={IEEE},}
IEEE VTC2024-Fall
EcoPredict: Assessing Distributed Machine Learning Methods for Predicting Urban Emissions
The growing number of vehicles has led to increased emissions of polluting gases, necessitating accurate forecasting for effective mitigation strategies and sustainable urban development. Leveraging computational resources in vehicles, this study presents a framework, called EcoPredict, for predicting CO2 emissions in collaborative vehicular network environments. The framework implements three forms of learning methods — centralized, federated, and split — using urban sensor networks for data collection. Experiments carried out in realistic vehicular mobility scenarios demonstrate the framework’s robustness and efficiency in providing real-time emission predictions. Each learning architecture has its own advantages and limitations regarding performance, training time, latency, communication overhead, and data privacy. Therefore, this work aims to assess their performance to analyze their effectiveness in urban environments.
IEEE FLTA 2024
Entropy and Mobility-based Model Assignment for Multi-model Vehicular Federated Learning
Machine Learning (ML) is extensively employed for key functions of Connected and Autonomous Vehicles (CAVs), where many models are executed simultaneously within a vehicle to provide diverse applications, from perception to planning and control. One of the most appealing ML approaches for CAVs is Federated Learning (FL) due to its privacy-preserving nature and distributed learning capabilities. However, current FL approaches mostly focus on single-model training and are unsuitable for parallel training of multiple models. Multi-model FL involves training multiple ML models to perform different tasks, often simultaneously, to meet the demands of different applications within the same context. In this way, this work introduces MELRO, an FL model assignment algorithm based on link duration, training latency, and data entropy from CAVs. MELRO balances computing resources and addresses high vehicle mobility while considering the heterogeneity of data and availability of resources in CAVs. The assignment algorithm takes advantage of data transmitted periodically by CAVs, such as beacons, to calculate link duration and training latency, define the model assignment matrix for CAVs, and consider data entropy. Finally, MELRO increases accuracy for FL applications by at least 11.76% while reducing training latency by 25% and maintaining computational resource usage.
IEEE FiCloud 2024
Adaptive Fit Fraction Based on Model Performance Evolution in Federated Learning
Federated Learning (FL) allows distributed training over data in clients devices, where distributed local models are aggregated in a central server to build the so-called global model. Selecting clients to participate in the distributed training process can help in improving training efficiency and computing performance by reducing training rounds and data transfers through the network. Since selecting involves creating a subset of clients who will participate in training, it is necessary to define the size of that subset (fit fraction) appropriately. This definition is an open problem. Using a constant fit fraction throughout the training is common, and its definition requires prior knowledge of the context where the training takes place. Proposals that modify it make aggressive changes, which can lead to overusage of training resources or under-usage of training data. The proposed solution, Adaptive Fit Fraction (AFF), tackles these issues by modifying the fit fraction, considering the changes in training performance over time. It uses an observation window and linear regression on the model’s performance in this window to identify the trend and intensity of training evolution. Based on these metrics, AFF defines the magnitude of the change on the following observation window’s fit fraction value. The results demonstrated that the proposal does not negatively impact the model’s performance and that it reduces costs by requiring less client participation in more stable scenarios.
@inproceedings{maciel2024adaptative,author={Maciel, Filipe and D, Joahannes B and Gonzalez, Luis F G and {de Souza}, Allan M and Villas, Leandro A and Bittencourt, Luiz F},title={Adaptive Fit Fraction Based on Model Performance Evolution in Federated Learning},year={2024},booktitle={11th International Conference on Future Internet of Things and Cloud (FiCloud)},pages={8},location={Vienna, Austria},series={IEEE FiCloud 2024}}
IEEE ISCC 2024
Combining Client Selection Strategy with Knowledge Distillation for Federated Learning in Non-IID Data
Federated Learning (FL) is a distributed approach in which multiple devices collaborate to train a shared global model. During its training, client devices must communicate their local gradients to the central server to update the global model weights. This incurs significant communication costs (bandwidth utilization and the number of messages exchanged), leading to many challenges (communication bottlenecks and scalability issues in FL). Furthermore, the heterogeneous nature of clients’ datasets poses an extra challenge to the model training. In this sense, we introduce FedCCSKD, a Federated Clustered Client Selection and Knowledge Distillation training algorithm, to decrease the overall communication costs for FL. FedCCSKD is an innovative combination of: (i) client selection, and (ii) knowledge distillation approaches with three main objectives: (i) reducing the number of devices training at every round; (ii) decreasing the number of rounds to reach convergence; and (iii) mitigating the effect of clients’ heterogeneous data on the global model effectiveness. Our experimental evaluations on two well-known datasets, MNIST and MotionSense, demonstrate that FedCCSKD is highly efficient in training the global model. FedCCSKD reaches a higher accuracy score and faster convergence than state-of-the-art baseline models. Our results also show higher performance when analyzing the accuracy scores on the client’s datasets.
@inproceedings{mohamed2024combining,author={Mohamed, Aissa H and {da Costa}, Joahannes B D and {de Souza}, Allan M and Villas, Leandro A and {dos Reis}, Julio C},title={Combining Client Selection Strategy with Knowledge Distillation for Federated Learning in {Non-IID} Data},year={2024},booktitle={29th IEEE Symposium on Computers and Communications (ISCC)},pages={7},location={Paris, France},series={IEEE ISCC 2024}}
IEEE DCOSS-IoT 2024
A Modular Plugin for Concept Drift in Federated Learning
Claudio Capanema, Joahannes B D da Costa, Fabricio A Silva, Leandro A Villas, and Antonio A Loureiro
In The 20th International Conference on Distributed Computing in Smart Systems and the Internet of Things (IEEE DCOSS-IoT 2024), Abu Dhabi, United Arab Emirates, 2024
In Federated Learning (FL), personalization-based solutions have emerged to improve clients’ performance, considering the statistical heterogeneity of local datasets. However, these methods are designed for a static environment and the previously learned model becomes obsolete as the local data distribution changes over time. This problem, known as concept drift, is widespread in several scenarios (e.g., change in user habits, different characteristics of visited geolocations, and seasonality effects, among others) but needs to be addressed by most solutions in the literature. In this work, we present FedPredict-Dynamic, a plugin that allows FL solutions to support statistically heterogeneous stationary and non-stationary local data. The proposed method is a lightweight and reproducible modular plugin and can be added to several FL solutions. Unlike state-of-the-art concept drift techniques, it can rapidly adapt clients to the new data context in the prediction stage without requiring additional training. Results show that when context changes, FedPredict-Dynamic can achieve accuracy improvements of up to 195% compared to concept drift-aware solutions and 210.7% compared to traditional FL methods.
@inproceedings{capanema2024modular,author={Capanema, Claudio and {da Costa}, Joahannes B D and Silva, Fabricio A and Villas, Leandro A and Loureiro, Antonio A},title={A Modular Plugin for Concept Drift in Federated Learning},year={2024},booktitle={The 20th International Conference on Distributed Computing in Smart Systems
and the Internet of Things (IEEE DCOSS-IoT 2024)},pages={},location={Abu Dhabi, United Arab Emirates},series={{IEEE DCOSS-IoT}~'24},doi={10.1109/DCOSS-IoT61029.2024.00024},}
Elsevier Ad Hoc Networks
Adaptive client selection with personalization for communication efficient Federated Learning
Federated Learning (FL) is a distributed approach to collaboratively training machine learning models. FL requires a high level of communication between the devices and a central server, thus imposing several challenges, including communication bottlenecks and network scalability. This article introduces ACSP-FL, a solution to reduce the overall communication and computation costs for training a model in FL environments. ACSP-FL employs a client selection strategy that dynamically adapts the number of devices training the model and the number of rounds required to achieve convergence. Moreover, ACSP-FL enables model personalization to improve clients performance. A use case based on human activity recognition datasets aims to show the impact and benefits of ACSP-FL when compared to state-of-the-art approaches. Experimental evaluations show that ACSP-FL minimizes the overall communication and computation overheads to train a model and converges the system efficiently. In particular, ACSP-FL reduces communication up to 95% compared to literature approaches while providing good convergence even in scenarios where data is distributed differently, non-independent and identical way between client devices.
@article{desouza2024adaptative,title={Adaptive client selection with personalization for communication efficient Federated Learning},author={{de Souza}, Allan M and Maciel, Filipe and {da Costa}, Joahannes BD and Bittencourt, Luiz F and Cerqueira, Eduardo and Loureiro, Antonio AF and Villas, Leandro A},journal={Ad Hoc Networks},volume={157},pages={103462},year={2024},issn={1570-8705},doi={10.1016/j.adhoc.2024.103462},publisher={Elsevier},}
2023
IEEE/ACM UCC
EntropicFL: Efficient Federated Learning via Data Entropy and Model Divergence
Rómulo Bustincio, Allan M de Souza, Joahannes B D da Costa, and Luiz F Bittencourt
In Proceedings of the 16th IEEE/ACM Utility and Cloud Computing Conference (UCC), Taormina (Messina), Italy, 2023
Federated Learning (FL) is a strategy for training distributed learning models. This approach gives rise to significant challenges including the non-independent and identically distributed (non-IID.) characteristics inherent to the training data, which can wield influence over the comprehensive accuracy of the global model. Moreover, the collaborative involvement of multiple clients in training protocols frequently engenders increased communication overheads and resource management overheads.In this research, we present an innovative strategy to address the inherent challenges of federated learning, including the communication overhead, data heterogeneity, and privacy preservation concerns. Our proposed approach centers on the concept of adaptive client selection, comprising a two-step process: firstly, the identification of a subset of clients possessing pertinent and representative data for participation in model training, and secondly, the determination of whether to transmit the local updates from these selected clients to the central aggregation server. Our methodology leverages the metrics of data entropy and model divergence to guide this client selection process. By applying this approach, we effectively mitigate communication overhead without compromising the accuracy achieved in the federated learning process.
@inproceedings{bustincio2023entropic,author={Bustincio, Rómulo and {de Souza}, Allan M and {da Costa}, Joahannes B D and Bittencourt, Luiz F},title={EntropicFL: Efficient Federated Learning via Data Entropy and Model Divergence},year={2023},publisher={Association for Computing Machinery},address={New York, NY, USA},url={https://doi.org/10.1145/3603166.3632563},doi={10.1145/3603166.3632563},booktitle={Proceedings of the 16th IEEE/ACM Utility and Cloud Computing Conference (UCC)},pages={},location={Taormina (Messina), Italy},series={{UCC}~'23},}
IEEE/ACM UCC
CCSF: Clustered Client Selection Framework for Federated Learning in non-IID Data
Federated Learning (FL) is a distributed approach where numerous devices train a shared global model for Machine Learning (ML) tasks. At every training round, the client devices must share their new local gradients with the central server to update the global model. Hence, FL requires high communication costs in terms of bandwidth and the number of messages exchanged between FL clients and the central server, leading to many issues, such as communication bottlenecks and scaling in the network. Consequently, having all devices participating in every training round is not practical. Moreover, the devices’ local datasets are usually not Independent and Identically Distributed (IID), posing additional challenges for training the global model. In this sense, we introduce a Clustered Client Selection Framework (CCSF) to decrease the overall communication costs for training an ML model in the FL environment. CCSF clusters the client devices and employs a biased client selection strategy with two main objectives: (i) reducing the number of devices training at every round; and (ii) the number of rounds required to reach convergence. Our experimental evaluations, conducted on two well-known datasets, MNIST and MotionSense, show that CCSF is highly efficient, where the clients’ local datasets can be grouped into homogeneous clusters. In MNIST, CCSF reaches an accuracy score above 60% in less than 50 rounds compared to FedAvg at 50% after 100 rounds. The performance gap is wider in the MotionSense data. CCSF reaches an accuracy score of 70% in a little more than 20 training rounds compared to FedAvg below 30% of accuracy in the first 100 FL rounds.
@inproceedings{mohamed2023ccsf,author={Mohamed, Aissa H and {de Souza}, Allan M and {da Costa}, Joahannes B D and Villas, Leandro A and {dos Reis}, Julio C},title={CCSF: Clustered Client Selection Framework for Federated Learning in non-IID Data},year={2023},publisher={Association for Computing Machinery},address={New York, NY, USA},url={https://doi.org/10.1145/3603166.3632563},doi={10.1145/3603166.3632563},booktitle={Proceedings of the 16th IEEE/ACM Utility and Cloud Computing Conference (UCC)},pages={},location={Taormina (Messina), Italy},series={{UCC}~'23},}
Elsevier Ad Hoc Networks
Mobility-aware Vehicular Cloud Formation Mechanism for Vehicular Edge Computing Environments
Rapid advancements in vehicular technology and increased vehicle modernization have led to the emergence of intelligent and interconnected entities. As a result, the Vehicular Edge Computing (VEC) paradigm has gained prominence. This paradigm enables the provision of cloud computing services close to vehicular users by utilizing the idle computational resources of vehicles to execute tasks that require computing power beyond what is available locally. Aggregating these computational resources in the vehicular context is known as Vehicular Cloud (VCloud) formation. However, leveraging and aggregating these resources poses several challenges due to the dynamic nature of the vehicular environment. One of the main challenges is the efficient selection of vehicles to assume management roles in the distribution of computational power within the group, often referred to as leading vehicles. This research presents a mobility-aware mechanism called PREDATOR to enhance the VCloud formation process. In this mechanism, the Roadside Unit (RSU) provides vehicular mobility predictions, enabling the selection of the most stable vehicles within the RSU coverage area to assume leadership roles in the VCloud. In this context, vehicle stability is associated with a vehicle’s time within the RSU coverage area, known as dwell time. PREDATOR employs a microscopic perspective to select vehicles with the longest dwell time in the VCloud, allowing for efficient management of computational resource utilization. Simulation results have demonstrated that PREDATOR not only increases the VCloud lifetime but also minimizes leader changes, reduces network message exchange, mitigates packet collisions, and facilitates the effective utilization of aggregated vehicular resources compared to state-of-the-art approaches.
@article{dacosta2023mobilityaware,title={Mobility-aware Vehicular Cloud Formation Mechanism for Vehicular Edge Computing Environments},author={{da Costa}, Joahannes B D and Lobato, Wellington and {de Souza}, Allan M and Ros{\'a}rio, Denis and Cerqueira, Eduardo and Sommer, Christoph and Villas, Leandro A},journal={Ad Hoc Networks},volume={151},pages={103300},year={2023},issn={1570-8705},doi={https://doi.org/10.1016/j.adhoc.2023.103300},publisher={Elsevier},}
IEEE VTC2023-Fall
Improving Fairness and Performance in Resource Usage for Vehicular Edge Computing
Vehicular Edge Computing (VEC) has emerged to offer cloud computing services closer to vehicular users by combining vehicles and edge computing nodes into Vehicular Clouds (VCs). In this scenario, an intelligent task scheduler must decide which VC will run which tasks, considering contextual aspects like vehicular mobility and tasks’ requirements. This is important to minimize both processing time and monetary costs. However, such direct optimization can lead to unfairness in resource usage, easily leading to (as we will show) decreased performance. Towards this end, in this work, we propose FARID, a task scheduling mechanism that considers contextual aspects of its decision process and applies a probabilistic selection function on VCs to balance the processing load and increase the fairness in the use of vehicular resources. Compared to state-of-the-art solutions, FARID has a higher level of fairness and can schedule more tasks while minimizing monetary costs and system latency.
IEEE T-ITS
Mobility and Deadline-aware Task Scheduling Mechanism for Vehicular Edge Computing
Vehicular Edge Computing (VEC) is a promising paradigm that provides cloud computing services closer to vehicular users. In VEC, vehicles and communication infrastructures can form pools with computational resources to meet vehicular services with low-latency constraints. These resource pools are known as Vehicular Cloud (VC). The usage of VC resources requires a task scheduling process. In this case, depending on its complexity, a vehicular service can be divided into different tasks. An efficient task scheduling needs to orchestrate where and for how long such tasks will run, considering the available pools, the mobility of nodes, and the tasks deadline constraints. Thus, this article proposes an efficient VC task scheduler based on an approximation heuristic and resources prediction to select the best VC for each task, called MARINA. MARINA aims to analyze the behavior of vehicles that share their computational resources with the VC and make scheduling decisions based on the mobility (VC availability) of these vehicles. Simulation results under a realistic scenario demonstrate the efficiency of MARINA compared to existing state-of-the-art mechanisms in terms of the number of tasks scheduled, monetary cost, system latency, and Central Processing Unit (CPU) utilization.
Elsevier Ad Hoc Networks
HARMONIC: Shapley values in market games for resource allocation in vehicular clouds
Real-time allocation of resources to fulfill service requests from road vehicles is becoming increasingly complex, for two main reasons: the continuous increase in the number of Internet-connected vehicles on roads all over the world, and the emergence of complex and resource-greedy applications that require fast execution, often under limited availability of computational resources. While many resource allocation solutions to this problem have been proposed recently, these solutions rely on unrealistic scenarios and constraints that limit their practical use. This paper presents HARMONIC, a Game Theory-based coalition game that aims to maximize resource utilization and dynamically balance resource usage across multiple Vehicular Clouds (VCs). HARMONIC employs a Shapley value-based strategy to determine the order of task allocation to available resources. It is built upon our proposed Market Game model, specifically designed to address resource allocation challenges in dynamic VCs. We conduct a comparative analysis with existing literature solutions under various scenarios and resource constraints to evaluate HARMONIC’s performance. Our simulation results demonstrate that HARMONIC achieves resource allocation in fewer rounds and with fewer failures. Furthermore, it effectively distributes tasks to more VCs, improving load balancing and overall system efficiency.
2022
IEEE VTC2022-Fall
Efficient Pareto Optimality-based Task Scheduling for Vehicular Edge Computing
Vehicular Edge Computing is a promising paradigm that provides cloud computing services closer to vehicular users. Vehicles and communication infrastructure can cooperatively provide vehicular services with low latency constraints through vehicular cloud formation and using these computational resources via task scheduling. An efficient task scheduler must decide which cloud will run the tasks, considering vehicular mobility and task requirements. This is important to minimize processing time and, consequently, monetary cost. However, the literature solutions do not consider these contextual aspects together, degrading the overall system efficiency. This work presents EFESTO, a task scheduling mechanism that considers contextual aspects in its decision process. The results show that, compared to state-of-the-art solutions, EFESTO can schedule more tasks while minimizing monetary cost and system latency.
IEEE VTC2022-Fall
Flexe: Investigating Federated Learning in Connected Autonomous Vehicle Simulations
Due to the increased computational capacity of Connected and Autonomous Vehicles (CAVs) and worries about transferring private information, it is becoming more and more appealing to store data locally and move network computing to the edge. This trend also extends to Machine Learning (ML) where Federated learning (FL) has emerged as an attractive solution for preserving privacy. Today, to evaluate the implemented vehicular FL mechanisms for ML training, researchers often disregard the impact of CAV mobility, network topology dynamics, or communication patterns, all of which have a large impact on the final system performance. To address this, this work presents FLEXE, an Open Source extension to Veins that offers researchers a simulation environment to run FL experiments in realistic scenarios. FLEXE combines the popular Veins framework with the OpenCV library. Using the example of traffic sign recognition, we demonstrate how FLEXE can support investigations of FL techniques in a vehicular environment.
2021
IEEE IWCMC 2021
TOVEC: Task Optimization Mechanism for Vehicular Clouds using Meta-heuristic Technique
The automotive industry has been continuously investing in the modernization of the vehicles by the addition of more sensors and computational power. With this evolution, Intelligent Transportation Systems (ITS) make up a services framework that seeks to mitigate problems in the road sector. Many ITS services are facilitated by creating vehicular clouds (VCs) by using the communication capabilities of other vehicles to provide cloud services closer to vehicular applications. However, often the computational resources present in the vehicles are underutilized. For this reason, we propose in this work a mechanism that efficiently allocates computational tasks to be performed in VCs. Simulation results on a realistic mobility trace show that, with our mechanism, tasks are more allocated, the reward from allocating these tasks was higher, resource waste was minimized, and less CPU is used in the allocation processing. Also, the proposed mechanism is statistically close to a globally optimal solution.
IEEE VTC-Spring 2020
Degree Centrality-based Caching Discovery Protocol for Vehicular Named-Data Networks
Services that aim to make the current transportation system more secure, sustainable, and efficient constitute the Traffic Management Systems (TMS). Vehicular Ad hoc Networks (VANETs) exert a strong influence for TMS applications, due to TMS services require data, communication, and processing for operation. Besides, VANET allows direct communication between vehicles, and data are exchanged and processed between them. Several TMS services require disseminated information among decision-making vehicles. However, such dissemination is a challenging task, due to the specific characteristics of VANETs, such as short-range communication and high node mobility, resulting in several variations in their topology. In this article, we introduce an extensive analysis of our proposed data dissemination protocol based on complex networks’ metrics for urban VANET scenarios, called DDRX. Each vehicle must build a subgraph to identify the relay node to continue the dissemination process. Based on the local graph, it is possible to select the relay nodes based on complex networks’ metrics. Simulation results show that DDRX offers high efficiency in terms of coverage, number of transmitted packets, delay, and packet collisions compared to well-known data dissemination protocols. Also, DDRX provides significant improvements to a TMS that needs efficient data dissemination.
IEEE ICC 2019
A virtual machine migration policy based on multiple attribute decision in vehicular cloud scenario