Introduction to federated learning

A decentralized approach to AI

Machine learning has become a cornerstone of modern digital systems, enabling applications to make data-driven decisions, automate tasks, and enhance user experiences.Traditionally, machine learning follows a centralized model. This approach involves collecting data from various sources into a single, centralized repository, typically in a cloud environment or a dedicated data center.Then, advanced algorithms are applied to this consolidated dataset, training predictive models that can be deployed to make decisions based on new incoming data.

However, the centralized machine learning paradigm is facing growing challenges. The explosion of connected devices, sensors, and distributed data sources has led to an exponential increase in the volume and complexity of data being generated.At the same time, concerns around privacy, security, and regulatory compliance have made it increasingly difficult to freely move and consolidate data from different sources.The data needed to train effective machine learning models is often distributed across organizations, devices, or clients, making centralization challenging due to privacy risks and high transfer costs.

Federated learning

Federated learning (FL) is a decentralized approach that tackles the issues of centralized machine learning by allowing models to be trained on data distributed across various locations without moving the data.

Instead, FL moves the computation to where the data exists, enabling the creation of strong, globally-informed models while preserving data privacy and security.

How federated learning works

In federated learning, AI models are trained across multiple devices or servers (called client nodes) without needing to move the data off those devices. Here’s a simplified breakdown of how it works:

1. Starting the global model: The process begins with a global model on a central server. This could be any type of machine learning model, like a neural network or decision tree.
2. Sending the model to clients: The server sends the global model’s parameters to a group of selected client nodes. Each client uses its own local dataset, which stays securely on the device.
3. Local training: Each client trains the model using its local data, adjusting the model’s parameters based on what it learns from the data. This training process is repeated for several rounds, rather than continuing until full accuracy is achieved.
4. Combining the updates: The updated models from each client are sent back to the central server, where they are combined. A common approach is called Federated Averaging, where the server takes a weighted average of the updates from each client.

At last, the improved global model is sent back to the clients for further training. This cycle continues until the model reaches a satisfactory level of accuracy.

Federated Learning Explained

This video explains the core concept of federated machine learning without getting into technical details.While federated learning is often recognized for its privacy-preserving capabilities, this video highlights additional benefits that could be transformative for various industries through a simple example.

Cross-silo and cross-device

ederated learning is often discussed in terms of the technology behind it, such as cross-silo and cross-device approaches or horizontal and vertical.

• Cross-silo: Cross-silo federated learning typically involves collaboration between organizations (silos), each of which holds its own dataset. This can be seen in contexts like healthcare or finance, where different organizations train a shared model using their own data without exposing it to others.
• Cross-device: Cross-device federated learning refers to training models across a vast number of devices, such as smartphones or IoT devices, where each device has a small local dataset. This scenario involves a large number of participants (devices), each contributing to the training process.
• Horizontal: Horizontal federated learning is when different parties have datasets that share the same feature space, meaning the data has the same types of information or characteristics (e.g., different hospitals with similar patient data). This type is most common in both cross-silo and cross-device scenarios.
• Vertical: Vertical federated learning is when different organizations have datasets with different features for the same samples. For example, a bank and an e-commerce platform might collaborate, where the bank has financial data, and the e-commerce platform has purchase history, both related to the same customers.

A data ownership perspective

Federated learning can also be understood from a business and collaboration perspective. It focuses on how it applies in different business contexts and helps deal with challenges related to data privacy, regulations, and managing different stakeholders.

Practical Applications of Federated Learning

Federated learning enables innovative solutions across various industries. Here are key application areas where this technology is making a significant impact:

Cross-Device AI Applications

Smartphones and Personal Devices

Federated learning is revolutionizing how AI services are delivered on personal devices while maintaining privacy:

• Value Proposition: The advancement in AI services demands privacy-preserving mechanisms, especially for applications that operate across various devices like mobiles and customer service systems. These services include personalized recommendations, user behavior analysis, and enhanced user experience without compromising user privacy. The aim is to deliver sophisticated AI features while adhering to strict privacy standards.
• Data Challenges: Cross-device AI applications typically require the aggregation and analysis of sensitive user data, such as personal preferences, behavior patterns, and communication content. This data is often subject to privacy regulations like GDPR, necessitating strict controls on data sharing and processing. The challenge is to leverage this data for AI without violating privacy norms or exposing sensitive information.
• Federated Learning Solution: Federated Learning is ideal for this scenario as it allows for the decentralized training of AI models directly on users' devices. This approach ensures that sensitive data remains on the device, reducing the risks associated with data transfer and storage. It enables the collaborative improvement of AI models while maintaining data privacy and compliance with regulations, thus offering a robust solution for cross-device AI applications.

Data Collaborations for Industry Innovation

Air Traffic Management

The aviation industry demonstrates how federated learning can transform complex operational systems:

• Value Proposition: The Air Traffic Management (ATM) industry is moving towards a digital European sky. Trajectory Based Operations (TBO) allows for the proper coordination of ATM constraints on traffic, before or during flight and the airspace users can fly the best trajectory possible safely and efficiently. The goal is to have more accurate trajectory predictions.
• Data Challenges: TBO consist of seamless accurate prediction & optimisation of trajectories and ATM constrains through all the planning phases. Important relevant data spread out over several stakeholders in TBO is non-sharable data: too sensitive business data or protected by GDPR.
• Federated Learning Solution: Federated Learning addresses these challenges by enabling privacy- preserving exploitation of private data (relevant for operations) for ML purposes, while stakeholders keeping the full ownership and control in their own data silo.

Healthcare and Medical AI

Medical Imaging and Diagnostics

The healthcare sector showcases federated learning's potential in handling sensitive medical data:

• Value Proposition: There is a large potential in using AI in medicine. Use case include digital pathology, segmentation of organs and tumor, and detecting anomalies in time series data. Today doctors manually generate annotated training data. The software needs to learn from this user generated data to improve machine learning models. Without the data, they do not reach the required scale of data to train high performing models.
• Data Challenges: The problem is that the data is sensitive, owned by the customers, and subject to health data regulations. Data cannot be be moved off site and shared, and cannot be used effectively to improve AI software. Single clinics struggle with obtaining diverse enough data in large enough quantities. Anonymization and pooling of data is both costly, time consuming, and risks reducing data utility.
• Federated Learning Solution: With federated learning the regulatory challenge can be addressed and no data is moved off site. Without sharing the data, learnings from multiple clinics can be integrated to improve the machine learning models.

Sustainable Building Management

Energy-Smart Buildings

Federated learning is helping combat climate change through smart building management:

• Value Proposition: Buildings, including homes, malls, factories, and hospitals, contribute to 37% of global carbon emissions, with energy consumption expected to rise by 70% by 2030 due to population growth and rising living standards. However, this can be mitigated by using AI-based smart systems connected to sensors in buildings, potentially reducing energy consumption and carbon emissions by up to 30% through optimized lighting, heating, and cooling.
• Data Challenges: Smart buildings generate vast amounts of data from sensors and connected devices. Conventional techniques that centralize this data for analysis are not applicable in all cases due to various data transfer barriers such as regulations such as regional data transfer legislations, the data might be sensitive from a business perspective and/or owned by someone else, or, the volume of data might simply be so large that data transfer becomes a practical problem.
• Federated Learning Solution: Federated learning addresses these challenges by enabling local processing of data and model training on edge hardware (e.g. gateway servers in the buildings). By learning from localized data, federated learning not only upholds user privacy but can also craft baseline models that effectively learn from different buildings or building zones. These generalizable models can then be further optimized for specific buildings and homes. Furthermore, this machine learning framework enables devices to collaboratively learn from each other, increasing the overall network's capabilities to optimize energy efficiency across multiple buildings. Last but not least, the concept of federated learning enables better management of combined geographical or property-specific models which can be adapted for new homes and buildings that lack prior training data.

Automotive and Transportation

Fleet Intelligence and Autonomous Driving

The transportation sector leverages federated learning for vehicle optimization:

Fleet Maintenance:

• Value Proposition: Predictive maintenance of truck fleets requires federated on-vehicle training of models to benefit from collective maintenance experiences.
• Data Challenges: Gathering data from entire truck fleets is costly and challenging due to regulatory constraints, connectivity issues, and the vast amount of data produced.
• Federated Learning Solution: Federated Learning solves these issues by enabling on-vehicle machine learning model training, allowing each truck to learn from the maintenance data of the entire fleet without the need for data transfer.

Autonomous Vehicles:

• Value Proposition: Autonomous driving systems need to enable federated on-device training of models in order to manage large scale, in-vehicle machine learning.
• Data Challenges: Collecting data from all cars in use is expensive and in many cases impossible due to connection problems and the sheer quantity of data generated by modern cars.
• Federated Learning Solution: Federated Learning addresses the challenges by training on-board machine learning models in a federated setting so that each single car can learn from individual, group and fleet data.

Edge Computing Applications

Edge AI Optimization

Federated learning plays a crucial role in edge computing scenarios, where computation needs to happen closer to where data is generated. This approach is becoming increasingly important with the proliferation of IoT devices and the need for real-time processing.

• Value Proposition: Edge devices require immediate response capabilities for time-critical applications like industrial automation, autonomous systems, and smart security systems.
• Data Challenges: Edge computing optimizes resource usage by processing data locally, reducing the burden on central servers and network infrastructure.
• Federated Learning Solution: Local processing ensures continued operation even when network connectivity is limited or unavailable. By minimizing data transfer and cloud computing requirements, organizations can reduce operational costs.

Challenges and considerations

Federated learning introduces a unique set of challenges that must be carefully managed to ensure the effectiveness and security of the learning process across distributed environments.

Complexity and coordination

Federated learning introduces added complexity compared to traditional machine learning. It involves training models across multiple devices or servers, each with its own data. This requires careful coordination of machine learning operations (MLOps) to ensure efficient distribution and aggregation of model updates, reliable communication between nodes, and robust security throughout the process.

Heterogeneity of systems and data

In FL, client devices can vary significantly in terms of hardware capabilities, software environments, and data quality. Additionally, the data on these devices is often non-IID (not independent and identically distributed), meaning it may not represent the overall population. This requires careful algorithm design, thoughtful aggregation of updates, and strategies to manage the variability in both systems and data.

Scalability

Federated learning systems must scale efficiently as the number of participating devices increases. Managing thousands or millions of devices simultaneously introduces challenges in coordinating updates, aggregating models, and handling device failures or dropouts.

Privacy and data leakage

Although federated learning enhances privacy by keeping data on client devices, there is still a risk of data leakage through model updates. Adversaries could attempt to infer private information from the gradients or updates shared during the training process. Safeguarding against such risks is crucial.