Ph. D. Project
Title:
FEDERATED LEARNING FOR PROGNOSTICS AND HEALTH MANAGEMENT (PHM)
Dates:
2023/04/07 - 2026/04/06
Student:
Supervisor(s):
Other supervisor(s):
CR Christophe CERISARA (christophe.cerisara@loria.fr)
Description:
General context:
The future PhD student will be recruited at CRAN laboratory to develop his PhD in the framework of the Horizon Europe project MODAPTO (Modular manufacturing and distributed control via interoperable digital twin). The MODAPTO project seeks to develop and deploy modular and reconfigurable manufacturing systems leveraging production modules enhanced by distributed intelligence via interoperable Digital Twins (DTs) based on industrial standards. At the same time, MODAPTO materializes the benefits of global production view by enabling collective intelligence within modular production schemes for effective module and production line design, reconfiguration, and decision support for, among other, predictive maintenance.
Within the MODAPTO framework, the objective of this PhD is to adapt and develop Federated Learning algorithm for predictive maintenance, notably for diagnostics and prognostics, distributed over the modules. This algorithm will be validated on use cases of the pilot sites of the project, including SEW USOCOME
The PhD student will be integrated to the project and as such will participate in meetings, writing deliverables, presenting progress and results, etc...
Scientific context:
Factory of the Future (FoF) has brought on the scene disruptive paradigms that change the way industry and manufacturing should consider and integrate new key technologies of Industry 4.0. One of these new paradigms is Cyber-Physical Systems (CPS) and their instantiation to manufacturing as Cyber Physical Production Systems (CPPS). CPPS aims to handle the significant shift of industry towards FoF-principles, like mass customization and flexibility/adpatability, while keeping optimal performance from both the economic and the sustainability viewpoints. One of the main features of CPPS to achieve this goal is autonomy (self-X), including self-awareness and self-decision. Machine Learning and Artificial Intelligence are the first candidate to provide such capability. Their success in application in relation to the digital world (e.g. GAFAM) is undoubtable. In addition to advance in ML and AI theory, such a success comes also from the availability of tremendous amount of data. Such availability has deeply changed the way Deep Neural Nets (DNN) are trained thanks to ImageNet revolution for computer vision and even more in Natural Language Processing with GPT3. Indeed, now, most of DNN used for computer vision and natural language processing are pre-trained.
Nevertheless, such revolution has not occurred yet in manufacturing. The main cause is the lack of huge dataset to enable DNN models pre-training. The first limit lies in the reluctance of companies to share their data with third parties. Such situation is increased while dealing with business area, such as Prognostics and Health Management, where the occurrence of relevant and labelled event is rare (nowadays, industrial machines are reliable and maintenance practices allows to reach good availability targets).
To face this twofold challenge, one original way, in manufacturing domain, is Federated Learning. Federated Learning is a decentralize way to learn where several nodes learned their own models on the edge and send their parameters to a central server which build an aggregated model. (McMahan et al., 2017) proposed the vanilla FL and obtain an aggregated model by iteratively averaging the parameters of distributed local node models in a central one. In return, the central model is send to update node models so that their models can benefit from the entire knowledge acquired by all the nodes. Such approach allows to partly answer the two afore mentioned limitation of AI in manufacturing:
Data privacy: indeed, only the parameters or gradient of the local models are sent to the central server, the data remains in the nodes.
Data scarcity: by leveraging decentralized learning, FL benefit from fleet of systems deployed like in industry
Nevertheless, when regarding the application of FL to predictive mainteance of CPPS (seeing CPPS as nodes) two of the main challenges to be handle, that are partly considered in computer science, are on one hand the contextual variability inherent to CPPS and on the other hand the Few Shot Learning (FSL), or even one-shot learning, in the nodes of the FL . Indeed, even when considering identical production systems, for instance like machine tools, their behavior cannot be considered as identical thanks to their environment, their ageing, the mission they performed (e.g. different material or product to be processed), the maintenance performed... This can be viewed as if the data at the node level were issued from a dynamic process whose generative distribution evolve with time. Moreover, even if, by aggregating the events of a set of nodes, their number becomes significant to train a DNN, the local learning in the FL will have only a small amount of data. The convergence of FL has then to be studied when FSL is at the node. Such evidence being, maybe, the reason why FL has not found much audience in industry so far.
The objective of the PhD will be to design FL strategy enabling to consider the two above mentioned challenges. To this end, three specific research questions in relation with FL will be addressed:
1. How to design a central model and an aggregation strategy enabling the storage of the whole process information as well as preventing from catastrophic forgetting?
2. How to design a distillation strategy from the central model to the node models enabling the distillation of the proper knowledge to each node?
3. What is the convergence of LF methods in the case of FSL in nodes?
The expected result will provide new capabilities to FL leading to propose new FL algorithm.
The PhD will explore two predictive maintenance processes: detection and prognostics. The nodes models to be considered will DNN like One-class models for detection and MLP-LSTM-MLP for prognostics.
Profile:
The candidate will mandatorily have skills in machine learning and deep learning completed, ideally, with skills in the areas of Manufacturing and Production Systems. The qualities expected of the candidate are strong programming skills (ideally python), organizational skills in project management and the ability to learn and work in a team.
The future PhD student will be recruited at CRAN laboratory to develop his PhD in the framework of the Horizon Europe project MODAPTO (Modular manufacturing and distributed control via interoperable digital twin). The MODAPTO project seeks to develop and deploy modular and reconfigurable manufacturing systems leveraging production modules enhanced by distributed intelligence via interoperable Digital Twins (DTs) based on industrial standards. At the same time, MODAPTO materializes the benefits of global production view by enabling collective intelligence within modular production schemes for effective module and production line design, reconfiguration, and decision support for, among other, predictive maintenance.
Within the MODAPTO framework, the objective of this PhD is to adapt and develop Federated Learning algorithm for predictive maintenance, notably for diagnostics and prognostics, distributed over the modules. This algorithm will be validated on use cases of the pilot sites of the project, including SEW USOCOME
The PhD student will be integrated to the project and as such will participate in meetings, writing deliverables, presenting progress and results, etc...
Scientific context:
Factory of the Future (FoF) has brought on the scene disruptive paradigms that change the way industry and manufacturing should consider and integrate new key technologies of Industry 4.0. One of these new paradigms is Cyber-Physical Systems (CPS) and their instantiation to manufacturing as Cyber Physical Production Systems (CPPS). CPPS aims to handle the significant shift of industry towards FoF-principles, like mass customization and flexibility/adpatability, while keeping optimal performance from both the economic and the sustainability viewpoints. One of the main features of CPPS to achieve this goal is autonomy (self-X), including self-awareness and self-decision. Machine Learning and Artificial Intelligence are the first candidate to provide such capability. Their success in application in relation to the digital world (e.g. GAFAM) is undoubtable. In addition to advance in ML and AI theory, such a success comes also from the availability of tremendous amount of data. Such availability has deeply changed the way Deep Neural Nets (DNN) are trained thanks to ImageNet revolution for computer vision and even more in Natural Language Processing with GPT3. Indeed, now, most of DNN used for computer vision and natural language processing are pre-trained.
Nevertheless, such revolution has not occurred yet in manufacturing. The main cause is the lack of huge dataset to enable DNN models pre-training. The first limit lies in the reluctance of companies to share their data with third parties. Such situation is increased while dealing with business area, such as Prognostics and Health Management, where the occurrence of relevant and labelled event is rare (nowadays, industrial machines are reliable and maintenance practices allows to reach good availability targets).
To face this twofold challenge, one original way, in manufacturing domain, is Federated Learning. Federated Learning is a decentralize way to learn where several nodes learned their own models on the edge and send their parameters to a central server which build an aggregated model. (McMahan et al., 2017) proposed the vanilla FL and obtain an aggregated model by iteratively averaging the parameters of distributed local node models in a central one. In return, the central model is send to update node models so that their models can benefit from the entire knowledge acquired by all the nodes. Such approach allows to partly answer the two afore mentioned limitation of AI in manufacturing:
Data privacy: indeed, only the parameters or gradient of the local models are sent to the central server, the data remains in the nodes.
Data scarcity: by leveraging decentralized learning, FL benefit from fleet of systems deployed like in industry
Nevertheless, when regarding the application of FL to predictive mainteance of CPPS (seeing CPPS as nodes) two of the main challenges to be handle, that are partly considered in computer science, are on one hand the contextual variability inherent to CPPS and on the other hand the Few Shot Learning (FSL), or even one-shot learning, in the nodes of the FL . Indeed, even when considering identical production systems, for instance like machine tools, their behavior cannot be considered as identical thanks to their environment, their ageing, the mission they performed (e.g. different material or product to be processed), the maintenance performed... This can be viewed as if the data at the node level were issued from a dynamic process whose generative distribution evolve with time. Moreover, even if, by aggregating the events of a set of nodes, their number becomes significant to train a DNN, the local learning in the FL will have only a small amount of data. The convergence of FL has then to be studied when FSL is at the node. Such evidence being, maybe, the reason why FL has not found much audience in industry so far.
The objective of the PhD will be to design FL strategy enabling to consider the two above mentioned challenges. To this end, three specific research questions in relation with FL will be addressed:
1. How to design a central model and an aggregation strategy enabling the storage of the whole process information as well as preventing from catastrophic forgetting?
2. How to design a distillation strategy from the central model to the node models enabling the distillation of the proper knowledge to each node?
3. What is the convergence of LF methods in the case of FSL in nodes?
The expected result will provide new capabilities to FL leading to propose new FL algorithm.
The PhD will explore two predictive maintenance processes: detection and prognostics. The nodes models to be considered will DNN like One-class models for detection and MLP-LSTM-MLP for prognostics.
Profile:
The candidate will mandatorily have skills in machine learning and deep learning completed, ideally, with skills in the areas of Manufacturing and Production Systems. The qualities expected of the candidate are strong programming skills (ideally python), organizational skills in project management and the ability to learn and work in a team.
Keywords:
Predictive Maintenance, PHM, Deep Learning, Federated Learning
Department(s):
| Modeling and Control of Industrial Systems |
