| 1. |
- Dakkak, Anas, et al.
(author)
-
Towards Continuous Data Collection from In-service Products: Exploring the Relation between Data Dimensions and Collection Challenges
- 2021
-
In: Proceedings - Asia-Pacific Software Engineering Conference, APSEC. - 1530-1362. ; 2021-December, s. 243-252
-
Conference paper (peer-reviewed)abstract
- Data collected from in-service products play an important role in enabling software-intensive embedded systems suppliers to embrace data-driven practices. Data can be used in many different ways such as to continuously learn and improve the product, enhance post-deployment services, reduce operational cost or create a better user experience. While there is no shortage of possible use cases leveraging data from in-service products, software-intensive embedded systems companies struggle to continuously collect data from their in-service products. Often, data collection is done in an ad-hoc way and targeting specific use cases or needs. Besides, few studies have investigated data collection challenges in relation to the data dimensions, which are the minimum set of quantifiable data aspects that can define software-intensive embedded product data from a collection point of view. To help address data collection challenges, and to provide companies with guidance on how to improve this process, we conducted a case study at a large multinational telecommunications supplier focusing on data characteristics and collection challenges from the Radio Access Networks (RAN) products. We further investigated the relations of these challenges to the data dimensions to increase our understanding of how data dominions contribute to the challenges.
|
|
| 2. |
- Zhang, Hongyi, 1996, et al.
(author)
-
AF-DNDF : Asynchronous Federated Learning of Deep Neural Decision Forests
- 2021
-
In: Proceedings - 2021 47th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2021. - : IEEE. - 9781665427050 ; , s. 308-315
-
Conference paper (peer-reviewed)abstract
- In recent years, with more edge devices being put into use, the amount of data that is created, transmitted and stored is increasing exponentially. Moreover, due to the development of machine learning algorithms, modern software-intensive systems are able to take advantage of the data to further improve their service quality. However, it is expensive and inefficient to transmit large amounts of data to a central location for the purpose of training and deploying machine learning models. Data transfer from edge devices across the globe to central locations may also raise privacy and concerns related to local data regulations. As a distributed learning approach, Federated Learning has been introduced to tackle those challenges. Since Federated Learning simply exchanges locally trained machine learning models rather than the entire data set throughout the training process, the method not only protects user data privacy but also improves model training efficiency. In this paper, we have investigated an advanced machine learning algorithm, Deep Neural Decision Forests (DNDF), which unites classification trees with the representation learning functionality from deep convolutional neural networks. In this paper, we propose a novel algorithm, AF-DNDF which extends DNDF with an asynchronous federated aggregation protocol. Based on the local quality of each classification tree, our architecture can select and combine the optimal groups of decision trees from multiple local devices. The introduction of the asynchronous protocol enables the algorithm to be deployed in the industrial context with heterogeneous hardware settings. Our AF-DNDF architecture is validated in an automotive industrial use case focusing on road objects recognition and demonstrated by an empirical experiment with two different data sets. The experimental results show that our AF-DNDF algorithm significantly reduces the communication overhead and accelerates model training speed without sacrificing model classification performance. The algorithm can reach the same classification accuracy as the commonly used centralized machine learning methods but also greatly improve local edge model quality.
|
|
| 3. |
- Zhang, Hongyi, 1996, et al.
(author)
-
Autonomous Navigation and Configuration of Integrated Access Backhauling for UAV Base Station Using Reinforcement Learning
- 2022
-
In: Proceedings - 2022 IEEE Future Networks World Forum, FNWF 2022. - : IEEE. ; , s. 184-189, s. 184-189
-
Conference paper (peer-reviewed)abstract
- Fast and reliable connectivity is essential to enhance situational awareness and operational efficiency for public safety mission-critical (MC) users. In emergency or disaster circumstances, where existing cellular network coverage and capacity may not be available to meet MC communication demands, deployable-network-based solutions such as cells-on-wheels/wings can be utilized swiftly to ensure reliable connection for MC users. In this paper, we consider a scenario where a macro base station (BS) is destroyed due to a natural disaster and an unmanned aerial vehicle carrying BS (UAV-BS) is set up to provide temporary coverage for users in the disaster area. The UAV-BS is integrated into the mobile network using the 5G integrated access and backhaul (IAB) technology. We propose a framework and signalling procedure for applying machine learning to this use case. A deep reinforcement learning algorithm is designed to jointly optimize the access and backhaul antenna tilt as well as the three-dimensional location of the UAV-BS in order to best serve the on-ground MC users while maintaining a good backhaul connection. Our result shows that the proposed algorithm can autonomously navigate and configure the UAV-BS to improve the throughput and reduce the drop rate of MC users.
|
|
| 4. |
- Zhang, Hongyi, 1996, et al.
(author)
-
Deep Reinforcement Learning for Multiple Agents in a Decentralized Architecture: A Case Study in the Telecommunication Domain
- 2023
-
In: Proceedings - IEEE 20th International Conference on Software Architecture Companion, ICSA-C 2023. - : IEEE COMPUTER SOC. ; , s. 183-186
-
Conference paper (peer-reviewed)abstract
- Deep reinforcement learning has made significant development in recent years, and it is currently applied not only in simulators and games but also in embedded systems. However, when implemented in a real-world context, reinforcement learning is frequently shown to be unstable and incapable of adapting to realistic situations, particularly when directing a large number of agents. In this paper, we develop a decentralized architecture for reinforcement learning to allow multiple agents to learn optimal control policies on their own devices of the same kind but in varied environments. For such multiple agents, the traditional centralized learning algorithm usually requires a costly or time-consuming effort to develop the best-regulating policy and is incapable of scaling to a large-scale system. To address this issue, we propose a decentralized reinforcement learning algorithm (DecRL) and information exchange scheme for each individual device, in which each agent shares the individual learning experience and information with other agents based on local model training. We incorporate the algorithm into each agent in the proposed collaborative architecture and validate it in the telecommunication domain under emergency conditions, in which a macro base station (BS) is broken due to a natural disaster, and three unmanned aerial vehicles carrying BSs (UAV-BSs) are deployed to provide temporary coverage for mission-critical (MC) users in the disaster area. Based on the findings, we show that the proposed decentralized reinforcement learning algorithm can successfully support multi-agent learning, while the learning speed and service quality can be further enhanced.
|
|
| 5. |
- Zhang, Hongyi, 1996, et al.
(author)
-
Deep Reinforcement Learning in a Dynamic Environment: A Case Study in the Telecommunication Industry
- 2022
-
In: Proceedings - 48th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2022. ; , s. 68-75
-
Conference paper (peer-reviewed)abstract
- Reinforcement learning, particularly deep reinforcement learning, has made remarkable progress in recent years and is now used not only in simulators and games but is also making its way into embedded systems as another software-intensive domain. However, when implemented in a real-world context, reinforcement learning is typically shown to be fragile and incapable of adapting to dynamic environments. In this paper, we provide a novel dynamic reinforcement learning algorithm for adapting to complex industrial situations. We apply and validate our approach using a telecommunications use case. The proposed algorithm can dynamically adjust the position and antenna tilt of a drone-based base station to maintain reliable wireless connectivity for mission-critical users. When compared to traditional reinforcement learning approaches, the dynamic reinforcement learning algorithm improves the overall service performance of a drone-based base station by roughly 20%. Our results demonstrate that the algorithm can quickly evolve and continuously adapt to the complex dynamic industrial environment.
|
|
| 6. |
- Zhang, Hongyi, 1996, et al.
(author)
-
End-to-End Federated Learning for Autonomous Driving Vehicles
- 2021
-
In: Proceedings of the International Joint Conference on Neural Networks. - : IEEE. - 9781665439008 - 9781665445979 ; 2021-July
-
Conference paper (peer-reviewed)abstract
- In recent years, with the development of computation capability in devices, companies are eager to investigate and utilize suitable ML/DL methods to improve their service quality. However, with the traditional learning strategy, companies need to first build up a powerful data center to collect and analyze data from the edge and then perform centralized model training, which turns out to be inefficient. Federated Learning has been introduced to solve this challenge. Because of its characteristics such as model-only exchange and parallel training, the technique can not only preserve user data privacy but also accelerate model training speed. The method can easily handle real-time data generated from the edge without taking up a lot of valuable network transmission resources. In this paper, we introduce an approach to end-to-end on-device Machine Learning by utilizing Federated Learning. We validate our approach with an important industrial use case in the field of autonomous driving vehicles, the wheel steering angle prediction. Our results show that Federated Learning can significantly improve the quality of local edge models and also reach the same accuracy level as compared to the traditional centralized Machine Learning approach without its negative effects. Furthermore, Federated Learning can accelerate model training speed and reduce the communication overhead, which proves that this approach has great strength when deploying ML/DL components to various real-world embedded systems.
|
|
| 7. |
- Zhang, Hongyi, 1996, et al.
(author)
-
Engineering Federated Learning Systems : A Literature Review
- 2021
-
In: Software Business. - Cham : Springer. - 9783030672911 - 9783030672928 ; 407, s. 210-218
-
Conference paper (peer-reviewed)abstract
- With the increasing attention on Machine Learning applications, more and more companies are involved in implementing AI components into their software products in order to improve the service quality. With the rapid growth of distributed edge devices, Federated Learning has been introduced as a distributed learning technique, which enables model training in a large decentralized network without exchanging collected edge data. The method can not only preserve sensitive user data privacy but also save a large amount of data transmission bandwidth and the budget cost of computation equipment. In this paper, we provide a state-of-the-art overview of the empirical results reported in the existing literature regarding Federated Learning. According to the problems they expressed and solved, we then categorize those deployments into different application domains, identify their challenges and then propose six open research questions.
|
|
| 8. |
- Zhang, Hongyi, 1996, et al.
(author)
-
Federated learning systems: Architecture alternatives
- 2020
-
In: Proceedings - Asia-Pacific Software Engineering Conference, APSEC. - : IEEE. - 1530-1362. ; 2020-December, s. 385-394, s. 385-394
-
Conference paper (peer-reviewed)abstract
- Machine Learning (ML) and Artificial Intelligence (AI) have increasingly gained attention in research and industry. Federated Learning, as an approach to distributed learning, shows its potential with the increasing number of devices on the edge and the development of computing power. However, most of the current Federated Learning systems apply a single-server centralized architecture, which may cause several critical problems, such as the single-point of failure as well as scaling and performance problems. In this paper, we propose and compare four architecture alternatives for a Federated Learning system, i.e. centralized, hierarchical, regional and decentralized architectures. We conduct the study by using two well-known data sets and measuring several system performance metrics for all four alternatives. Our results suggest scenarios and use cases which are suitable for each alternative. In addition, we investigate the trade-off between communication latency, model evolution time and the model classification performance, which is crucial to applying the results into real-world industrial systems.
|
|
| 9. |
- Zhang, Hongyi, 1996, et al.
(author)
-
Multi-Agent Reinforcement Learning in Dynamic Industrial Context
- 2023
-
In: Proceedings - International Computer Software and Applications Conference. - 0730-3157. ; 2023-June, s. 448-457
-
Conference paper (peer-reviewed)abstract
- Deep reinforcement learning has advanced signifi-cantly in recent years, and it is now used in embedded systems in addition to simulators and games. Reinforcement Learning (RL) algorithms are currently being used to enhance device operation so that they can learn on their own and offer clients better services. It has recently been studied in a variety of industrial applications. However, reinforcement learning, especially when controlling a large number of agents in an industrial environment, has been demonstrated to be unstable and unable to adapt to realistic situations when used in a real-world setting. To address this problem, the goal of this study is to enable multiple reinforcement learning agents to independently learn control policies on their own in dynamic industrial contexts. In order to solve the problem, we propose a dynamic multi-agent reinforcement learning (dynamic multi-RL) method along with adaptive exploration (AE) and vector-based action selection (VAS) techniques for accelerating model convergence and adapting to a complex industrial environment. The proposed algorithm is tested for validation in emergency situations within the telecommunications industry. In such circumstances, three unmanned aerial vehicles (UAV-BSs) are used to provide temporary coverage to mission-critical (MC) customers in disaster zones when the original serving base station (BS) is destroyed by natural disasters. The algorithm directs the participating agents automatically to enhance service quality. Our findings demonstrate that the proposed dynamic multi-RL algorithm can proficiently manage the learning of multiple agents and adjust to dynamic industrial environments. Additionally, it enhances learning speed and improves the quality of service.
|
|
| 10. |
- Zhang, Hongyi, 1996, et al.
(author)
-
QuaFedAsync: Quality-based Asynchronous Federated Learning for the Embedded Systems
- 2023
-
In: Proceedings - 2023 49th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2023. - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 70-73
-
Conference paper (peer-reviewed)abstract
- In recent years, Federated Learning, as an approach to distributed learning, has shown its potential with the increasing number of devices on the edge and the development of computing power. The method enables large-scale training on the device that creates the data but with the sensitive data remaining within the data's owner. In reality, however, the vast majority of enterprises have the problem of low data volume and poor model quality to support the implementation of Federated Learning methods. Learning quality assurance for edge devices is still the major issue which prevents Federated Learning to be applied in industrial contexts, especially in safety-critical applications. In this paper, we propose a quality-based asynchronous Federated Learning algorithm (QuaFedAsync) to address these challenges. We report on a study in which we used two well-known data sets, i.e., DDAD and KITTI datasets, and validate the proposed algorithm on an industrial use case concerned with monocular depth estimation in the automotive domain. Our results show that the proposed algorithm significantly improves the prediction performance compared to the commonly applied aggregation protocols while maintaining the same level of accuracy as centralized machine learning. Based on the results, we prove the learning efficiency and robustness when applying the algorithm to industrial scenarios.
|
|
| 11. |
- Zhang, Hongyi, 1996, et al.
(author)
-
Real-time end-to-end federated learning: An automotive case study
- 2021
-
In: Proceedings - 2021 IEEE 45th Annual Computers, Software, and Applications Conference, COMPSAC 2021. - : IEEE. ; , s. 459-468
-
Conference paper (peer-reviewed)abstract
- With the development and the increasing interests in ML/DL fields, companies are eager to apply Machine Learning/Deep Learning approaches to increase service quality and customer experience. Federated Learning was implemented as an effective model training method for distributing and accelerating time-consuming model training while protecting user data privacy. However, common Federated Learning approaches, on the other hand, use a synchronous protocol to conduct model aggregation, which is inflexible and unable to adapt to rapidly changing environments and heterogeneous hardware settings in real-world scenarios. In this paper, we present an approach to real-time end-to-end Federated Learning combined with a novel asynchronous model aggregation protocol. Our method is validated in an industrial use case in the automotive domain, focusing on steering wheel angle prediction for autonomous driving. Our findings show that asynchronous Federated Learning can significantly improve the prediction performance of local edge models while maintaining the same level of accuracy as centralized machine learning. Furthermore, by using a sliding training window, the approach can minimize communication overhead, accelerate model training speed and consume real-time streaming data, proving high efficiency when deploying ML/DL components to heterogeneous real-world embedded systems.
|
|
| 12. |
- Zhang, Hongyi, 1996, et al.
(author)
-
Towards Federated Learning: A Case Study in the Telecommunication Domain
- 2021
-
In: Lecture Notes in Business Information Processing. - Cham : Springer International Publishing. - 1865-1356 .- 1865-1348. ; 434 LNBIP, s. 238-253, s. 238-253
-
Conference paper (peer-reviewed)abstract
- Federated Learning, as a distributed learning technique, has emerged with the improvement of the performance of IoT and edge devices. The emergence of this learning method alters the situation in which data must be centrally uploaded to the cloud for processing and maximizes the utilization of edge devices’ computing and storage capabilities. The learning approach eliminates the need to upload large amounts of local data and reduces data transfer latency with local data processing. Since the Federated Learning technique does not require centralized data for model training, it is better suited to edge learning scenarios in which nodes have limited data. However, despite the fact that Federated Learning has significant benefits, we discovered that companies struggle with integrating Federated Learning components into their systems. In this paper, we present case study research that describes reasons why companies anticipate Federated Learning as an applicable technique. Secondly, we summarize the services that a complete Federated Learning system needs to support in industrial scenarios and then identify the key challenges for industries to adopt and transition to Federated Learning. Finally, based on our empirical findings, we suggest five criteria for companies implementing reliable Federated Learning systems.
|
|
| 13. |
- Zhang, Hongyi, 1996
(author)
-
Towards Real-World Federated Learning: Empirical Studies in the Domain of Embedded Systems
- 2022
-
Licentiate thesis (other academic/artistic)abstract
- Context: Artificial intelligence (AI) has led a new phase of technical revolution and industrial development around the world since the twenty-first century, revolutionizing the way of production. Artificial intelligence (AI), an emerging information technology, is thriving, and AI application technologies are gaining traction, particularly in professional services such as healthcare, education, finance, security, etc. More machine learning technologies have begun to be thoroughly applied to the production stage as big data and cloud computing capabilities have improved. With the increased focus on Machine Learning applications and the rapid growth of distributed edge devices in the industry, we believe that utilizing a large number of edge devices will become increasingly important. The introduction of Federated Learning changes the situation in which data must be centrally uploaded to the cloud for processing and maximizes the use of edge devices' computing and storage capabilities. With local data processing, the learning approach eliminates the need to upload large amounts of local data and reduces data transfer latency. Because Federated Learning does not require centralized data for model training, it is better suited to edge learning scenarios with limited data and privacy concerns. Objective: The purpose of this research is to identify the characteristics and problems of the Federated Learning methods, our new algorithms and frameworks that can assist companies in making the transition to Federated Learning, and empirically validate the proposed approaches. Method: To achieve these objectives, we adopted an empirical research approach with design science being our primary research method. We conducted a literature review, case studies, including semi-structured interviews and simulation experiments in close collaboration with software-intensive companies in the embedded systems domain. Results: We present four major findings in this paper. First, we present a state-of-the-art review of the empirical results reported in the existing Federated Learning literature. We then categorize those Federated Learning implementations into different application domains, identify their challenges, and propose six open research questions based on the problems identified in the literature. Second, we conduct a case study to explain why companies anticipate Federated Learning as a potential solution to the challenges they encountered when implementing machine learning components. We summarize the services that a comprehensive Federated Learning system must enable in industrial settings. Furthermore, we identify the primary barriers that companies must overcome in order to embrace and transition to Federated Learning. Based on our empirical findings, we propose five requirements for companies implementing reliable Federated Learning systems. Third, we develop and evaluate four architecture alternatives for a Federated Learning system, including centralized, hierarchical, regional, and decentralized architectures. We investigate the trade-o between communication latency, model evolution time, and model classification performance, which is critical for applying our findings to real-world industrial systems. Fourth, we introduce techniques and asynchronous frameworks for end-to-end on-device Federated Learning. The method is validated using a steering wheel angle prediction case. The local models of each edge vehicle can be continuously trained and shared with other vehicles to improve their local model prediction accuracy. Furthermore, we combine the asynchronous Federated Learning approach with Deep Neural Decision Forests and validate our method using important industry use cases in the automotive domain. Our findings show that Federated Learning can improve model training speed while lowering communication overhead without sacrificing accuracy, demonstrating that this technique has significant benefits to a wide range of real-world embedded systems. Future Work: In the future, we plan to test our approach in other use cases and look into more sophisticated neural networks integrated with our approach. In order to improve model training performance on resource-constrained edge devices in real-world embedded systems, we intend to design more appropriate aggregation methods and protocols. Furthermore, we intend to use the Federated Learning and Reinforcement Learning methods to assist the edge in evolving themselves autonomously and fully utilizing the computation capabilities of the edge devices.
|
|
