| 1. |
|
|
| 2. |
- Iucci, Alessandro, et al.
(författare)
-
Explainable Reinforcement Learning for Human-Robot Collaboration
- 2021
-
Ingår i: 2021 20Th International Conference On Advanced Robotics (ICAR). - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 927-934
-
Konferensbidrag (refereegranskat)abstract
- Reinforcement learning (RL) is getting popular in the robotics field due to its nature to learn from dynamic environments. However, it is unable to provide explanations of why an output was generated. Explainability becomes therefore important in situations where humans interact with robots, such as in human-robot collaboration (HRC) scenarios. Attempts to address explainability in robotics usually are restricted to explain a specific decision taken by the RL model, but not to understand the complete behavior of the robot. In addition, the explainability methods are restricted to be used by domain experts as queries and responses are not translated to natural language. This work overcomes these limitations by proposing an explainability solution for RL models applied to HRC. It is mainly formed by the adaptation of two methods: (i) Reward decomposition gives an insight into the factors that impacted the robot's choice by decomposing the reward function. It further provides sets of relevant reasons for each decision taken during the robot's operation; (ii) Autonomous policy explanation provides a global explanation of the robot's behavior by answering queries in the form of natural language, thus making understandable to any human user. Experiments in simulated HRC scenarios revealed an increased understanding of the optimal choices made by the robots. Additionally, our solution demonstrated as a powerful debugging tool to find weaknesses in the robot's policy and assist in its improvement.
|
|
| 3. |
- Ruggeri, Franco, et al.
(författare)
-
Evaluation of Intrinsic Explainable Reinforcement Learning in Remote Electrical Tilt Optimization
- 2024
-
Ingår i: Proceedings of 8th International Congress on Information and Communication Technology - ICICT 2023. - : Springer Nature. ; , s. 835-854
-
Konferensbidrag (refereegranskat)abstract
- This paper empirically evaluates two intrinsic Explainable Reinforcement Learning (XRL) algorithms on the Remote Electrical Tilt (RET) optimization problem. In RET optimization, where the electrical downtilt of the antennas in a cellular network is controlled to optimize coverage and capacity, explanations are necessary to understand the reasons behind a specific adjustment. First, we formulate the RET problem in the reinforcement learning (RL) framework and describe how we apply Decomposed Reward Deep Q Network (drDQN) and Linear ModelU-Tree (LMUT), which are two state-of-the-art XRL algorithms. Then, we train and test such agents in a realistic simulated network. Our results highlight both advantages and disadvantages of the algorithms. DrDQN provides intuitive contrastive local explanations for the agent’s decisions to adjust the downtilt of an antenna, while achieving the same performance as the original DQN algorithm. LMUT reaches high performance while employing a fully transparent linear model capable of generating both local and global explanations. On the other hand, drDQN adds a constraint on the reward design that might be problematic for the specification of the objective, whereas LMUT could generate misleading global feature importance and needs additional developments to provide more user-interpretable local explanations.
|
|
| 4. |
- Ruggeri, Franco, et al.
(författare)
-
Safety-based Dynamic Task Offloading for Human-Robot Collaboration using Deep Reinforcement Learning
- 2022
-
Ingår i: 2022 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS). - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 2119-2126
-
Konferensbidrag (refereegranskat)abstract
- Robots with constrained hardware resources usually rely on Multi-access Edge Computing infrastructures to offload computationally expensive tasks to meet real-time and safety requirements. Offloading every task might not be the best option due to dynamic changes in the network conditions and can result in network congestion or failures. This work proposes a task offloading strategy for mobile robots in a Human-Robot Collaboration scenario that optimizes the edge resource usage and reduces network delays, leading to safety enhancement. The solution utilizes a Deep Reinforcement Learning (DRL) agent that observes safety and network metrics to dynamically decide at runtime if (i) a less accurate model should run on the robot; (ii) a more complex model should run on the edge; or (iii) the previous output should be reused through temporal coherence verification. Experiments are performed in a simulated warehouse where humans and robots have close interactions and safety needs are high. Our results show that the proposed DRL solution outperforms the baselines in several aspects. The edge is used only when the network performance is reliable, reducing the number of failures (up to 47%). The latency is not only decreased (up to 68%) but also adapted to the safety requirements (risk x latency reduced up to 48%), avoiding unnecessary network congestion in safe situations and letting other devices use the network. Overall, the safety metrics get improved, such as the increased time in the safe zone by up to 3.1%.
|
|
| 5. |
- Terra, Ahmad, et al.
(författare)
-
BEERL : Both Ends Explanations for Reinforcement Learning
- 2022
-
Ingår i: Applied Sciences. - : MDPI AG. - 2076-3417. ; 12:21, s. 10947-
-
Tidskriftsartikel (refereegranskat)abstract
- Deep Reinforcement Learning (RL) is a black-box method and is hard to understand because the agent employs a neural network (NN). To explain the behavior and decisions made by the agent, different eXplainable RL (XRL) methods are developed; for example, feature importance methods are applied to analyze the contribution of the input side of the model, and reward decomposition methods are applied to explain the components of the output end of the RL model. In this study, we present a novel method to connect explanations from both input and output ends of a black-box model, which results in fine-grained explanations. Our method exposes the reward prioritization to the user, which in turn generates two different levels of explanation and allows RL agent reconfigurations when unwanted behaviors are observed. The method further summarizes the detailed explanations into a focus value that takes into account all reward components and quantifies the fulfillment of the explanation of desired properties. We evaluated our method by applying it to a remote electrical telecom-antenna-tilt use case and two openAI gym environments: lunar lander and cartpole. The results demonstrated fine-grained explanations by detailing input features’ contributions to certain rewards and revealed biases of the reward components, which are then addressed by adjusting the reward’s weights.
|
|
| 6. |
- Terra, Ahmad, et al.
(författare)
-
Using Counterfactuals to Proactively Solve Service Level Agreement Violations in 5G Networks
- 2022
-
Ingår i: 2022 IEEE 20TH INTERNATIONAL CONFERENCE ON INDUSTRIAL INFORMATICS (INDIN). - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 552-559
-
Konferensbidrag (refereegranskat)abstract
- A main challenge of using 5G network slices is to meet all the quality of service requirements of the slices (which are agreed with the customer in a service level agreement (SLA)), throughout the network slices' lifecycle. To avoid the penalty for violation, a proactive solution is presented, including predicting the SLA violation, explaining the violation cause, and then providing an adaptation to traffic. This work uses counterfactual (CF) explanations to 1) explain the main factors affecting the identified model's SLA violation prediction and 2) present modifications in the input values, which are required to configure the network traffic to avoid such a violation. We evaluate the CF explanation at two different levels where the generated CF instance is fed to the predictive model, and then actuation data are generated to evaluate the result in the real network. Our solution minimizes the violation up to 98%. This information can be utilized to reconfigure the system, either by humans or by the system automatically, to make it fully autonomous on the one hand and comply with the 'right to explanation' introduced by the European Union's General Data Protection Regulation on the other hand.
|
|
