| 1. |
- Aaltonen, Harri, et al.
(författare)
-
Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning Approach
- 2022
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 15:14
-
Tidskriftsartikel (refereegranskat)abstract
- Battery storage is emerging as a key component of intelligent green electricitiy systems. The battery is monetized through market participation, which usually involves bidding. Bidding is a multiâ€objective optimization problem, involving targets such as maximizing market compensation and minimizing penalties for failing to provide the service and costs for battery aging. In this article, battery participation is investigated on primary frequency reserve markets. Reinforcement learning is applied for the optimization. In previous research, only simplified formulations of battery aging have been used in the reinforcement learning formulation, so it is unclear how the optimizer would perform with a real battery. In this article, a physicsâ€based battery aging model is used to assess the aging. The contribution of this article is a methodology involving a realistic battery simulation to assess the performance of the trained RL agent with respect to battery aging in order to inform the selection of the weighting of the aging term in the RL reward formula. The RL agent performs day-ahead bidding on the Finnish Frequency Containment Reserves for Normal Operation market, with the objective of maximizing market compensation, minimizing market penalties and minimizing aging costs.
|
|
| 2. |
- Arndt, Karol, et al.
(författare)
-
Meta Reinforcement Learning for Sim-to-real Domain Adaptation
- 2020
-
Ingår i: 2020 IEEE International Conference On Robotics And Automation (ICRA). - : IEEE. ; , s. 2725-2731
-
Konferensbidrag (refereegranskat)abstract
- Modern reinforcement learning methods suffer from low sample efficiency and unsafe exploration, making it infeasible to train robotic policies entirely on real hardware. In this work, we propose to address the problem of sim-to-real domain transfer by using meta learning to train a policy that can adapt to a variety of dynamic conditions, and using a task-specific trajectory generation model to provide an action space that facilitates quick exploration. We evaluate the method by performing domain adaptation in simulation and analyzing the structure of the latent space during adaptation. We then deploy this policy on a KUKA LBR 4+ robot and evaluate its performance on a task of hitting a hockey puck to a target. Our method shows more consistent and stable domain adaptation than the baseline, resulting in better overall performance.
|
|
| 3. |
- Bekiroglu, Yasemin, et al.
(författare)
-
Assessing Grasp Stability Based on Learning and Haptic Data
- 2011
-
Ingår i: IEEE Transactions on robotics. - : IEEE Robotics and Automation Society. - 1552-3098 .- 1941-0468. ; 27:3, s. 616-629
-
Tidskriftsartikel (refereegranskat)abstract
- An important ability of a robot that interacts with the environment and manipulates objects is to deal with the uncertainty in sensory data. Sensory information is necessary to, for example, perform online assessment of grasp stability. We present methods to assess grasp stability based on haptic data and machinelearning methods, including AdaBoost, support vector machines (SVMs), and hidden Markov models (HMMs). In particular, we study the effect of different sensory streams to grasp stability. This includes object information such as shape; grasp information such as approach vector; tactile measurements fromfingertips; and joint configuration of the hand. Sensory knowledge affects the success of the grasping process both in the planning stage (before a grasp is executed) and during the execution of the grasp (closed-loop online control). In this paper, we study both of these aspects. We propose a probabilistic learning framework to assess grasp stability and demonstrate that knowledge about grasp stability can be inferred using information from tactile sensors. Experiments on both simulated and real data are shown. The results indicate that the idea to exploit the learning approach is applicable in realistic scenarios, which opens a number of interesting venues for the future research.
|
|
| 4. |
|
|
| 5. |
|
|
| 6. |
- Bekiroglu, Yasemin, 1982, et al.
(författare)
-
Learning grasp stability based on haptic data
- 2010
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- Grasping is an essential skill for a general purpose service robot, working in an industrial or home-like environment. The classical work in robotic grasping assumes that the object parameters such as pose, shape, weight and material properties are known. If precise knowledge of these is available, grasp planning using analytical approaches, such as form or force closure, may be enough for successful grasp execution. However, in unstructured environments the information is usually uncertain, which presents a great challenge for the current state-of-the-art work in this area. Sensors can be used to alleviate the problem of uncertainty. To determine the shape and pose of an object, vision has been commonly used. However, the accuracy of vision is limited and small errors in object pose are frequent even for known objects. It is not uncommon that even these small errors cause failures in grasping. These failures are also difficult to prevent at the grasp planning stage. This problem is magnified when also the object models are acquired on-line using vision or other similar sensors. While the tactile and finger force sensors can be used to reduce this problem, a grasp may fail even when all fingers have adequate contact forces and the hand pose is not dramatically different from the planned one. The main contribution of this paper is to show that it is possible to infer knowledge about grasp stability using information from tactile sensors while grasping an object before being further manipulated. This is very useful, because if failures can be detected, objects can be regrasped before trying to lift them. However, the relationship between tactile measurements and grasp stability is embodiment specific and very complex. For this reason, we propose to use machine earning techniques for the inference.
|
|
| 7. |
- Bekiroglu, Yasemin, et al.
(författare)
-
Learning grasp stability based on tactile data and HMMs
- 2010
-
Konferensbidrag (refereegranskat)abstract
- In this paper, the problem of learning grasp stability in robotic object grasping based on tactile measurements is studied. Although grasp stability modeling and estimation has been studied for a long time, there are few robots today able of demonstrating extensive grasping skills. The main contribution of the work presented here is an investigation of probabilistic modeling for inferring grasp stability based on learning from examples. The main objective is classification of a grasp as stable or unstable before applying further actions on it, e.g. lifting. The problem cannot be solved by visual sensing which is typically used to execute an initial robot hand positioning with respect to the object. The output of the classification system can trigger a regrasping step if an unstable grasp is identified. An off-line learning process is implemented and used for reasoning about grasp stability for a three-fingered robotic hand using Hidden Markov models. To evaluate the proposed method, experiments are performed both in simulation and on a real robot system.
|
|
| 8. |
- Bekiroglu, Yasemin, 1982, et al.
(författare)
-
Learning grasp stability with tactile data and HMMs
- 2010
-
Ingår i: IEEE International Symposium on Robot and Human Interactive Communication. - 1944-9437 .- 1944-9445. ; , s. 132-137
-
Konferensbidrag (refereegranskat)abstract
- In this paper, the problem of learning grasp stability in robotic object grasping based on tactile measurements is studied. Although grasp stability modeling and estimation has been studied for a long time, there are few robots today able of demonstrating extensive grasping skills. The main contribution of the work presented here is an investigation of probabilistic modeling for inferring grasp stability based on learning from examples. The main objective is classification of a grasp as stable or unstable before applying further actions on it, e.g. lifting. The problem cannot be solved by visual sensing which is typically used to execute an initial robot hand positioning with respect to the object. The output of the classification system can trigger a regrasping step if an unstable grasp is identified. An off-line learning process is implemented and used for reasoning about grasp stability for a three-fingered robotic hand using Hidden Markov models. To evaluate the proposed method, experiments are performed both in simulation and on a real robot system.
|
|
| 9. |
- Chen, Xi
(författare)
-
Data-Efficient Reinforcement and Transfer Learning in Robotics
- 2020
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- In the past few years, deep reinforcement learning (RL) has shown great potential in learning action selection policies for solving different tasks.Despite its impressive success in games, several challenges remain, such as designing appropriate reward functions, collecting large amounts of interactive data, and dealing with unseen cases, which make it difficult to apply RL algorithms to real-world robotics tasks. The ability of data-efficient learning and rapid adaptation to novel cases is essential for an RL agent to solve real-world problems.In this thesis, we discuss algorithms to address the challenges in RL by reusing past experiences gained while learning other tasks to improve the efficiency of learning new tasks.Instead of learning directly from the target task, which is complicated and sometimes unavailable during training, we propose first learning from relevant tasks that contain valuable information about the target environment, and reuse the obtained solutions in solving the target task.We follow two approaches to achieve knowledge sharing between tasks. In the first approach, we model the problem as a transfer learning problem and learn to minimize the distance between the representations found based on the training and the target data, such that the learned solution can be applied to the target task using a small amount of data from the target environment.In the second approach, we formulate it as a meta-learning problem and obtain a model that is explicitly trained for rapid adaptation using a small amount of data. At test time, we can learn quickly on top of the trained model in a few iterations when facing a new task.We demonstrate the effectiveness of the proposed frameworks by evaluating the methods on a number of real and simulated robotic tasks, including robot navigation, motion control, and manipulation. We show how these methods can be applied to challenging tasks with high-dimensional state/action spaces, limited data, sparse rewards, and requiring diverse skills.
|
|
| 10. |
- Ghadirzadeh, Ali, 1987-
(författare)
-
Sensorimotor Robot Policy Training using Reinforcement Learning
- 2018
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Robots are becoming more ubiquitous in our society and taking over many tasks that were previously considered as human hallmarks. Many of these tasks, e.g., autonomously driving a car, collaborating with humans in dynamic and changing working conditions and performing household chores, require human-level intelligence to perceive the world and to act appropriately. In this thesis, we pursue a different approach compared to classical methods that often construct a robot controller based on the perception-then-action paradigm. We devise robotic action-selection policies by considering action-selection and perception processes as being intertwined, emphasizing that perception comes prior to action and action is key to perception. The main hypothesis is that complex robotic behaviors come as the result of mastering sensorimotor contingencies (SMCs), i.e., regularities between motor actions and associated changes in sensory observations, where SMCs can be seen as building blocks to skillful behaviors. We elaborate and investigate this hypothesis by deliberate design of frameworks which enable policy training merely based on data experienced by a robot,without intervention of human experts for analytical modelings or calibrations. In such circumstances, action policies can be obtained by reinforcement learning (RL) paradigm by making exploratory action decisions and reinforcing patterns of SMCs that lead to reward events for a given task. However, the dimensionality of sensorimotor spaces, complex dynamics of physical tasks, sparseness of reward events, limited amount of data from real-robot experiments, ambiguities of crediting past decisions and safety issues, which arise from exploratory actions of a physical robot, pose challenges to obtain a policy based on data-driven methods alone. In this thesis, we introduce our contributions to deal with the aforementioned issues by devising learning frameworks which endow a robot with the ability to integrate sensorimotor data to obtain action-selection policies. The effectiveness of the proposed frameworks is demonstrated by evaluating the methods on a number of real robotic tasks and illustrating the suitability of the methods to acquire different skills, to make sequential action-decisions in high-dimensional sensorimotor spaces, with limited data and sparse rewards.
|
|
