| 1. |
- Yin, Wenjie
(författare)
-
Developing Data-Driven Models for Understanding Human Motion
- 2024
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Humans are the primary subjects of interest in the realm of computer vision. Specifically, perceiving, generating, and understanding human activities have long been a core pursuit of machine intelligence. Over the past few decades, data-driven methods for modeling human motion have demonstrated great potential across various interactive media and social robotics domains. Despite its impressive achievements, challenges still remain in analyzing multi-agent/multi-modal behaviors and in producing high-fidelity and highly varied motions. This complexity arises because human motion is inherently dynamic, uncertain, and intertwined with its environment. This thesis aims to introduce challenges and data-driven methods of understanding human motion and then elaborate on the contributions of the included papers. We present this thesis mainly in ascending order of complexity: recognition, synthesis, and transfer, which includes the tasks of perceiving, generating, and understanding human activities. Firstly, we present methods to recognize human motion (Paper A). We consider a conversational group scenario where people gather and stand in an environment to converse. Based on transformer-based networks and graph convolutional neural networks, we demonstrate how spatial-temporal group dynamics can be modeled and perceived on both the individual and group levels. Secondly, we investigate probabilistic autoregressive approaches to generate controllable human locomotion. We employ deep generative models, namely normalizing flows (Paper B) and diffusion models (Paper C), to generate and reconstruct the 3D skeletal poses of humans over time. Finally, we deal with the problem of motion style transfer. We propose style transfer systems that allow transforming motion styles while attempting to preserve motion context through GAN-based (Paper D) and diffusion-based (Paper E) methods. Compared with previous research mainly focusing on simple locomotion or exercise, we consider more complex dance movements and multimodal information. In summary, this thesis aims to propose methods that can effectively perceive, generate, and transfer 3D human motion. In terms of network architectures, we employ graph formulation to exploit the correlation of human skeletons, thereby introducing inductive bias through graph structures. Additionally, we leverage transformers to handle long-term data dependencies and weigh the importance of varying data components. In terms of learning frameworks, we adopt generative models to represent joint distribution over relevant variables and multiple modalities, which are flexible to cover a wide range of tasks. Our experiments demonstrate the effectiveness of the proposed frameworks by evaluating the methods on our own collected dataset and public datasets. We show how these methods are applied to various challenging tasks.
|
|
| 2. |
- Abdul Khader, Shahbaz
(författare)
-
Data-Driven Methods for Contact-Rich Manipulation: Control Stability and Data-Efficiency
- 2021
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Autonomous robots are expected to make a greater presence in the homes and workplaces of human beings. Unlike their industrial counterparts, autonomous robots have to deal with a great deal of uncertainty and lack of structure in their environment. A remarkable aspect of performing manipulation in such a scenario is the possibility of physical contact between the robot and the environment. Therefore, not unlike human manipulation, robotic manipulation has to manage contacts, both expected and unexpected, that are often characterized by complex interaction dynamics.Skill learning has emerged as a promising approach for robots to acquire rich motion generation capabilities. In skill learning, data driven methods are used to learn reactive control policies that map states to actions. Such an approach is appealing because a sufficiently expressive policy can almost instantaneously generate appropriate control actions without the need for computationally expensive search operations. Although reinforcement learning (RL) is a natural framework for skill learning, its practical application is limited for a number of reasons. Arguably, the two main reasons are the lack of guaranteed control stability and poor data-efficiency. While control stability is necessary for ensuring safety and predictability, data-efficiency is required for achieving realistic training times. In this thesis, solutions are sought for these two issues in the context of contact-rich manipulation.First, this thesis addresses the problem of control stability. Despite unknown interaction dynamics during contact, skill learning with stability guarantee is formulated as a model-free RL problem. The thesis proposes multiple solutions for parameterizing stability-aware policies. Some policy parameterizations are partly or almost wholly deep neural networks. This is followed by policy search solutions that preserve stability during random exploration, if required. In one case, a novel evolution strategies-based policy search method is introduced. It is shown, with the help of real robot experiments, that Lyapunov stability is both possible and beneficial for RL-based skill learning.Second, this thesis addresses the issue of data-efficiency. Although data-efficiency is targeted by formulating skill learning as a model-based RL problem, only the model learning part is addressed. In addition to benefiting from the data-efficiency and uncertainty representation of the Gaussian process, this thesis further investigates the benefits of adopting the structure of hybrid automata for learning forward dynamics models. The method also includes an algorithm for predicting long-term trajectory distributions that can represent discontinuities and multiple modes. The proposed method is shown to be more data-efficient than some state-of-the-art methods.
|
|
| 3. |
- Bekiroglu, Yasemin
(författare)
-
Learning to Assess Grasp Stability from Vision, Touch and Proprioception
- 2012
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Grasping and manipulation of objects is an integral part of a robot’s physical interaction with the environment. In order to cope with real-world situations, sensor based grasping of objects and grasp stability estimation is an important skill. This thesis addresses the problem of predicting the stability of a grasp from the perceptions available to a robot once fingers close around the object before attempting to lift it. A regrasping step can be triggered if an unstable grasp is identified. The percepts considered consist of object features (visual), gripper configurations (proprioceptive) and tactile imprints (haptic) when fingers contact the object. This thesis studies tactile based stability estimation by applying machine learning methods such as Hidden Markov Models. An approach to integrate visual and tactile feedback is also introduced to further improve the predictions of grasp stability, using Kernel Logistic Regression models.Like humans, robots are expected to grasp and manipulate objects in a goal-oriented manner. In other words, objects should be grasped so to afford subsequent actions: if I am to hammer a nail, the hammer should be grasped so to afford hammering. Most of the work on grasping commonly addresses only the problem of finding a stable grasp without considering the task/action a robot is supposed to fulfill with an object. This thesis also studies grasp stability assessment in a task-oriented way based on a generative approach using probabilistic graphical models, Bayesian Networks. We integrate high-level task information introduced by a teacher in a supervised setting with low-level stability requirements acquired through a robot’s exploration. The graphical model is used to encode probabilistic relationships between tasks and sensory data (visual, tactile and proprioceptive). The generative modeling approach enables inference of appropriate grasping configurations, as well as prediction of grasp stability. Overall, results indicate that the idea of exploiting learning approaches for grasp stability assessment is applicable in realistic scenarios.
|
|
| 4. |
- Bergström, Niklas, 1978-
(författare)
-
Interactive Perception : From Scenes to Objects
- 2012
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis builds on the observation that robots, like humans, do not have enough experience to handle all situations from the start. Therefore they need tools to cope with new situations, unknown scenes and unknown objects. In particular, this thesis addresses objects. How can a robot realize what objects are if it looks at a scene and has no knowledge about objects? How can it recover from situations where its hypotheses about what it sees are wrong? Even if it has built up experience in form of learned objects, there will be situations where it will be uncertain or mistaken, and will therefore still need the ability to correct errors. Much of our daily lives involves interactions with objects, and the same will be true robots existing among us. Apart from being able to identify individual objects, the robot will therefore need to manipulate them.Throughout the thesis, different aspects of how to deal with these questions is addressed. The focus is on the problem of a robot automatically partitioning a scene into its constituting objects. It is assumed that the robot does not know about specific objects, and is therefore considered inexperienced. Instead a method is proposed that generates object hypotheses given visual input, and then enables the robot to recover from erroneous hypotheses. This is done by the robot drawing from a human's experience, as well as by enabling it to interact with the scene itself and monitoring if the observed changes are in line with its current beliefs about the scene's structure.Furthermore, the task of object manipulation for unknown objects is explored. This is also used as a motivation why the scene partitioning problem is essential to solve. Finally aspects of monitoring the outcome of a manipulation is investigated by observing the evolution of flexible objects in both static and dynamic scenes. All methods that were developed for this thesis have been tested and evaluated on real robotic platforms. These evaluations show the importance of having a system capable of recovering from errors and that the robot can take advantage of human experience using just simple commands.
|
|
| 5. |
- Bohg, Jeannette, 1981-
(författare)
-
Multi-Modal Scene Understanding for Robotic Grasping
- 2011
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Current robotics research is largely driven by the vision of creatingan intelligent being that can perform dangerous, difficult orunpopular tasks. These can for example be exploring the surface of planet mars or the bottomof the ocean, maintaining a furnace or assembling a car. They can also be more mundane such as cleaning an apartment or fetching groceries. This vision has been pursued since the 1960s when the first robots were built. Some of the tasks mentioned above, especially those in industrial manufacturing, arealready frequently performed by robots. Others are still completelyout of reach. Especially, household robots are far away from beingdeployable as general purpose devices. Although advancements have beenmade in this research area, robots are not yet able to performhousehold chores robustly in unstructured and open-ended environments givenunexpected events and uncertainty in perception and execution.In this thesis, we are analyzing which perceptual andmotor capabilities are necessaryfor the robot to perform common tasks in a household scenario. In that context, an essential capability is tounderstand the scene that the robot has to interact with. This involvesseparating objects from the background but also from each other.Once this is achieved, many other tasks becomemuch easier. Configuration of objectscan be determined; they can be identified or categorized; their pose can be estimated; free and occupied space in the environment can be outlined.This kind of scene model can then inform grasp planning algorithms to finally pick up objects.However, scene understanding is not a trivial problem and evenstate-of-the-art methods may fail. Given an incomplete, noisy andpotentially erroneously segmented scene model, the questions remain howsuitable grasps can be planned and how they can be executed robustly.In this thesis, we propose to equip the robot with a set of predictionmechanisms that allow it to hypothesize about parts of the sceneit has not yet observed. Additionally, the robot can alsoquantify how uncertain it is about this prediction allowing it toplan actions for exploring the scene at specifically uncertainplaces. We consider multiple modalities includingmonocular and stereo vision, haptic sensing and information obtainedthrough a human-robot dialog system. We also study several scene representations of different complexity and their applicability to a grasping scenario. Given an improved scene model from this multi-modalexploration, grasps can be inferred for each objecthypothesis. Dependent on whether the objects are known, familiar orunknown, different methodologies for grasp inference apply. In thisthesis, we propose novel methods for each of these cases. Furthermore,we demonstrate the execution of these grasp both in a closed andopen-loop manner showing the effectiveness of the proposed methods inreal-world scenarios.
|
|
| 6. |
- Bratt, Mattias, 1973-
(författare)
-
Teleoperation with significant dynamics
- 2009
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The subject of this thesis is teleoperation, and especially teleoperation with demanding time constraints due to significant dynamics inherent in the task. A comprehensive background is given, describing many aspects of teleoperation, from history and applications to operator interface hardware and relevant control theory concepts. Then follows a presentation of the research done by the author. Two prototypical highly dynamic teleoperation tasks have been attempted: high speed driving, and ball catching. Systems have been developed for both, employing operator interfaces tailored to facilitate perception of the remote scene and including assistive features to promote successful task completion within the required time frame. Prediction of the state at the remote site as well as of operator action has been applied to address the problem of delays arising when using the Internet as the communication channel.
|
|
| 7. |
- Varava, Anastasiia, 1992-
(författare)
-
Path-Connectivity of the Free Space : Caging and Path Existence
- 2019
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The notion of configuration space is a tool that allows to reason aboutan object’s mobility in a unified manner. The problem of verifying path non-existence can be considered as dual to path planning. The question here iswhether a body (a robot, a vehicle, or an object) can move between start andgoal configurations without colliding with obstacles. Caging is a notion fromrobotic manipulation that can be seen as a special case of this problem: anobject is caged when it cannot escape arbitrarily far from its initial position.In this thesis, we address the problems of caging and path non-existence indifferent settings. Firstly, we design a theoretical framework and verificationalgorithms for caging of three-dimensional partially-deformable objects withspecific global geometric features that can be described as narrow parts. Sec-ondly, we formulate and address the problem of herding by caging: given agroup of moving agents and a team of mobile robots, the task is to guide theagents to a predefined goal region without letting them escape at any mo-ment of time. Thirdly, we propose an algorithm for efficient approximationof three- and six-dimensional configuration spaces of arbitrary rigid objects.This approximation can be later used to identify caging configurations as wellas to verify path existence between given configurations. Finally, we reportour preliminary results on molecular caging screening. This project buildsupon our previous work on configuration space approximation.
|
|
| 8. |
- Welle, Michael C.
(författare)
-
Learning Structured Representations for Rigid and Deformable Object Manipulation
- 2021
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The performance of learning based algorithms largely depends on the given representation of data. Therefore the questions arise, i) how to obtain useful representations, ii) how to evaluate representations, and iii) how to leverage these representations in a real-world robotic setting. In this thesis, we aim to answer all three of this questions in order to learn structured representations for rigid and deformable object manipulation. We firstly take a look into how to learn structured representation and show that imposing structure, informed from task priors, into the representation space is beneficial for certain robotic tasks. Furthermore we discuss and present suitable evaluation practices for structured representations as well as a benchmark for bimanual cloth manipulation. Finally, we introduce the Latent SpaceRoadmap (LSR) framework for visual action planning, where raw observations are mapped into a lower-dimensional latent space. Those are connected via the LSR, and visual action plans are generated that are able to perform a wide range of tasks. The framework is validated on a simulated rigid box stacking task, a simulated hybrid rope-box manipulation task, and a T-shirt folding task performed on a real robotic system.
|
|
| 9. |
- Antonova, Rika
(författare)
-
Transfer-Aware Kernels, Priors and Latent Spaces from Simulation to Real Robots
- 2020
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Consider challenging sim-to-real cases lacking high-fidelity simulators and allowing only 10-20 hardware trials. This work shows that even imprecise simulation can be beneficial if used to build transfer-aware representations.First, the thesis introduces an informed kernel that embeds the space of simulated trajectories into a lower-dimensional space of latent paths. It uses a sequential variational autoencoder (sVAE) to handle large-scale training from simulated data. Its modular design enables quick adaptation when used for Bayesian optimization (BO) on hardware. The thesis and the included publications demonstrate that this approach works for different areas of robotics: locomotion and manipulation. Furthermore, a variant of BO that ensures recovery from negative transfer when using corrupted kernels is introduced. An application to task-oriented grasping validates its performance on hardware.For the case of parametric learning, simulators can serve as priors or regularizers. This work describes how to use simulation to regularize a VAE's decoder to bind the VAE's latent space to simulator parameter posterior. With that, training on a small number of real trajectories can quickly shift the posterior to reflect reality. The included publication demonstrates that this approach can also help reinforcement learning (RL) quickly overcome the sim-to-real gap on a manipulation task on hardware.A longer-term vision is to shape latent spaces without needing to mandate a particular simulation scenario. A first step is to learn general relations that hold on sequences of states from a set of related domains. This work introduces a unifying mathematical formulation for learning independent analytic relations. Relations are learned from source domains, then used to help structure the latent space when learning on target domains. This formulation enables a more general, flexible and principled way of shaping the latent space. It formalizes the notion of learning independent relations, without imposing restrictive simplifying assumptions or requiring domain-specific information. This work presents mathematical properties, concrete algorithms and experimental validation of successful learning and transfer of latent relations.
|
|
| 10. |
- Arnekvist, Isac, 1986-
(författare)
-
Transfer Learning using low-dimensional Representations in Reinforcement Learning
- 2020
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Successful learning of behaviors in Reinforcement Learning (RL) are often learned tabula rasa, requiring many observations and interactions in the environment. Performing this outside of a simulator, in the real world, often becomes infeasible due to the large amount of interactions needed. This has motivated the use of Transfer Learning for Reinforcement Learning, where learning is accelerated by using experiences from previous learning in related tasks. In this thesis, I explore how we can transfer from a simple single-object pushing policy, to a wide array of non-prehensile rearrangement problems. I then explain how we can model task differences using a low-dimensional latent variable representation to make adaption to novel tasks efficient. Lastly, the dependence of accurate function approximation is sometimes problematic, especially in RL, where statistics of target variables are not known a priori. I present observations, along with explanations, that small target variances along with momentum optimization of ReLU-activated neural network parameters leads to dying ReLU.
|
|
| 11. |
- Billing, Erik, 1981-
(författare)
-
Cognition Rehearsed : Recognition and Reproduction of Demonstrated Behavior
- 2012
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The work presented in this dissertation investigates techniques for robot Learning from Demonstration (LFD). LFD is a well established approach where the robot is to learn from a set of demonstrations. The dissertation focuses on LFD where a human teacher demonstrates a behavior by controlling the robot via teleoperation. After demonstration, the robot should be able to reproduce the demonstrated behavior under varying conditions. In particular, the dissertation investigates techniques where previous behavioral knowledge is used as bias for generalization of demonstrations. The primary contribution of this work is the development and evaluation of a semi-reactive approach to LFD called Predictive Sequence Learning (PSL). PSL has many interesting properties applied as a learning algorithm for robots. Few assumptions are introduced and little task-specific configuration is needed. PSL can be seen as a variable-order Markov model that progressively builds up the ability to predict or simulate future sensory-motor events, given a history of past events. The knowledge base generated during learning can be used to control the robot, such that the demonstrated behavior is reproduced. The same knowledge base can also be used to recognize an on-going behavior by comparing predicted sensor states with actual observations. Behavior recognition is an important part of LFD, both as a way to communicate with the human user and as a technique that allows the robot to use previous knowledge as parts of new, more complex, controllers. In addition to the work on PSL, this dissertation provides a broad discussion on representation, recognition, and learning of robot behavior. LFD-related concepts such as demonstration, repetition, goal, and behavior are defined and analyzed, with focus on how bias is introduced by the use of behavior primitives. This analysis results in a formalism where LFD is described as transitions between information spaces. Assuming that the behavior recognition problem is partly solved, ways to deal with remaining ambiguities in the interpretation of a demonstration are proposed. The evaluation of PSL shows that the algorithm can efficiently learn and reproduce simple behaviors. The algorithm is able to generalize to previously unseen situations while maintaining the reactive properties of the system. As the complexity of the demonstrated behavior increases, knowledge of one part of the behavior sometimes interferes with knowledge of another parts. As a result, different situations with similar sensory-motor interactions are sometimes confused and the robot fails to reproduce the behavior. One way to handle these issues is to introduce a context layer that can support PSL by providing bias for predictions. Parts of the knowledge base that appear to fit the present context are highlighted, while other parts are inhibited. Which context should be active is continually re-evaluated using behavior recognition. This technique takes inspiration from several neurocomputational models that describe parts of the human brain as a hierarchical prediction system. With behavior recognition active, continually selecting the most suitable context for the present situation, the problem of knowledge interference is significantly reduced and the robot can successfully reproduce also more complex behaviors.
|
|
| 12. |
- Caccamo, Sergio, 1987-
(författare)
-
Enhancing geometric maps through environmental interactions
- 2018
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The deployment of rescue robots in real operations is becoming increasingly commonthanks to recent advances in AI technologies and high performance hardware. Rescue robots can now operate for extended period of time, cover wider areas andprocess larger amounts of sensory information making them considerably more usefulduring real life threatening situations, including both natural or man-made disasters.In this thesis we present results of our research which focuses on investigating ways of enhancing visual perception for Unmanned Ground Vehicles (UGVs) through environmental interactions using different sensory systems, such as tactile sensors and wireless receivers.We argue that a geometric representation of the robot surroundings built upon vision data only, may not suffice in overcoming challenging scenarios, and show that robot interactions with the environment can provide a rich layer of new information that needs to be suitably represented and merged into the cognitive world model. Visual perception for mobile ground vehicles is one of the fundamental problems in rescue robotics. Phenomena such as rain, fog, darkness, dust, smoke and fire heavily influence the performance of visual sensors, and often result in highly noisy data, leading to unreliable or incomplete maps.We address this problem through a collection of studies and structure the thesis as follow:Firstly, we give an overview of the Search & Rescue (SAR) robotics field, and discuss scenarios, hardware and related scientific questions.Secondly, we focus on the problems of control and communication. Mobile robotsrequire stable communication with the base station to exchange valuable information. Communication loss often presents a significant mission risk and disconnected robotsare either abandoned, or autonomously try to back-trace their way to the base station. We show how non-visual environmental properties (e.g. the WiFi signal distribution) can be efficiently modeled using probabilistic active perception frameworks based on Gaussian Processes, and merged into geometric maps so to facilitate the SAR mission. We then show how to use tactile perception to enhance mapping. Implicit environmental properties such as the terrain deformability, are analyzed through strategic glancesand touches and then mapped into probabilistic models.Lastly, we address the problem of reconstructing objects in the environment. Wepresent a technique for simultaneous 3D reconstruction of static regions and rigidly moving objects in a scene that enables on-the-fly model generation. Although this thesis focuses mostly on rescue UGVs, the concepts presented canbe applied to other mobile platforms that operates under similar circumstances. To make sure that the suggested methods work, we have put efforts into design of user interfaces and the evaluation of those in user studies.
|
|
| 13. |
- Güler, Püren, 1987-
(författare)
-
Learning Object Properties From Manipulation for Manipulation
- 2017
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The world contains objects with various properties - rigid, granular, liquid, elastic or plastic. As humans, while interacting with the objects, we plan our manipulation by considering their properties. For instance, while holding a rigid object such as a brick, we adapt our grasp based on its centre of mass not to drop it. On the other hand while manipulating a deformable object, we may consider additional properties to the centre of mass such elasticity, brittleness etc. for grasp stability. Therefore, knowing object properties is an integral part of skilled manipulation of objects. For manipulating objects skillfully, robots should be able to predict the object properties as humans do. To predict the properties, interactions with objects are essential. These interactions give rise distinct sensory signals that contains information about the object properties. The signals coming from a single sensory modality may give ambiguous information or noisy measurements. Hence, by integrating multi-sensory modalities (vision, touch, audio or proprioceptive), a manipulated object can be observed from different aspects and this can decrease the uncertainty in the observed properties. By analyzing the perceived sensory signals, a robot reasons about the object properties and adjusts its manipulation based on this information. During this adjustment, the robot can make use of a simulation model to predict the object behavior to plan the next action. For instance, if an object is assumed to be rigid before interaction and exhibit deformable behavior after interaction, an internal simulation model can be used to predict the load force exerted on the object, so that appropriate manipulation can be planned in the next action. Thus, learning about object properties can be defined as an active procedure. The robot explores the object properties actively and purposefully by interacting with the object, and adjusting its manipulation based on the sensory information and predicted object behavior through an internal simulation model.This thesis investigates the necessary mechanisms that we mentioned above to learn object properties: (i) multi-sensory information, (ii) simulation and (iii) active exploration. In particular, we investigate these three mechanisms that represent different and complementary ways of extracting a certain object property, the deformability of objects. Firstly, we investigate the feasibility of using visual and/or tactile data to classify the content of a container based on the deformation observed when a robotic hand squeezes and deforms the container. According to our result, both visual and tactile sensory data individually give high accuracy rates while classifying the content type based on the deformation. Next, we investigate the usage of a simulation model to estimate the object deformability that is revealed through a manipulation. The proposed method identify accurately the deformability of the test objects in synthetic and real-world data. Finally, we investigate the integration of the deformation simulation in a robotic active perception framework to extract the heterogenous deformability properties of an environment through physical interactions. In the experiments that we apply on real-world objects, we illustrate that the active perception framework can map the heterogeneous deformability properties of a surface.
|
|
| 14. |
- Haustein, Joshua Alexander, 1987-
(författare)
-
Robot Manipulation Planning Among Obstacles: Grasping, Placing and Rearranging
- 2020
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis presents planning algorithms for three different robot manipulation tasks: fingertip grasping, object placing and rearranging. Herein, we place special attention on addressing these tasks in the presence of obstacles. Obstacles are frequently encountered in human-centered environments and constrain a robot's motion and ability to manipulate objects. In narrow shelves, for example, even the common task of pick-and-place becomes challenging. A shelf is difficult to navigate and many potential grasps and placements are inaccessible. Hence, to solve such tasks, specialized manipulation planning algorithms are required that can cope with the presence of obstacles.For fingertip grasping, we first present a framework to learn models that encode which grasps a given dexterous robot hand can reach. These models are then used to facilitate planning and optimization of fingertip grasps. Next, we address the presence of obstacles and integrate fingertip grasp and motion planning to generate grasps that are reachable by a robot in complex scenes.For object placing, we first present an algorithm that plans the placement of a grasped object among obstacles so that a user-given placement objective is maximized. We then extend this algorithm, and incorporate planning in-hand manipulation to increase the set of placements a robot can reach.Lastly, we go beyond pure collision avoidance and study object rearrangement planning. Specifically, we consider the special case of non-prehensile rearrangement, where a robot rearranges multiple objects through pushing. First, we present how a kinodynamic motion planning algorithm can be augmented with learned models to rearrange a few target objects among movable and static obstacles. We then present how we can use Monte Carlo tree search to solve a large-scale rearrangement problem, where a robot is tasked to spatially sort many objects according to a user-assigned class membership.
|
|
| 15. |
- Hjelm, Martin, 1980-
(författare)
-
Holistic Grasping: Affordances, Grasp Semantics, Task Constraints
- 2019
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Most of us perform grasping actions over a thousand times per day without giving it much consideration, be it from driving to drinking coffee. Learning robots the same ease when it comes to grasping has been a goal for the robotics research community for decades.The reason for the slow progress lays mainly in the inferiority of the robot sensorimotor system. Robotic grippers are often non-compliant, lack the degrees of freedom of human hands, and haptic sensors are rudimentary involving significantly less resolution and sensitivity than in humans.Research has therefore focused on engineering solutions that center on the stability of the grasp. This involves specifying complex functions and search strategies detailing the interaction between the digits of the robot and the surface of the object. Given the amount of variation in materials, shapes, and ability to deform it seems infeasible to analytically formulate such a gripper-to-shape mapping. Many researchers have instead looked to data-driven methods for learning the gripper-to-shape mapping as does this thesis.Humans obviously have a similar mapping capability. However, how we grasp an object is determined foremost by what we are going to do with the object. We have priors on task, material, and the dynamics of objects that help guide the grasping process. We also have a deeper understanding of how shape and material relate to our own embodiment.We tie all these aspects together: our understanding of what an object can be used for, how that affects our interaction with it, and how our hand can form to achieve the goal of the manipulation. For us humans grasping is not just a gripper-to-shape mapping it is a holistic process where all parts of the chain matters to the outcome. The focus of this thesis is thus on how to incorporate such a holistic process into robotic grasp planning. We will address the holistic grasping process through three jointly connected modules. The first is affordance detection and learning to infer the common parts for objects that afford an action, a form of conceptualization of the affordance categories. The second is learning grasp semantics, how shape relates to the gripper configuration. And finally the third is to learn how task constrains the grasping process.We will explore these three parts through the concept of similarity. This translates directly into the idea that we should learn a representation that puts similar types of the entities that we are describing, that is, objects, grasps, and tasks, close to each other in space. We will show that the idea of similarity based representations will help the robot reason about which parts of an object is important for affordance inference, which grasps and tasks are similar, and how the categories relate to each other. Finally, the similarity-based approach will help us tie all parts together in the conceptual demonstration of how a holistic grasping process might be realized.
|
|
| 16. |
- Hyttinen, Emil, 1988-
(författare)
-
Adaptive Grasping Using Tactile Sensing
- 2017
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Grasping novel objects is challenging because of incomplete object data and because of uncertainties inherent in real world applications. To robustly perform grasps on previously unseen objects, feedback from touch is essential. In our research, we study how information from touch sensors can be used to improve grasping novel objects. Since it is not trivial to extract relevant object properties and deduce appropriate actions from touch sensing, we employ machine learning techniques to learn suitable behaviors. We have shown that grasp stability estimation based on touch can be improved by including an approximate notion of object shape. Further we have devised a method to guide local grasp adaptations based on our stability estimation method. Grasp corrections are found by simulating tactile data for grasps in the vicinity of the current grasp. We present several experiments to demonstrate the applicability of our methods. The thesis is concluded by discussing our results and suggesting potential topics for further research.
|
|
| 17. |
- Maki, Atsuto, et al.
(författare)
-
In Memoriam : Jan-Olof Eklundh
- 2022
-
Ingår i: IEEE Transactions on Pattern Analysis and Machine Intelligence. - : IEEE COMPUTER SOC. - 0162-8828 .- 1939-3539. ; 44:9, s. 4488-4489
-
Tidskriftsartikel (refereegranskat)
|
|
| 18. |
- Marchetti, Giovanni Luca
(författare)
-
On Symmetries and Metrics in Geometric Inference
- 2024
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Spaces of data naturally carry intrinsic geometry. Statistics and machine learning can leverage on this rich structure in order to achieve efficiency and semantic generalization. Extracting geometry from data is therefore a fundamental challenge which by itself defines a statistical, computational and unsupervised learning problem. To this end, symmetries and metrics are two fundamental objects which are ubiquitous in continuous and discrete geometry. Both are suitable for data-driven approaches since symmetries arise as interactions and are thus collectable in practice while metrics can be induced locally from the ambient space. In this thesis, we address the question of extracting geometry from data by leveraging on symmetries and metrics. Additionally, we explore methods for statistical inference exploiting the extracted geometric structure. On the metric side, we focus on Voronoi tessellations and Delaunay triangulations, which are classical tools in computational geometry. Based on them, we propose novel non-parametric methods for machine learning and statistics, focusing on theoretical and computational aspects. These methods include an active version of the nearest neighbor regressor as well as two high-dimensional density estimators. All of them possess convergence guarantees due to the adaptiveness of Voronoi cells. On the symmetry side, we focus on representation learning in the context of data acted upon by a group. Specifically, we propose a method for learning equivariant representations which are guaranteed to be isomorphic to the data space, even in the presence of symmetries stabilizing data. We additionally explore applications of such representations in a robotics context, where symmetries correspond to actions performed by an agent. Lastly, we provide a theoretical analysis of invariant neural networks and show how the group-theoretical Fourier transform emerges in their weights. This addresses the problem of symmetry discovery in a self-supervised manner.
|
|
| 19. |
- Marzinotto, Alejandro, 1990-
(författare)
-
Flexible Robot to Object Interactions Through Rigid and Deformable Cages
- 2017
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- In this thesis we study the problem of robotic interaction with objects from a flexible perspective that complements the rigid force-closure approach. In a flexible interaction the object is not firmly bound to the robot (immobilized), which leads to many interesting scenarios. We focus on the secure kind of flexible interactions, commonly referred to as caging grasps. In this context, the adjective secure implies that the object is not able to escape arbitrarily far away from the robot which is caging it. A cage is a secure flexible interaction because it does not immobilize the object, but restricts its motion to a finite set of possible configurations. We study cages in two novel scenarios for objects with holes: caging through multi-agent cooperation and through dual-arm knotting with a rope. From these two case studies, we were able to analyze the caging problem in a broader perspective leading to the definition of a hierarchical classification of flexible interactions and cages.In parallel to the geometric and physical problem of flexible interactions with objects, we study also the problem of discrete action scheduling through a novel control architecture called Behavior Trees (BTs). In this thesis we propose a formulation that unifies the competing BT philosophies into a single framework. We analyze how the mainstream BT formulations differ from each other, as well as their benefits and limitations. We also compare the plan representation capabilities of BTs with respect to the traditional approach of Controlled Hybrid Dynamical Systems (CHDSs). In this regard, we present bidirectional translation algorithms between such representations as well as the necessary and sufficient conditions for translation convergence. Lastly, we demonstrate our action scheduling BT architecture showcasing the aforementioned caging scenarios, as well as other examples that show how BTs can be interfaced with other high level planners.
|
|
| 20. |
- Mitsioni, Ioanna, 1991-, et al.
(författare)
-
Interpretability in Contact-Rich Manipulation via Kinodynamic Images
- 2021
-
Ingår i: Proceedings - IEEE International Conference on Robotics and Automation. - : Institute of Electrical and Electronics Engineers (IEEE). - 1050-4729. ; 2021-May, s. 10175-10181
-
Konferensbidrag (refereegranskat)abstract
- Deep Neural Networks (NNs) have been widely utilized in contact-rich manipulation tasks to model the complicated contact dynamics. However, NN-based models are often difficult to decipher which can lead to seemingly inexplicable behaviors and unidentifiable failure cases. In this work, we address the interpretability of NN-based models by introducing the kinodynamic images. We propose a methodology that creates images from kinematic and dynamic data of contact-rich manipulation tasks. By using images as the state representation, we enable the application of interpretability modules that were previously limited to vision-based tasks. We use this representation to train a Convolutional Neural Network (CNN) and we extract interpretations with Grad-CAM to produce visual explanations. Our method is versatile and can be applied to any classification problem in manipulation tasks to visually interpret which parts of the input drive the model's decisions and distinguish its failure modes, regardless of the features used. Our experiments demonstrate that our method enables detailed visual inspections of sequences in a task, and high-level evaluations of a model's behavior.
|
|
| 21. |
- Pinto Basto de Carvalho, Joao Frederico, 1988-
(författare)
-
Topological Methods for Motion Prediction and Caging
- 2020
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- To fulfill the requirements of automation in unstructured environmentsit will be necessary to endow robots with the ability to plan actions thatcan handle the dynamic nature of changing environments and are robust toperceptual errors. This thesis focuses on the design of algorithms to facilitatemotion planning in human environments and rigid object manipulation.Understanding human motion is a necessary first step to be able to performmotion planning in spaces that are inhabited by humans. Specifically throughlong-term prediction a robot should be able to plan collision-avoiding paths tocarry out whatever tasks are required of it. In this thesis we present a methodto classify motions by clustering paths, together with a method to translatethe resulting clusters into motion patterns that can be used to predict motion.Another challenge of robotics is the manipulation of everyday objects.Even in the realm of rigid objects, safe object-manipulation by either grippersor dexterous robotic hands requires complex physical parameter estimation.Such estimations are often error-prone and misestimations may cause completefailure to execute the desired task. Caging is presented as an alternativeapproach to classical manipulation by employing topological invariants todetermine whether an object is secured with only bounded mobility. Wepresent a method to decide whether a rigid object is in fact caged by a givengrasp or not, relying only on a rough approximation of the object and thegripper.
|
|
| 22. |
- Romero, Javier, 1983-
(författare)
-
From Human to Robot Grasping
- 2011
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Imagine that a robot fetched this thesis for you from a book shelf. How doyou think the robot would have been programmed? One possibility is thatexperienced engineers had written low level descriptions of all imaginabletasks, including grasping a small book from this particular shelf. A secondoption would be that the robot tried to learn how to grasp books from yourshelf autonomously, resulting in hours of trial-and-error and several bookson the floor.In this thesis, we argue in favor of a third approach where you teach therobot how to grasp books from your shelf through grasping by demonstration.It is based on the idea of robots learning grasping actions by observinghumans performing them. This imposes minimum requirements on the humanteacher: no programming knowledge and, in this thesis, no need for specialsensory devices. It also maximizes the amount of sources from which therobot can learn: any video footage showing a task performed by a human couldpotentially be used in the learning process. And hopefully it reduces theamount of books that end up on the floor. This document explores the challenges involved in the creation of such asystem. First, the robot should be able to understand what the teacher isdoing with their hands. This means, it needs to estimate the pose of theteacher's hands by visually observing their in the absence of markers or anyother input devices which could interfere with the demonstration. Second,the robot should translate the human representation acquired in terms ofhand poses to its own embodiment. Since the kinematics of the robot arepotentially very different from the human one, defining a similarity measureapplicable to very different bodies becomes a challenge. Third, theexecution of the grasp should be continuously monitored to react toinaccuracies in the robot perception or changes in the grasping scenario.While visual data can help correcting the reaching movement to the object,tactile data enables accurate adaptation of the grasp itself, therebyadjusting the robot's internal model of the scene to reality. Finally,acquiring compact models of human grasping actions can help in bothperceiving human demonstrations more accurately and executing them in a morehuman-like manner. Moreover, modeling human grasps can provide us withinsights about what makes an artificial hand design anthropomorphic,assisting the design of new robotic manipulators and hand prostheses. All these modules try to solve particular subproblems of a grasping bydemonstration system. We hope the research on these subproblems performed inthis thesis will both bring us closer to our dream of a learning robot andcontribute to the multiple research fields where these subproblems arecoming from.
|
|
