| 1. |
- Akyol, Gamze, et al.
(författare)
-
A Variational Graph Autoencoder for Manipulation Action Recognition and Prediction
- 2021
-
Konferensbidrag (refereegranskat)abstract
- Despite decades of research, understanding human manipulation activities is, and has always been, one of the most attractive and challenging research topics in computer vision and robotics. Recognition and prediction of observed human manipulation actions have their roots in the applications related to, for instance, human-robot interaction and robot learning from demonstration. The current research trend heavily relies on advanced convolutional neural networks to process the structured Euclidean data, such as RGB camera images. These networks, however, come with immense computational complexity to be able to process high dimensional raw data.Different from the related works, we here introduce a deep graph autoencoder to jointly learn recognition and prediction of manipulation tasks from symbolic scene graphs, instead of relying on the structured Euclidean data. Our network has a variational autoencoder structure with two branches: one for identifying the input graph type and one for predicting the future graphs. The input of the proposed network is a set of semantic graphs which store the spatial relations between subjects and objects in the scene. The network output is a label set representing the detected and predicted class types. We benchmark our new model against different state-of-the-art methods on two different datasets, MANIAC and MSRC-9, and show that our proposed model can achieve better performance. We also release our source code https://github.com/gamzeakyol/GNet.
|
|
| 2. |
- Cortinhal, Tiago, 1990-, et al.
(författare)
-
SalsaNext : Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving
- 2021
-
Ingår i: Advances in Visual Computing. - Cham : Springer. - 9783030645595 - 9783030645588 ; , s. 207-222
-
Konferensbidrag (refereegranskat)abstract
- In this paper, we introduce SalsaNext for the uncertainty-aware semantic segmentation of a full 3D LiDAR point cloud in real-time. SalsaNext is the next version of SalsaNet which has an encoder-decoder architecture where the encoder unit has a set of ResNet blocks and the decoder part combines upsampled features from the residual blocks. In contrast to SalsaNet, we introduce a new context module, replace the ResNet encoder blocks with a new residual dilated convolution stack with gradually increasing receptive fields and add the pixel-shuffle layer in the decoder. Additionally, we switch from stride convolution to average pooling and also apply central dropout treatment. To directly optimize the Jaccard index, we further combine the weighted cross entropy loss with Lovász-Softmax loss. We finally inject a Bayesian treatment to compute the epistemic and aleatoric uncertainties for each point in the cloud. We provide a thorough quantitative evaluation on the Semantic-KITTI dataset, which demonstrates that the proposed SalsaNext outperforms other published semantic segmentation networks and achieves 3.6% more accuracy over the previous state-of-the-art method. We also release our source code1. © 2020, Springer Nature Switzerland AG.[1] https://github.com/TiagoCortinhal/SalsaNext
|
|
| 3. |
- Cortinhal, Tiago, 1990-, et al.
(författare)
-
Semantics-aware Multi-modal Domain Translation : From LiDAR Point Clouds to Panoramic Color Images
- 2021
-
Ingår i: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). - Los Alamitos : IEEE Computer Society. - 9781665401913 - 9781665401920 ; , s. 3032-3041
-
Konferensbidrag (refereegranskat)abstract
- In this work, we present a simple yet effective framework to address the domain translation problem between different sensor modalities with unique data formats. By relying only on the semantics of the scene, our modular generative framework can, for the first time, synthesize a panoramic color image from a given full 3D LiDAR point cloud. The framework starts with semantic segmentation of the point cloud, which is initially projected onto a spherical surface. The same semantic segmentation is applied to the corresponding camera image. Next, our new conditional generative model adversarially learns to translate the predicted LiDAR segment maps to the camera image counterparts. Finally, generated image segments are processed to render the panoramic scene images. We provide a thorough quantitative evaluation on the SemanticKitti dataset and show that our proposed framework outperforms other strong baseline models. Our source code is available at https://github. com/halmstad-University/TITAN-NET. © 2021 IEEE.
|
|
| 4. |
- Englund, Cristofer, 1977-, et al.
(författare)
-
AI Perspectives in Smart Cities and Communities to Enable Road Vehicle Automation and Smart Traffic Control
- 2021
-
Ingår i: Smart Cities. - Basel : MDPI. - 2624-6511. ; 4:2, s. 783-802
-
Tidskriftsartikel (refereegranskat)abstract
- Smart Cities and Communities (SCC) constitute a new paradigm in urban development. SCC ideates on a data-centered society aiming at improving efficiency by automating and optimizing activities and utilities. Information and communication technology along with internet of things enables data collection and with the help of artificial intelligence (AI) situation awareness can be obtained to feed the SCC actors with enriched knowledge. This paper describes AI perspectives in SCC and gives an overview of AI-based technologies used in traffic to enable road vehicle automation and smart traffic control. Perception, Smart Traffic Control and Driver Modelling are described along with open research challenges and standardization to help introduce advanced driver assistance systems and automated vehicle functionality in traffic. To fully realize the potential of SCC, to create a holistic view on a city level, the availability of data from different stakeholders is need. Further, though AI technologies provide accurate predictions and classifications there is an ambiguity regarding the correctness of their outputs. This can make it difficult for the human operator to trust the system. Today there are no methods that can be used to match function requirements with the level of detail in data annotation in order to train an accurate model. Another challenge related to trust is explainability, while the models have difficulties explaining how they come to a certain conclusions it is difficult for humans to trust it. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.
|
|
| 5. |
- Inceoglu, Arda, et al.
(författare)
-
FINO-Net : A Deep Multimodal Sensor Fusion Framework for Manipulation Failure Detection
- 2021
-
Ingår i: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). - : IEEE. - 9781665417143 - 9781665417150 ; , s. 6841-6847
-
Konferensbidrag (refereegranskat)abstract
- We need robots more aware of the unintended outcomes of their actions for ensuring safety. This can be achieved by an onboard failure detection system to monitor and detect such cases. Onboard failure detection is challenging with a limited set of onboard sensor setup due to the limitations of sensing capabilities of each sensor. To alleviate these challenges, we propose FINO-Net, a novel multimodal sensor fusion based deep neural network to detect and identify manipulation failures. We also introduce FAILURE, a multimodal dataset, containing 229 real-world manipulation data recorded with a Baxter robot. Our network combines RGB, depth and audio readings to effectively detect failures. Results indicate that fusing RGB with depth and audio modalities significantly improves the performance. FINO-Net achieves %98.60 detection accuracy on our novel dataset. Code and data are publicly available at https://github.com/ardai/fino-net.
|
|
