1. |
- Larsson, Måns, 1989, et al.
(författare)
-
A cross-season correspondence dataset for robust semantic segmentation
- 2019
-
Ingår i: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. - 1063-6919. ; 2019-June, s. 9524-9534
-
Konferensbidrag (refereegranskat)abstract
- In this paper, we present a method to utilize 2D-2D point matches between images taken during different image conditions to train a convolutional neural network for semantic segmentation. Enforcing label consistency across the matches makes the final segmentation algorithm robust to seasonal changes. We describe how these 2D-2D matches can be generated with little human interaction by geometrically matching points from 3D models built from images. Two cross-season correspondence datasets are created providing 2D-2D matches across seasonal changes as well as from day to night. The datasets are made publicly available to facilitate further research. We show that adding the correspondences as extra supervision during training improves the segmentation performance of the convolutional neural network, making it more robust to seasonal changes and weather conditions.
|
|
2. |
- Larsson, Måns, 1989, et al.
(författare)
-
Fine-Grained Segmentation Networks: Self-Supervised Segmentation for Improved Long-Term Visual Localization
- 2019
-
Ingår i: Proceedings of the IEEE International Conference on Computer Vision. - 1550-5499. ; 2019-October:October, s. 31-41
-
Konferensbidrag (refereegranskat)abstract
- Long-term visual localization is the problem of estimating the camera pose of a given query image in a scene whose appearance changes over time. It is an important problem in practice, for example, encountered in autonomous driving. In order to gain robustness to such changes, long-term localization approaches often use segmantic segmentations as an invariant scene representation, as the semantic meaning of each scene part should not be affected by seasonal and other changes. However, these representations are typically not very discriminative due to the limited number of available classes. In this paper, we propose a new neural network, the Fine-Grained Segmentation Network (FGSN), that can be used to provide image segmentations with a larger number of labels and can be trained in a self-supervised fashion. In addition, we show how FGSNs can be trained to output consistent labels across seasonal changes. We demonstrate through extensive experiments that integrating the fine-grained segmentations produced by our FGSNs into existing localization algorithms leads to substantial improvements in localization performance.
|
|
3. |
- Lundgren, Malin, 1979, et al.
(författare)
-
Vehicle self-localization using off-the-shelf sensors and a detailed map
- 2014
-
Ingår i: IEEE Intelligent Vehicles Symposium, Proceedings. - 9781479936380 ; , s. 522-528
-
Konferensbidrag (refereegranskat)abstract
- In the research on autonomous vehicles, self-localization is an important problem to solve. In this paper we present a localization algorithm based on a map and a set of off-the-shelf sensors, with the purpose of evaluating this low-cost solution with respect to localization performance. The used test vehicle is equipped with a Global Positioning System receiver, a gyroscope, wheel speed sensors, a camera providing information about lane markings, and a radar detecting landmarks along the road. Evaluation shows that the localization result is within or close to the requirements for autonomous driving when lane markers and good radar landmarks are present. However, it also indicates that the solution is not robust enough to handle situations when one of these information sources is absent.
|
|
4. |
- Sattler, Torsten, et al.
(författare)
-
Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions
- 2018
-
Ingår i: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. - 1063-6919. - 9781538664209 ; , s. 8601-8610
-
Konferensbidrag (refereegranskat)abstract
- Visual localization enables autonomous vehicles to navigate in their surroundings and augmented reality applica-tions to link virtual to real worlds. Practical visual localization approaches need to be robust to a wide variety of viewing condition, including day-night changes, as well as weather and seasonal variations, while providing highly accurate 6 degree-of-freedom (6DOF) camera pose estimates. In this paper, we introduce the first benchmark datasets specifically designed for analyzing the impact of such factors on visual localization. Using carefully created ground truth poses for query images taken under a wide variety of conditions, we evaluate the impact of various factors on 6DOF camera pose estimation accuracy through extensive experiments with state-of-the-art localization approaches. Based on our results, we draw conclusions about the difficulty of different conditions, showing that long-term localization is far from solved, and propose promising avenues for future work, including sequence-based localization approaches and the need for better local features. Our benchmark is available at visuallocalization.net
|
|
5. |
- Stenborg, Erik, 1980
(författare)
-
Localization for Autonomous Vehicles
- 2017
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- There has been a huge interest in self-driving cars lately, which is understandable, given the improvements it is predicted to bring in terms of safety and comfort in transportation. One enabling technology for self-driving cars, is accurate and reliable localization. Without it, one would not be able to use map information for path planning, and instead be left to solely rely on sensor input, to figure out what the road ahead looks like. This thesis is focused on the problem of cost effective localization of self-driving cars, which fulfill accuracy and reliability requirements for safe operation. In an initial study, a car equipped with the sensors of an advanced driver-assistance system is analyzed with respect to its localization performance. It is found that although performance is acceptable in good conditions, it needs improvements to reach the level required for autonomous vehicles. The global navigational satellite system (GNSS) receiver, and the automotive camera system are found to not provide as good information as expected. This presents the opportunity to improve the solution, with only marginally increased cost, by utilizing the existing sensors better. A first improvement is regarding global navigational satellite systems (GNSS) receivers. A novel solution using time relative GNSS observations, is proposed. The proposed solution is tested on data from the DriveMe project in Göteborg, and found capable of providing highly accurate time-relative positioning without use of expensive dual frequency receivers, base stations, or complex solutions that require long convergence time. Error introduced over 30 seconds of driving is found to be less than 1 dm on average.A second improvement is regarding how to use more information from the vehicle mounted cameras, without needing extremely large maps that would be required if using traditional image feature descriptors. This should be realized while maintaining localization performance over an extended period of time, despite the challenge of large visual changes over the year. A novel localization solution based on semantic descriptors is proposed, and is shown to be superior to a solution using traditional image features in terms of size of map, at a certain accuracy level.
|
|
6. |
- Stenborg, Erik, 1980
(författare)
-
Long-Term Localization for Self-Driving Cars
- 2020
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Long-term localization is hard due to changing conditions, while relative localization within time sequences is much easier. To achieve long-term localization in a sequential setting, such as, for self-driving cars, relative localization should be used to the fullest extent, whenever possible. This thesis presents solutions and insights both for long-term sequential visual localization, and localization using global navigational satellite systems (GNSS), that push us closer to the goal of accurate and reliable localization for self-driving cars. It addresses the question: How to achieve accurate and robust, yet cost-effective long-term localization for self-driving cars? Starting in this question, the thesis explores how existing sensor suites for advanced driver-assistance systems (ADAS) can be used most efficiently, and how landmarks in maps can be recognized and used for localization even after severe changes in appearance. The findings show that: * State-of-the-art ADAS sensors are insufficient to meet the requirements for localization of a self-driving car in less than ideal conditions. GNSS and visual localization are identified as areas to improve. * Highly accurate relative localization with no convergence delay is possible by using time relative GNSS observations with a single band receiver, and no base stations. * Sequential semantic localization is identified as a promising focus point for further research based on a benchmark study comparing state-of-the-art visual localization methods in challenging autonomous driving scenarios including day-to-night and seasonal changes. * A novel sequential semantic localization algorithm improves accuracy while significantly reducing map size compared to traditional methods based on matching of local image features. * Improvements for semantic segmentation in challenging conditions can be made efficiently by automatically generating pixel correspondences between images from a multitude of conditions and enforcing a consistency constraint during training. * A segmentation algorithm with automatically defined and more fine-grained classes improves localization performance. * The performance advantage seen in single image localization for modern local image features, when compared to traditional ones, is all but erased when considering sequential data with odometry, thus, encouraging to focus future research more on sequential localization, rather than pure single image localization.
|
|
7. |
- Stenborg, Erik, 1980, et al.
(författare)
-
Long-term Visual Localization using Semantically Segmented Images
- 2018
-
Ingår i: Proceedings - IEEE International Conference on Robotics and Automation. - 1050-4729. - 9781538630815 ; , s. 6484-6490
-
Konferensbidrag (refereegranskat)abstract
- Robust cross-seasonal localization is one of the major challenges in long-term visual navigation of autonomous vehicles. In this paper, we exploit recent advances in semantic segmentation of images, i.e., where each pixel is assigned a label related to the type of object it represents, to attack the problem of long-term visual localization. We show that semantically labeled 3D point maps of the environment, together with semantically segmented images, can be efficiently used for vehicle localization without the need for detailed feature descriptors (SIFT, SURF, etc.). Thus, instead of depending on hand-crafted feature descriptors, we rely on the training of an image segmenter. The resulting map takes up much less storage space compared to a traditional descriptor based map. A particle filter based semantic localization solution is compared to one based on SIFT-features, and even with large seasonal variations over the year we perform on par with the larger and more descriptive SIFT-features, and are able to localize with an error below 1 m most of the time.
|
|
8. |
- Stenborg, Erik, 1980, et al.
(författare)
-
Using a single band GNSS receiver to improve relative positioning in autonomous cars
- 2016
-
Ingår i: IEEE Intelligent Vehicles Symposium, Proceedings. ; 2016-August, s. Art no 7535498, Pages 921-926
-
Konferensbidrag (refereegranskat)abstract
- We show how the combination of a single band global navigation satellite systems (GNSS) receiver, standard automotive level inertial measurement unit (IMU), and wheel speed sensors, can be used for relative positioning with accuracy on a decimeter scale. It is realized without the need for expensive dual band receivers, base stations or long initialization times. This is implemented and evaluated in a natural driving environment against a reference systems and against two simple base line systems; one using only IMU and wheel speed sensors, the other also adding basic GNSS. The proposed solution provides substantially slower error growth than either of the two base line systems.
|
|
9. |
- Stenborg, Erik, 1980, et al.
(författare)
-
Using Image Sequences for Long-Term Visual Localization
- 2020
-
Ingår i: Proceedings - 2020 International Conference on 3D Vision, 3DV 2020. ; , s. 938-948
-
Konferensbidrag (refereegranskat)abstract
- Estimating the pose of a camera in a known scene, i.e., visual localization, is a core task for applications such as self-driving cars. In many scenarios, image sequences are available and existing work on combining single-image localization with odometry offers to unlock their potential for improving localization performance. Still, the largest part of the literature focuses on single-image localization and ignores the availability of sequence data. The goal of this paper is to demonstrate the potential of image sequences in challenging scenarios, e.g., under day-night or seasonal changes. Combining ideas from the literature, we describe a sequence-based localization pipeline that combines odometry with both a coarse and a fine localization module. Experiments on long-term localization datasets show that combining single-image global localization against a prebuilt map with a visual odometry / SLAM pipeline improves performance to a level where the extended CMU Seasons dataset can be considered solved. We show that SIFT features can perform on par with modern state-of-the-art features in our framework, despite being much weaker and a magnitude faster to compute. Our code is publicly available at github.com/rulllars.
|
|
10. |
- Toft, Carl, 1990, et al.
(författare)
-
Long-Term Visual Localization Revisited
- 2022
-
Ingår i: IEEE Transactions on Pattern Analysis and Machine Intelligence. - 1939-3539 .- 0162-8828. ; 44:4, s. 2074-2088
-
Tidskriftsartikel (refereegranskat)abstract
- Visual localization enables autonomous vehicles to navigate in their surroundings and augmented reality applications to link virtual to real worlds. Practical visual localization approaches need to be robust to a wide variety of viewing conditions, including day-night changes, as well as weather and seasonal variations, while providing highly accurate six degree-of-freedom (6DOF) camera pose estimates. In this paper, we extend three publicly available datasets containing images captured under a wide variety of viewing conditions, but lacking camera pose information, with ground truth pose information, making evaluation of the impact of various factors on 6DOF camera pose estimation accuracy possible. We also discuss the performance of state-of-the-art localization approaches on these datasets. Additionally, we release around half of the poses for all conditions, and keep the remaining half private as a test set, in the hopes that this will stimulate research on long-term visual localization, learned local image features, and related research areas. Our datasets are available at visuallocalization.net, where we are also hosting a benchmarking server for automatic evaluation of results on the test set. The presented state-of-the-art results are to a large degree based on submissions to our server.
|
|
11. |
- Toft, Carl, 1990, et al.
(författare)
-
Semantic Match Consistency for Long-Term Visual Localization
- 2018
-
Ingår i: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). - Cham : Springer International Publishing. - 1611-3349 .- 0302-9743. ; 11206 LNCS, s. 391-408
-
Konferensbidrag (refereegranskat)abstract
- Robust and accurate visual localization across large appearance variations due to changes in time of day, seasons, or changes of the environment is a challenging problem which is of importance to application areas such as navigation of autonomous robots. Traditional feature-based methods often struggle in these conditions due to the significant number of erroneous matches between the image and the 3D model. In this paper, we present a method for scoring the individual correspondences by exploiting semantic information about the query image and the scene. In this way, erroneous correspondences tend to get a low semantic consistency score, whereas correct correspondences tend to get a high score. By incorporating this information in a standard localization pipeline, we show that the localization performance can be significantly improved compared to the state-of-the-art, as evaluated on two challenging long-term localization benchmarks.
|
|