Semi-Automatic Video Frame Annotation for Construction Equipment Automation Using Scale-Models

• Data collection and annotation is a time consuming and costly process, yet necessary for machine vision. Automation of construction equipment relies on seeing and detecting different objects in the vehicle’s surroundings. Construction equipment is commonly used to perform frequent repetitive tasks, which are interesting to automate. An example of such a task is the short-loading cycle, where the material is moved from a pile into the tipping body of a dump truck for transport. To complete this task, the wheel loader needs to have the capability to locate the tipping body of the dump truck. The machine vision system also allows the vehicle to detect unforeseen dangers such as other vehicles and more importantly human workers. In this work, we investigate the viability to perform semi-automatic annotation of video data using linear interpolation. The data is collected using scale-models mimicking a wheel-loaders approach towards a dump truck during the short-loading cycle. To measure the viability of this type of solution, the workload is compared to the accuracy of the model, YOLOv3. The results indicate that it is possible to maintain the performance while decreasing the annotation workload by about 95%. This is an interesting result for this application domain, as safety is critical and retaining the vision system performance is more important than decreasing the annotation workload. The fact that the performance seems to retain with a large workload decrease is an encouraging sign.

Subject headings

NATURVETENSKAP  -- Data- och informationsvetenskap -- Datorseende och robotik (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Computer Vision and Robotics (hsv//eng)

Keyword

Autonomous construction equipment

semi-automatic annotation

video-stream data

object detection

linear interpolation

Cyberfysiska system

Cyber-Physical Systems

Maskininlärning

Machine Learning

Publication and Content Type

ref (subject category)

kon (subject category)