Extrinsic calibration of the omnidirectional vision system and 3D reconstruction of an indoor environment

Autonomous navigation of mobile robots indoors has attracted the attention of many computer vision researchers over the years. A wide variety of approaches and algorithms were proposed to solve this problem. The proper perception of the environment becomes an important part for such robots. Robots must be able to evaluate the three-dimensional structure of the environment in order to perform their algorithms. However, visual sensors, such as conventional cameras, do not allow processing enough information due to the limited viewing angle. This article presents a comprehensive approach for three-dimensional modeling of an indoor environment. The vision system considered in this paper consists of an omnidirectional camera and a structured light. The omnidirectional camera captures a wide range of information, while the laser beam is easy to detect and extract for further analysis. To obtain reliable measurement results, the vision system must be calibrated. For this purpose, the paper considers an improved method of external calibration. The paper also considers the 3D reconstruction algorithm of an indoor environment that includes a semantic segmentation neural network. A single input image is required to perform the calibration method as well as the 3D modeling method. These methods significantly speed up the data processing process, without losing accuracy in measurements. In turn, recent advances in neural networks require a large amount of training data in environments with different conditions. Thus, developing and testing navigation algorithms can be expensive and time-consuming. This article evaluates the proposed methods experimentally using data generated by a previously developed simulator.