A neural network solution to segmentation and recovery of superquadric models from 3D image data

Research statement

The objective of this research proposal isto develop a CNN solution for real-time segmentation and superquadric model recovery from large 3D point clouds. In addition to the development of CNNs for segmentation and model recovery of superquadrics from 3D point clouds, we would like to find out if these CNNs for segmentation and model recovery from 3D point clouds could be adapted to use as input just 2D intensity or color images. Since no other method exists that would be as fast as using deep neural networks for recovery of volumetric part-based models, the success of the proposed research would have a huge impact in application areas where real-time processing is required and when huge sets of 3D data points need to be interpreted. The application areas where real-time processing is needed are primarily robotics in unconstrained environments, where previously unknown objects can be encountered and must be modeled, such as in autonomous driving, robot manipularion, path planning, etc. Knowledge-based interpretation of huge sets of 3D data points obtained by LiDAR and multi-image photogrammetry could be much easier achieved with faster methods.

Segmentation

Figure 1: Iterative segmentation and reconstruction of superquadric volumetric models from 3D image of a complex scene [4]: (a) intensity image of the scene; (b) range image - 3D points of the scene; (c) initial superquadric seeds overlayed on 3D points; (d) and (e) intermediate steps in the iterative procedure; (f) final result of superquadric reconstruction and segmentation.

Researchers

Franc Solina, Peter Peer, Vitomir Štruc, Aleš Jaklič, Jaka Šircelj, Niko Gamulin

Publications

  1. ŠIRCELJ, Jaka, OBLAK, Tim, GRM, Klemen, PETKOVIĆ, Uroš, JAKLIČ, Aleš, PEER, Peter, ŠTRUC, Vitomir, SOLINA, Franc. Segmentation and recovery of superquadric models using convolutional neural networks. V: LUKEŽIČ, Alan (ur.), TABERNIK, Domen (ur.), GRM, Klemen (ur.). Proceedings of the 25th Computer Vision Winter Workshop Conference, Rogaška Slatina, February 3-5, 2020. Electronic ed. Ljubljana: Slovensko društvo za razpoznavanje vzorcev: = Slovenian Pattern Recognition Society, 2020. Str. 74-81, ilustr. ISBN 978-961-90901-9-0. http://data.vicos.si/cvww20/CVWW20-proceedings.pdf. [COBISS.SI-ID 1538520259]
  2. OBLAK, Tim, GRM, Klemen, JAKLIČ, Aleš, PEER, Peter, ŠTRUC, Vitomir, SOLINA, Franc. Recovery of superquadrics from range images using deep learning : a preliminary study. V: SZAKÁL, Anikó (ur.). IWOBI 2019 : proceedings. Danvers (MA): IEEE, cop. 2019. Str. 45-52, ilustr. ISBN 978-1-7281-0967-1. [COBISS.SI-ID 1538269123]
  3. SLABANJA, Jurij, MEDEN, Blaž, PEER, Peter, JAKLIČ, Aleš, SOLINA, Franc. Segmentation and reconstruction of 3D models from a point cloud with deep neural networks. V: ICT convergence powered by smart intelligence : ICTC 2018. [S. l.: s. n.], 2018. Str. 118-123, ilustr. [COBISS.SI-ID 1537994435]
This research in supported by the Slovenian Research Agency - J2-9228 (B).

ARRS