University of Illinois at Urbana-Champaign
1Purdue University
Reconstructing the 3D shape of objects observed in a single image is a challenging task. Recent approaches rely on visual cues extracted from a given image learned from a deep net. In this work, we leverage recent advances in monocular scene understanding to incorporate an additional geometric cue of surface normals. For this, we proposed a novel optimization layer that encourages the face normals of the reconstructed shape to be aligned with estimated surface normals. We develop a computationally efficient conjugate-gradient-based method that avoids the computation of high-dimensional sparse matrices. We show this framework to achieve compelling shape reconstruction results on the challenging Pix3D and ShapeNet datasets.
@inproceedings{hu-icml2023-surface,
title = {Surface Snapping Optimization Layer for Single Image Object Shape Reconstruction},
author = {Yuan-Ting Hu and Schwing, Alexander G and Yeh, Raymond A},
booktitle = {International Conference on Machine Learning (ICML)},
year = {2023},
}