This project addresses the problem of photometric stereo for non-Lambertian surfaces. Existing approaches often adopt simplified reflectance models to make the problem more tractable, but this greatly hinders their applications on real-world objects. In this project, we propose a deep fully convolutional network, called PS-FCN, that takes an arbitrary number of images of a static object captured under different light directions with a fixed camera as input, and predicts a normal map of the object in a fast feed-forward pass. Unlike the recently proposed learning based method, PS-FCN does not require a pre-defined set of light directions during training and testing, and can handle multiple images and light directions in an order-agnostic manner. Although we train PS-FCN on synthetic data, it can generalize well to real datasets. We further show that PS-FCN can be easily extended to handle the problem of uncalibrated photometric stereo. Extensive experiments on public real datasets show that PS-FCN outperforms existing approaches in calibrated photometric stereo, and promising results are achieved in uncalibrated scenario, clearly demonstrating its effectiveness.
Publications
[1] Guanying Chen, Kai Han, Boxin Shi, Yasuyuki Matsushita, Kwan-Yee K. Wong
Deep Photometric Stereo for Non-Lambertian Surfaces
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020. [project page] [code]
[2] Guanying Chen, Kai Han, Boxin Shi, Yasuyuki Matsushita, Kwan-Yee K. Wong
Self-calibrating Deep Photometric Stereo Networks
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Oral presentation [project page] [code]
[3] Guanying Chen, Kai Han, Kwan-Yee K. Wong
PS-FCN: A Flexible Learning Framework for Photometric Stereo
European Conference on Computer Vision (ECCV), 2018. [project page] [code]