Magic Glasses: From 2D to 3D
Abstract: This paper proposes a virtual 3D eyeglasses try try-on on system driven by a 2D Internet image of a human face wearing with a pair of eyeglasses. The main technical challenge of this system is the automatic 3D eyeglasses model reconstruction from the 2D glasses on n a frontal human face. Against this challenge, this paper first proposes an eyeglasses segmentation method using a convolutional neural network-based based parsing algorithm to label the glasses pixels, followed with a proposed symmetry-based based level level-set optimization tion algorithm to refine the contour of the eyeglasses. With the precisely extracted silhouette image, we take advantages of the smoothness and the symmetry priors of the eyeglasses, and propose a silhouette-based based 3D deformation method to deform a 3D eyeglasses eyegl model selected from a predefined 3D model database. The obtained model is plausibly approximated to the input 2D eyeglasses after a texture mapping step. Finally, we develop a virtual try try-on on system to interactively synthesize the reconstructed eyeglasses sses on a moving target face in real time. The experimental results demonstrate the efficient and convincing virtual try try-on on performance of our approach and the commercial potential of our proposed system.