Applying deep learning to scene sketch recognition and 3D sketch-based 3D model retrieval




Ye, Yuxiang

Journal Title

Journal ISSN

Volume Title



Human’s sketch understanding is an important and challenging research problem. It has many applications in human computer interaction, multimedia, and computer vision. Most previous methods focus on single-object sketch recognition. Human’s scene sketch understanding has not been studied. In this work, we make the first attempt to tackle this problem. We create the first scene sketch dataset “Scene250” and explore deep learning scheme for scene sketch understanding. We propose a deep CNN model “Scene-Net” and build a novel scene sketch recognition system based on this model. Our system has been tested on the collected scene sketch dataset and compared with other state-of-the-art CNNs and sketch recognition approaches. We also extend our work to sketch-based 3D model retrieval and design a complete retrieval system (CNN-SBR) based on deep CNN. It also achieves better performance than other comparison systems. The experimental results demonstrate the effectiveness and potential of our method.



Deep learning, Convolutional neural network, Pattern recognition


Ye, Y. (2016). <i>Applying deep learning to scene sketch recognition and 3D sketch-based 3D model retrieval</i> (Unpublished thesis). Texas State University, San Marcos, Texas.


Rights Holder

Rights License

Rights URI