A novel sparse auto-encoder for deep unsupervised learning

作者：Xiaojuan Jiang, Yinghua Zhang, Wensheng Zhang, Xian Xiao · 年份：2013 · DOI：10.1109/icaci.2013.6748512 · 被引用次数：31 · 研究领域：Advanced Neural Network Applications、Generative Adversarial Networks and Image Synthesis、Human Pose and Action Recognition

This paper proposes a novel sparse variant of auto-encoders as a building block to pre-train deep neural networks. Compared with sparse auto-encoders through KL-divergence, our method requires fewer hyper-parameters and the sparsity level of the hidden units can be learnt automatically. We have compared our method with several other unsupervised leaning algorithms on the benchmark databases. The satisfactory classification accuracy (97.92% on MNIST and 87.29% on NORB) can be achieved by a 2-hidden-layer neural network pre-trained using our algorithm, and the whole training procedure (including pre-training and fine-tuning) takes far less time than the state-of-art results.