Improving convergence of restricted Boltzmann machines via a learning adaptive step size



Restricted Boltzmann Machines (RBMs) have recently received much attention due to their potential to integrate more complex and deeper architectures. Despite their success, in many applications, training an RBM remains a tricky task. In this paper we present a learning adaptive step size method which accelerates its convergence. The results for the MNIST database demonstrate that the proposed method can drastically reduce the time necessary to achieve a good RBM reconstruction error. Moreover, the technique excels the fixed learning rate configurations, regardless of the momentum term used.


Restricted Boltzmann Machines, Deep Belief Networks, Deep learning, Adaptive step size


Deep learning, Deep Belief Networks, Restricted Boltzmann Machines


17th Iberoamerican Congress on Pattern Recognition (CIARP 2012), LNCS vol. 7441, pp: 511-518, September 2012


Cited by

Year 2015 : 1 citations

 Gao Qiang, Yang Wu , & Li Qian . (2015). DBN level trend and its fault recognition in aerial imagery applications. Instrumentation Technology, 36 ( 6 ) , 1267-1274 .