Deep Belief Network

Jump to: navigation, search

A Deep Belief Network (DBN) is a directed conditional probability network that is a deep network.







A deep belief net can be viewed as a composition of simple learning modules each of which is a restricted type of Boltzmann machine that contains a layer of visible units that represent the data and a layer of hidden units that learn to represent features that capture higher-order correlations in the data. The two layers are connected by a matrix of symmetrically weighted connections, [math]W[/math] , and there are no connections within a layer. Given a vector of activities [math]v[/math] for the visible units, the hidden units are all conditionally independent so it is easy to sample a vector, [math]h[/math], from the factorial posterior distribution over hidden vectors, [math]p(h|v,W)[/math] . It is also easy to sample from [math]p(v|h,W)[/math] . By starting with an observed data vector on the units and alternating several times between sampling from [math]p(h|v,W)[/math] and [math]p(v|h,W)[/math], it is easy to get a learning signal. This signal is simply the difference between the pairwise correlations of the visible and hidden units at the beginning and end of the sampling (see Boltzmann machine for details).