mlatoz
Boltzmann Machine
Boltzmann Machine
| Supervised | Artificial Neural Networks | Used for Regression & Classification |
|---|---|---|
| Convolutional Neural Networks | Used for Computer Vision | |
| Recurrent Neural Networks | Used for Time Series Analysis |
| Unsupervised | Self-Organizing Maps | Used for Feature Detection |
|---|---|---|
| Deep Boltzmann Machines | Used for Recommendation Systems | |
| AutoEncoders | Used for Recommendation Systems |
Energy-Based Model (EBM)
Boltzmann Distribution:
pi = e-εi/kT / j = 1∑M e-εj/kT
where:-
pi = Probability of a certain state of a system (Here i)
εi = Energy of that system
k = Constant
T = Temperature
"pi" is inversely proportional to "εi"
Energy Function for a Restricted Boltzmann Machine:-
E(v, h) = -i∑ aivi - j∑ bjhj - i∑j∑ vi wi,j hj
Probability of being in a certain state:-
P(v, h) = 1/Z e-E(v, h)
where
Z = Sum of all the values of the possible states
Additional Reading
A Tutorial on Energy-Based Learning
By Yann LeCun et al. (2006)
A Tutorial on Energy-Based Learning - Yann LeCun
Additional Reading
Mr. Nobody (film)
By Jaco Van Dormael (2009)
.png)
Mr. Nobody (film) - Jaco Van Dormael
Reference:- Boltzmann Distribution
Contrastive Divergence
- It allows Restricted Boltzmann Machines to learn.
∂logp(v0) / ∂wij = < vi0 hj0 > - < vi∞ hj∞ >
where
It is a Gradient Formula.
Additional Reading
A fast learning algorithm for deep belief nets
By Geoffrey Hinton et al. (2006)
A fast learning algorithm for deep belief nets - Geoffrey Hinton
Additional Reading
Notes on Contrastive Divergence
By Oliver Woodford (2012)
Notes on Contrastive Divergence - Oliver Woodford
Deep Belief Networks
Additional Reading
Greedy Layer - Wise Training of Deep Networks
By Yoshua Bengio et al. (2006)
Greedy Layer-Wise Training of Deep Networks - Yoshua Bengio
Additional Reading
The wake-sleep algorithm for unsupervised neural networks
By Geoffrey Hinton et al. (1995)
The wake-sleep algorithm for unsupervised neural networks - Geoffrey Hinton
Deep Boltzmann Machines
-
NOTE: Deep Belief Networks are not the same as Deep Boltzmann Machines.
-
DBMs can extract features that are more sophisticated, more complex and therefore they could be used for more complex tasks.
Additional Reading
Deep Boltzmann Machines
By Ruslan Salakhutdinov et al. (2009)
Deep Boltzmann Machines - Ruslan Salakhutdinov
| «Previous |