Hinton introduced Free Energies in his 1994 paper, Autoencoders, minimum description length, and Helmholtz Free Energy This paper, along with his wake-sleep algorithm, set the foundations
Interview with a Data Scientist
It’s always fun to be interviewed. Check out my recent chat with Max Mautner, the Accidental Engineer http://theaccidentalengineer.com/charles-martin-principal-consultant-calculation-consulting/
Normalization in Deep Learning
A few days ago (Jun 2017), a 100 page on Self-Normalizing Networks appeared. An amazing piece of theoretical work, it claims to have solved the
Why Deep Learning Works 3: BackProp minimizes the Free Energy ?
?Deep Learning is presented as Energy-Based Learning Indeed, we train a neural network by running BackProp, thereby minimizing the model error–which is like minimizing an Energy. But
Foundations: Mean Field Boltzmann Machines 1987
A friend from grad school pointed out a great foundational paper on Boltzmann Machines. It is a 1987 paper from complex systems theory A Mean
Improving RBMs with physical chemistry
Restricted Boltzmann Machines (RBMs) are like the H atom of Deep Learning. They are basically a solved problem, and while of academic interest, not really used
