A preview of my upcoming talk at mmds this week:
Stay tuned for a video link, and a blog post to accompany this.
Comments, questions, and bug fixes are welcome.
Thoughts on Data Science, Machine Learning, and AI
Thoughts on Data Science, Machine Learning, and AI
A preview of my upcoming talk at mmds this week:
Stay tuned for a video link, and a blog post to accompany this.
Comments, questions, and bug fixes are welcome.
calculated | content 
Using generative models to approximate energy funnels sounds most natural to me. In the past, my advisor and I were able to show that structure populations (near-optimal solutions, in our case) indeed correlated with the phenomenon of multiple binding modes. This phenomenon, is in turn believed to be related to energy funnels (http://www.ncbi.nlm.nih.gov/pubmed/18058908). My last comp-chem job was about 8 years ago and I may have missed the recent developments in the field. Besides your presentation, have there been other publications that explore generative models in the context of energy funnels and entropy computations?
LikeLiked by 1 person
I don’t know of anyone else who has pointed out the link between energy landscape theory and deep learning. I may learn more this week at mmds. thanks for posting.
LikeLiked by 1 person
For example here, chapter 8.2.2 : http://www.deeplearningbook.org/contents/optimization.html
LikeLike
Thanks!. This book came out after I prepped for the talk . This has been suspected for a very long time. It was discussed in Duda and Stork https://books.google.com/books/about/Pattern_Classification.html?id=Br33IRC3PkQC
I know about it from spin glass theory, where it was shown that random spin glasses don’t have high lying local minima above the glass transition, by studying the TAP equations. This is ancient history, dating back in the 70s. or 80s. (I’d have to dig up the old references)
LikeLike
It is interesting to see the connection to spin glasses.
LikeLike
Any video available ?
LikeLike
I am waiting for UC Berkeley to put it up
LikeLike
this is the channel: https://www.youtube.com/channel/UCmLB71obuOBLMdWCQUI-DiA
LikeLike
Thanks, I keep an eye on it.
LikeLike
The slides are very interesting, spin glasses have crossed my mind too, this question keeps bugging me, I have also background in theoretical solid state physics and machine learning.
In my understanding the question is :
-Model (e.g. a ball is jumping up and down in a room) generating data (light reflects from the moving ball) in high dimensional space (camera records it, produces pixels)
-Machine Learning is an inverse problem, we seek the Model (equations of motions of the ball) that generated the data in the high dimensional space
This also reminds me of scattering (inverse) problems.
LikeLike
I would think that unlike random matrices / spin glasses, these systems are strongly correlated. After all, they are supervised methods.
LikeLike