I am frequently asked, why does weightwatcher work ? The weightwatcher tool uses power law fits to model the eigenvalue…
WeightWatcher: Empirical Quality Metrics for Deep Neural Networks
We introduce the weightwatcher (ww) , a python tool for a python tool for computing quality metrics of trained, and…
Towards a new Theory of Learning: Statistical Mechanics of Deep Neural Networks
Introduction For the past few years, we have talked a lot about how we can understand the properties of Deep…
This Week in Machine Learning and AI: Implicit Self-Regularization
Big thanks to and the team at This Week in Machine Learning and AI for my recent interview: Implicit Self-Regularization…
SF Bay ACM Talk: Heavy Tailed Self Regularization in Deep Neural Networks
My Collaborator did a great job giving a talk on our research at the local San Francisco Bay ACM Meetup…
Heavy Tailed Self Regularization in Deep Neural Nets: 1 year of research
My talk at ICSI-the International Computer Science Institute at UC Berkeley. ICSI is a leading independent, nonprofit center for research…
Don’t Peek part 2: Predictions without Test Data
This is a followup to a previous post: DON’T PEEK: DEEP LEARNING WITHOUT LOOKING … AT TEST DATA The idea…suppose…
Machine Learning and AI for the Lean Start Up
Machine Learning and AI for the Lean Start Up My recent talk at the French Tech Hub Startup Accelerator
Don’t Peek: Deep Learning without looking … at test data
What is the purpose of a theory ? To explain why something works. Sure. But what good is a theory…
Rank Collapse in Deep Learning
We can learn a lot about Why Deep Learning Works by studying the properties of the layer weight matrices of…
calculated | content 