Fantastic Measures of Generalization — That Actually Work

In the next few posts, I am going to discuss how to use the generalization metrics included in the open-source … More

Model Monitoring with WeightWatcher: Data-Free DEEP LEARNING Diagnostics

Weighwatcher is an open-source Model Monitoring tool that provides Data-Free Diagnostics for production-quality Deep Neural Networks (DNNs). It. can tell … More

How to tell if you have trained your Model with enough data ?

Deep Neural Networks (DNN) require a lot of training data. Even fine-tuning a model can require a lot. A LOT. … More

is your model overtrained ?

Are your models over-trained ? The weightwatcher tool can detect the signatures of overtraining in specific layers of a pre/trained … More

Simpson’s Paradox and Deep Learning Metrics with Weightwatcher

What is WeightWatcher ? The WeightWatcher tool is an open-source python package that can be used to predict the test … More

Why WeightWatcher Works

I am frequently asked, why does weightwatcher work ? The weightwatcher tool uses power law fits to model the eigenvalue … More

WeightWatcher: Empirical Quality Metrics for Deep Neural Networks

We introduce the weightwatcher (ww) , a python tool for a python tool for computing quality metrics of trained, and … More

Towards a new Theory of Learning: Statistical Mechanics of Deep Neural Networks

Introduction For the past year or two, we have talked a lot about how we can understand the properties of … More

This Week in Machine Learning and AI: Implicit Self-Regularization

Big thanks to and the team at This Week in Machine Learning and AI for my recent interview: Implicit Self-Regularization … More

SF Bay ACM Talk: Heavy Tailed Self Regularization in Deep Neural Networks

My Collaborator did a great job giving a talk on our research at the local San Francisco Bay ACM Meetup … More