There is a deep connection between the open-source weightwatcher tool, which implements ideas from the theory of Heavy Tailed Self-Regularization … More
Tag: Deep Learning
Evaluating Fine-Tuned LLMs with WeightWatcher Part II: PEFT / LoRa Models
Evaluating LLMs is hard. Especially when you don’t have a lot of test data.In the last post, we saw how to … More
WeightWatcher new feature: fix_fingers=’clip_xmax’
WeightWatcher 0.7 has just been released, and it includes the new and improved advanced feature for analyzing Deep Neural Networks … More
WeightWatcher 0.7: March 2023
First, let me say thanks to all the users in our great community — we have reached over 93K downloads … More
Deep Learning and Effective Correlation Spaces
AI has taken the world by storm. With recent advances like AlphaFold, Stable Diffusion, and ChatGPT, Deep Neural Networks (DNNs) … More
Why WeightWatcher Works
I am frequently asked, why does weightwatcher work ? The weightwatcher tool uses power law fits to model the eigenvalue … More
WeightWatcher: Empirical Quality Metrics for Deep Neural Networks
We introduce the weightwatcher (ww) , a python tool for a python tool for computing quality metrics of trained, and … More
Heavy Tailed Self Regularization in Deep Neural Nets: 1 year of research
My talk at ICSI-the International Computer Science Institute at UC Berkeley. ICSI is a leading independent, nonprofit center for research … More
Why Deep Learning Works II: the Renormalization Group
Deep Learning is amazing. But why is Deep Learning so successful? Is Deep Learning just old-school Neural Networks on modern … More
Why does Deep Learning work?
Why does Deep Learning work? This is the big question on everyone’s mind these days. C’mon we all know the … More
calculated | content 