First, let me say thanks to all the users in our great community — we have reached over 93K downloads as of March 2023 !
The latest release of the open-source weightwatcher tool includes several important advances, including
- removing explicit dependence on tensorflow and torch on install
- the ability to process very large models, directly from their pytorch statdict files
- GPU-enabled SVD calculations
- Much faster and more stable power law calculations
- Lower memory footprint on GPU enabled machines
- an improved method for finding the weightwatcher shape-metric alpha, with the option
fix_fingers='clip_xmax'(to remove structural outliers, called fingers) - a new landing page: https://weightwatcher.ai with lots of examples
To learn more now, join our Discord channel (*Documentation to come)
WeightWatcher is a one-of-a-kind must-have tool for anyone training, deploying, or monitoring Deep Neural Networks (DNNs).
WeightWatcher (WW) is an open-source, diagnostic tool for analyzing Deep Neural Networks (DNN), without needing access to training or even test data. It is based on theoretical research into Why Deep Learning Works, based on our Theory of Heavy-Tailed Self-Regularization (HT-SR). It uses ideas from Random Matrix Theory (RMT), Statistical Mechanics, and Strongly Correlated Systems.
It can be used to:
- analyze pre/trained pyTorch and Keras DNN models (Conv2D and Dense layers)
- monitor models, and the model layers, to see if they are over-trained or over-parameterized
- predict test accuracies across different models, with or without training data
- detect potential problems when compressing or fine-tuning pretrained models
- layer warning labels: over-trained; under-trained
It is based on theoretical research into Why Deep Learning Works, using the new Theory of Heavy-Tailed Self-Regularization (HT-SR), published in JMLR and Nature Communications.
The weightwatcher tool has been developed by Calculation Consulting. We provide consulting to companies looking to implement Data Science, Machine Learning, and/or AI solutions. Reach out today to learn how to get started with your own AI project. Email: Info@CalculationConsulting.com
calculated | content 