Recently, Microsoft Research published the LASER method: ”Layer-Selective Rank Reduction” in this recent, very popular paper The Truth is in There: … More
Tag: AI
Evaluating LLMs with WeightWatcher Part III: The Magic of Mistral, a Story of Dragon Kings
Recently, the Mistral models have taken the LLM world by storm. The Mistral Mixture of Experts (MOE) 8x7b model outperforms other … More
Evaluating Fine-Tuned LLMs with WeightWatcher
if you are fine-tuning your own LLMs, you need a way to evaluate them. And while there are over a dozen … More
Deep Learning and Effective Correlation Spaces
AI has taken the world by storm. With recent advances like AlphaFold, Stable Diffusion, and ChatGPT, Deep Neural Networks (DNNs) … More
WeightWatcher: Empirical Quality Metrics for Deep Neural Networks
We introduce the weightwatcher (ww) , a python tool for a python tool for computing quality metrics of trained, and … More
Heavy Tailed Self Regularization in Deep Neural Nets: 1 year of research
My talk at ICSI-the International Computer Science Institute at UC Berkeley. ICSI is a leading independent, nonprofit center for research … More