What is the purpose of a theory ? To explain why something works. Sure. But what good is a theory (i.e VC) that is totally
Rank Collapse in Deep Learning
We can learn a lot about Why Deep Learning Works by studying the properties of the layer weight matrices of pre-trained neural networks. And, hopefully,
Power Laws in Deep Learning 2: Universality
Power Law Distributions in Deep Learning In a previous post, we saw that the Fully Connected (FC) layers of the most common pre-trained Deep Learning
Power Laws in Deep Learning
Why Does Deep Learning Work ? If we could get a better handle on this, we could solve some very hard problems in Deep Learning. A
Self-Regularization in Deep Neural Networks: a preview
Why Deep Learning Works: Self Regularization in DNNs An early talk describing details in this paper Implicit Self-Regularization in Deep Neural Networks: Evidence from Random
Rethinking–or Remembering–Generalization in Neural Networks
Rethinking Generalization: the Google paper Last year, ICLR 2017, a very interesting paper came out of Google claiming that Understanding Deep Learning Requires Rethinking Generalization
