Prior to this work, it was becoming apparent, due to ref. The symmetric and very simple geometry induced by the TPT confers important benefits, including better generalization performance, better robustness, and better interpretability. (NC4) For a given activation, the classifierвЂ™s decision collapses to simply choosing whichever class has the closest train class mean (i.e., the nearest class center decision rule). (NC3) Up to rescaling, the last-layer classifiers collapse to the class means or in other words, to the simplex ETF (i.e., to a self-dual configuration). (NC2) The class means collapse to the vertices of a simplex equiangular tight frame (ETF).

(NC1) Cross-example within-class variability of last-layer training activations collapses to zero, as the individual activations themselves collapse to their class means. Direct measurements of TPT, for three prototypical deepnet architectures and across seven canonical classification datasets, expose a pervasive inductive bias we call neural collapse (NC), involving four deeply interconnected phenomena. During TPT, the training error stays effectively zero, while training loss is pushed toward zero.

Modern practice for training classification deepnets involves a terminal phase of training (TPT), which begins at the epoch where training error first vanishes.

Ther trminal