Releases: tfjgeorge/nngeometry
Releases · tfjgeorge/nngeometry
Internal changes in Jacobian generator, new features, new layers
Internal changes:
- The DataLoader is not part of the generator anymore
- Full backward hooks in jacobian generator
- Migrates gradient computation outside of Jacobian class to JacobianFactory
- Adds efficient grads for convolution implementation, with switch to choose between 2 implementations
- Changes solve implementation
New features
- PVector add_to_model method
- Matrix-matrix products for KFAC, Dense, BlockDiag; Diag representations
- Solve for PMatLowRank
- Dot between 2 PVector objects
- Pickle for PMat and PVector objects
- Grad method for PVector objects
- MC Fisher helper for segmentation
- Adds get_kfe method to PMatEKFAC
New layers
- Custom layers: Cosine, Weightnorm, Affine
- Support for additional layers: ConvTranspose2d
Minor release: Jacobian generator now raises an Exception when using batch norm in training mode instead of silently failing
This minor releases makes sure that users are aware of the fact that we currently do not support batch norm in training mode. Prior releases would instead silently fail, without telling the user that the returned objects are likely wrong.
Layer Collections, support for GroupNorm, QuasiDiag representation
This release revamps NNGeometry in order to have LayerCollection at its heart, which allows:
- flexibility in parameter space definition (e.g. if you want to compute FIM or NTK for a subset of your layers)
- flexbility in some operations, e.g. in a Fv product
F.mv(v)
F and v do not need to be created from the same model instance, but as long as they share the same LayerCollection it is enough
We also added support for GroupNorm layers, and included the QuasiDiag parameter space representation.
This release includes some documentation (not perfect !), and supports continuous integration tools.