uuujf / SGDNoise Public

Notifications You must be signed in to change notification settings
Fork 1
Star 14

[ICML 2019] The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects

arxiv.org/abs/1803.00195

14 stars 1 fork Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
2Dim		2Dim
CIFAR10		CIFAR10
FashionMNIST		FashionMNIST
OneHiddenLayer		OneHiddenLayer
SVHN		SVHN
.gitignore		.gitignore
README.md		README.md

Repository files navigation

The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects

An implementation for the paper:

The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects

Zhanxing Zhu^*, Jingfeng Wu^*, Bing Yu, Lei Wu, Jinwen Ma

Usage

Two-dimensional experiments

See folder 2Dim.

One hidden layer experiments

See folder OneHiddenLayer.

FashionMNIST

See folder FashionMNIST.

SVHN adn CIFAR-10

See folder SVHN and CIFAR-10.

Note

Bibtex, Slides, Poster.

About

[ICML 2019] The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects

arxiv.org/abs/1803.00195

sgd noise

Report repository

Languages

Python 100.0%