Some notebooks for educational purposes. (In portuguese)
Done:
- Gradient Descent
- Conjugate Gradient
- Nonlinear Conjugate Gradient
- Coordinated Descent
- Stochastic Gradient Descent
- Newton methods (Full Newton and Barzilai-Browein)
- Stochastic Gradient Descent with variance reduction (Centroid Method)
To come:
- Stochastic Gradient Descent with variance reduction (SVRG, SAG and SAGA)
- Stochastic Gradient Descent with momentum (Polyak and Nesterov)
- Adaptative with Momentum (Adam)