Finish. the remaining rules.

Fine tuning accepting more places with factory.

Changelog.md

@@ -28,12 +28,29 @@ In general we introduce a few factories, that avoid having to pass the manifold
   `Nesterov` works as before, but the manifold as its first parameter is no longer necessary and `p` is now a keyword argument.
 * `ConjugateDescentCoefficient` is now called `ConjugateDescentCoefficientRule`.
   `ConjugateDescentCoefficient` works as before, but can now use the factory in between
+* the `ConjugateGradientBealeRestart` is now called `ConjugateGradientBealeRestartRule`.
+  For the `ConjugateGradientBealeRestart` the manifold is now a first paramaater, that is not necessary and no longer the `manifold=` keyword.
 * `DaiYuanCoefficient` is now called `DaiYuanCoefficientRule`.
-  `DaiYuanCoefficient` works as before, but the manifold as its first parameter is no longer necessary.
+  For the `DaiYuanCoefficient` the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the `vector_transport_method=` keyword.
 * `FletcherReevesCoefficient` is now called `FletcherReevesCoefficientRule`.
   `FletcherReevesCoefficient` works as before, but can now use the factory in between
 * `HagerZhangCoefficient` is now called `HagerZhangCoefficientRule`.
-  `HagerZhangCoefficient` works as before, but the manifold as its first parameter is no longer necessary.
+  For the `HagerZhangCoefficient` the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the `vector_transport_method=` keyword.
+* `HestenesStiefelCoefficient` is now called `HestenesStiefelCoefficientRule`.
+  For the `HestenesStiefelCoefficient` the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the `vector_transport_method=` keyword.
+* `LiuStoreyCoefficient` is now called `LiuStoreyCoefficientRule`.
+  For the `LiuStoreyCoefficient` the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the `vector_transport_method=` keyword.
+* `PolakRibiereCoefficient` is now called `PolakRibiereCoefficientRule`.
+  For the `PolakRibiereCoefficient` the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the `vector_transport_method=` keyword.
+* the `SteepestDirectionUpdateRule` is now called `SteepestDescentCoefficientRule`.
+  The `SteepestDescentCoefficient` is equivalent, but creates the new factory interims wise.
+* `AbstractGradientGroupProcessor` is now called `AbstractGradientGroupDirectionRule`
+* the `StochasticGradient` is now called `StochasticGradientRule`.
+  The `StochasticGradient` is equivalent, but creates the new factory interims wise,
+  so that the manifold is not longer necessary.
+* the `AlternatingGradient` is now called `AlternatingGradientRule`.
+  The `AlternatingGradient` is equivalent, but creates the new factory interims wise,
+  so that the manifold is not longer necessary.
 * `quasi_Newton` had a keyword `scale_initial_operator=` that was inconsistently declared (sometimes bool, sometimes real) and was unused.
   It is now called `initial_scale=1.0` and scales the initial (diagonal, unit) matrix within the approximation of the Hessian additionally to the $\frac{1}{\lVert g_k\rVert}$ scaling with the norm of the oldes gradient for the limited memory variant. For the full matrix variant the initial identity matrix is now scaled with this parameter.
 * Unify doc strings and presentation of keyword arguments

docs/src/solvers/alternating_gradient_descent.md

@@ -21,8 +21,10 @@ The most inner one should always be the following one though.
 
 ```@docs
 AlternatingGradient
+Manopt.AlternatingGradientRule
 ```
 
+which internally uses
 
 ## [Technical details](@id sec-agd-technical-details)
 

docs/src/solvers/conjugate_gradient_descent.md

@@ -22,13 +22,14 @@ The update rules act as [`DirectionUpdateRule`](@ref), which internally always f
 
 ```@docs
 ConjugateDescentCoefficient
+ConjugateGradientBealeRestart
 DaiYuanCoefficient
 FletcherReevesCoefficient
 HagerZhangCoefficient
 HestenesStiefelCoefficient
 LiuStoreyCoefficient
 PolakRibiereCoefficient
-SteepestDirectionUpdateRule
+SteepestDescentCoefficient
 ```
 
 ## Internal rule storages
@@ -38,6 +39,10 @@ Manopt.ConjugateGradientBealeRestartRule
 Manopt.DaiYuanCoefficientRule
 Manopt.FletcherReevesCoefficientRule
 Manopt.HagerZhangCoefficientRule
+Manopt.HestenesStiefelCoefficientRule
+Manopt.LiuStoreyCoefficientRule
+Manopt.PolakRibiereCoefficientRule
+Manopt.SteepestDescentCoefficientRule
 ```
 
 ## [Technical details](@id sec-cgd-technical-details)

docs/src/solvers/stochastic_gradient_descent.md

@@ -13,17 +13,24 @@ stochastic_gradient_descent!
 
 ```@docs
 StochasticGradientDescentState
+Manopt.default_stepsize(::AbstractManifold, ::Type{StochasticGradientDescentState})
 ```
 
 Additionally, the options share a [`DirectionUpdateRule`](@ref),
 so you can also apply [`MomentumGradient`](@ref) and [`AverageGradient`](@ref) here.
 The most inner one should always be.
 
 ```@docs
-AbstractGradientGroupProcessor
 StochasticGradient
 ```
 
+which internally uses
+
+```@docs
+AbstractGradientGroupDirectionRule
+StochasticGradientRule
+```
+
 ## [Technical details](@id sec-sgd-technical-details)
 
 The [`stochastic_gradient_descent`](@ref) solver requires the following functions of a manifold to be available

src/Manopt.jl

@@ -452,15 +452,15 @@ export objective_cache_factory
 # Direction Update Rules
 export DirectionUpdateRule
 export Gradient, StochasticGradient, AverageGradient, MomentumGradient, Nesterov
-export SteepestDirectionUpdateRule,
+export SteepestDescentCoefficient,
     HestenesStiefelCoefficient,
     FletcherReevesCoefficient,
     PolakRibiereCoefficient,
     ConjugateDescentCoefficient,
     LiuStoreyCoefficient,
     DaiYuanCoefficient,
     HagerZhangCoefficient,
-    ConjugateGradientBealeRestartRule
+    ConjugateGradientBealeRestart
 #
 # Solvers
 export adaptive_regularization_with_cubics,

src/documentation_glossary.jl

@@ -122,30 +122,40 @@ _math(args...; kwargs...) = glossary(:Math, args...; kwargs...)
 # Links
 # Collect short forms for links, especially Interdocs ones.
 _link(args...; kwargs...) = glossary(:Link, args...; kwargs...)
-define!(:Link, :Manopt, "[`Manopt.jl`](https://manoptjl.org)")
+
 define!(
     :Link,
-    :rand,
+    :AbstractManifold,
+    "[`AbstractManifold`](@extref `ManifoldsBase.AbstractManifold`)",
+)
+define!(
+    :Link,
+    :manifold_dimension,
     (; M="M") ->
-        "[`rand`](@extref Base.rand-Tuple{AbstractManifold})$(length(M) > 0 ? "`($M)`" : "")",
+        "[`manifold_dimension`](@extref `ManifoldsBase.manifold_dimension-Tuple{AbstractManifold}`)$(length(M) > 0 ? "`($M)`" : "")",
+)
+define!(:Link, :Manopt, "[`Manopt.jl`](https://manoptjl.org)")
+define!(
+    :Link, :Manifolds, "[`Manifolds.jl`](https://juliamanifolds.github.io/Manifolds.jl/)"
 )
 define!(
     :Link,
-    :zero_vector,
-    (; M="M", p="p") ->
-        "[`zero_vector`](@extref `ManifoldsBase.zero_vector-Tuple{AbstractManifold, Any}`)$(length(M) > 0 ? "`($M, $p)`" : "")",
+    :ManifoldsBase,
+    "[`ManifoldsBase.jl`](https://juliamanifolds.github.io/ManifoldsBase.jl/)",
 )
 define!(
     :Link,
-    :manifold_dimension,
+    :rand,
     (; M="M") ->
-        "[`manifold_dimension`](@extref `ManifoldsBase.manifold_dimension-Tuple{AbstractManifold}`)$(length(M) > 0 ? "`($M)`" : "")",
+        "[`rand`](@extref Base.rand-Tuple{AbstractManifold})$(length(M) > 0 ? "`($M)`" : "")",
 )
 define!(
     :Link,
-    :AbstractManifold,
-    "[`AbstractManifold`](@extref `ManifoldsBase.AbstractManifold`)",
+    :zero_vector,
+    (; M="M", p="p") ->
+        "[`zero_vector`](@extref `ManifoldsBase.zero_vector-Tuple{AbstractManifold, Any}`)$(length(M) > 0 ? "`($M, $p)`" : "")",
 )
+
 #
 #
 # Notes / Remarks