Struct juice::solver::SolverConfig

pub struct SolverConfig {

    pub name: String,
    pub network: LayerConfig,
    pub objective: LayerConfig,
    pub solver: SolverKind,
    pub minibatch_size: usize,
    pub lr_policy: LRPolicy,
    pub base_lr: f32,
    pub gamma: f32,
    pub stepsize: usize,
    pub clip_gradients: Option<f32>,
    pub weight_decay: Option<f32>,
    pub regularization_method: Option<RegularizationMethod>,
    pub momentum: f32,
}

Configuration for a Solver

Fields

name: String

Name of the solver.

network: LayerConfig

The [LayerConfig][1] that is used to initialize the network. [1]: ../layer/struct.LayerConfig.html

objective: LayerConfig

The [LayerConfig][1] that is used to initialize the objective. [1]: ../layer/struct.LayerConfig.html

solver: SolverKind

The [Solver implementation][1] to be used. [1]: ../solvers/index.html

minibatch_size: usize

Accumulate gradients over minibatch_size instances.

Default: 1

lr_policy: LRPolicy

The learning rate policy to be used.

Default: Fixed

base_lr: f32

The base learning rate.

Default: 0.01

gamma: f32

gamma as used in the calculation of most learning rate policies.

Default: 0.1

stepsize: usize

The stepsize used in Step and Sigmoid learning policies.

Default: 10

clip_gradients: Option<f32>

The threshold for clipping gradients.

Gradient values will be scaled to their L2 norm of length clip_gradients if their L2 norm is larger than clip_gradients. If set to None gradients will not be clipped.

Default: None

weight_decay: Option<f32>

The global [weight decay][1] multiplier for [regularization][2]. [1]: http://www.alglib.net/dataanalysis/improvinggeneralization.php#header3 [2]: https://cs231n.github.io/neural-networks-2/#reg

Regularization can prevent overfitting.

If set to None no regularization will be performed.

regularization_method: Option<RegularizationMethod>

The method of [regularization][1] to use. [1]: https://cs231n.github.io/neural-networks-2/#reg

There are different methods for regularization. The two most common ones are [L1 regularization][1] and [L2 regularization][1].

See RegularizationMethod for all implemented methods.

Currently only L2 regularization is implemented. See Issue #23.

momentum: f32

The [momentum][1] multiplier for [SGD solvers][2]. [1]: https://en.wikipedia.org/wiki/Stochastic_gradient_descent#Momentum [2]: ../solvers/sgd/index.html

For more information see [SGD with momentum][3] [3]: ../solvers/sgd/momentum/index.html

The value should always be between 0 and 1 and dictates how much of the previous gradient update will be added to the current one.

Default: 0

Implementations

impl SolverConfig

pub fn get_learning_rate(&self, iter: usize) -> f32

Return the learning rate for a supplied iteration.

The way the learning rate is calculated depends on the configured LRPolicy.

Used by the Solver to calculate the learning rate for the current iteration. The calculated learning rate has a different effect on training dependent on what type of Solver you are using.

Trait Implementations

impl Clone for SolverConfig

fn clone(&self) -> SolverConfig

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SolverConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SolverConfig

fn default() -> SolverConfig

Returns the “default value” for a type.