The degree to which weights are updated on each iteration of the training. The degree to which weights are updated on each iteration of the training, or more specifically, the learning rate parameter scales the magnitude of weight updates in order to minimize the network’s loss function. The learning rate often a hyperparameter setting for the optimizer. Smaller updates to the data improves convergence. Typical learning rates are 0.0001, 0.001, and 0.01.
Menu
