The learning rate is a hyperparameter that controls how much a model's weights are adjusted in response to estimated error each time they are updated during training. Too high and the model diverges, never settling on a good solution; too low and training is painfully slow or gets stuck in suboptimal solutions. Finding the right learning rate is one of the most important steps in model training, often guided by learning rate schedules that start high and decay over time.
Fundamentals
Learning Rate
Related terms
Learn to use these concepts in practice.
Join the 212AY Academy →