Data Science & ML
Advanced

Gradient Descent Update

Iterative optimization step to minimize a cost function.

Formula

θ=θ−α∇J(θ)\theta = \theta - \alpha \nabla J(\theta)

Variables

\thetaParameters
\alphaLearning rate
\nabla JGradient of cost

Example

Step size is controlled by learning rate \alpha

Did You Know?

Choosing the learning rate is one of the most important tuning decisions.

optimizationtraining