Gradient Descent
An optimization algorithm that iteratively updates model weights by moving in the direction of steepest loss decrease.
In more detail
Gradient descent computes the gradient of loss with respect to weights and updates weights opposite to this gradient. Variants include batch GD (all data), stochastic GD (one example), and mini-batch GD. Fundamental optimization method for neural networks.
Examples
- Vanilla gradient descent
- Stochastic Gradient Descent (SGD)
- Mini-batch gradient descent
- Momentum-based variants (Adam, RMSprop)
Related terms
Ready to Build Production AI Systems?
Join a small cohort learning directly from an engineer who's shipped AI at Atlassian and PhonePe scale.
20 weeks · Weekend live classes · 1:1 mentorship · 20+ projects.
20 Weeks
Live Weekend Classes
< 30 Seats
Small Batch
Next Soon
Get priority access
7-Day
Money-Back Guarantee
Taught by Debasish Maji · Senior AI Engineer · Ex-Atlassian (Rovo Agent) · Ex-PhonePe (550M+ users)
Get notified when the next batch opens + free AI resources