What is a SGD?

Answer 1

Stochastic Gradient Descent (SGD) is an optimization algorithm commonly used in machine learning and deep learning to minimize a loss function. Unlike traditional gradient descent, which calculates the gradient using the entire dataset, SGD updates model parameters using only a single data point or a small batch at each iteration. This approach allows for faster convergence and can help escape local minima, but it introduces more noise in the updates. As a result, SGD is particularly effective for large datasets and online learning scenarios.