Which optimization method uses the entire training dataset to compute the gradient for each update?

Which optimization method uses the entire training dataset to compute the gradient for each update?

• A. Stochastic Gradient Descent
• B. Mini-Batch Gradient Descent
• C. Batch Gradient Descent
• D. Dynamic Learning Rate

Answer: (C)

The key idea is that the gradient used to update the model parameters is computed from the entire training dataset. This approach, batch gradient descent, evaluates the total gradient of the loss by considering every training example before making a single update, then moves the parameters in that direction. Because the update reflects the full dataset, the optimization path tends to be smooth and stable, with updates aligned to the true overall cost surface. The trade-off is that each update can be slow on large datasets and requires holding all data in memory. In contrast, stochastic gradient descent updates after each individual example, and mini-batch gradient descent updates after a small subset of examples. Dynamic learning rate refers to adjusting the step size during training and is about how large each update is rather than how many samples are used to compute the gradient.

The key idea is that the gradient used to update the model parameters is computed from the entire training dataset. This approach, batch gradient descent, evaluates the total gradient of the loss by considering every training example before making a single update, then moves the parameters in that direction. Because the update reflects the full dataset, the optimization path tends to be smooth and stable, with updates aligned to the true overall cost surface. The trade-off is that each update can be slow on large datasets and requires holding all data in memory. In contrast, stochastic gradient descent updates after each individual example, and mini-batch gradient descent updates after a small subset of examples. Dynamic learning rate refers to adjusting the step size during training and is about how large each update is rather than how many samples are used to compute the gradient.