The Pegasos ("Primal Estimated Sub-Gradient Solver for SVM") algorithm is an efficient method for solving the optimization problem of a linear [[Support Vector Machine|SVM]]. It combines stochastic [[Gradient Descent]] with a decaying learning rate and a regularization term to find the optimal parameters $\theta$ and $\theta_0$​. **Parameters:** - $\eta$: Decaying factor that will decrease over time. - $\lambda$: Constant regularizing parameter, which decides how strong the previous $\theta$ is being weighted. **Algorithm:** ``` if y_i * (theta * x_i) <= 1: theta = (1 - eta * lambda) * theta * y_i * x_i theta_0 += eta * y_i else: theta = (1 - eta * lambda) * theta* ```