Implicit SGD

NumPy implementation and empirical study of implicit stochastic gradient descent (Toulis et al.) alongside standard explicit SGD baselines. Implicit SGD evaluates the gradient at the next iterate rather than the current one, which dramatically improves stability on ill-conditioned problems where explicit SGD diverges or oscillates with even modest step sizes. The repo runs a controlled comparison on synthetic regression problems with adjustable condition numbers and noise levels. Companion to BU DS 522 coursework on optimization (Adam / AMSGrad / RMSProp comparison and article evaluations of Reddi 2018 and Toulis 2016).