LSTM and QRNN Language Model Toolkit for PyTorch

NFNets and Adaptive Gradient Clipping for SGD implemented in PyTorch. Find explanation at tourdeml.github.io/blog/

Efficient, transparent deep learning in hundreds of lines of code.

深度学习常用优化方法详解

MATLAB/Octave library for stochastic optimization algorithms: Version 1.0.20

Machine learning algorithms in Dart programming language

Ternary Gradients to Reduce Communication in Distributed Deep Learning (TensorFlow)

This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"

Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers

A Deep Learning and preprocessing framework in Rust with support for CPU and GPU.

A tour of different optimization algorithms in PyTorch.

Lua implementation of Entropy-SGD

Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers https://arxiv.org/abs/1802.00124

Java based sample code for developing on Android. The demos in this repository are stored on separate branches. To navigate to a demo, please click branches.

PyTorch implementation of Federated Learning algorithms FedSGD, FedAvg, FedAvgM, FedIR, FedVC, FedProx and standard SGD, applied to visual classification. Client distributions are synthesized with arbitrary non-identicalness and imbalance (Dirichlet priors). Client systems can be arbitrarily heterogeneous. Several mobile-friendly models are prov…

Unofficial implementation of Switching from Adam to SGD optimization in PyTorch.

Riemannian stochastic optimization algorithms: Version 1.0.3

Distributed Learning by Pair-Wise Averaging

A C++ toolkit for Convex Optimization (Logistic Loss, SVM, SVR, Least Squares etc.), Convex Optimization algorithms (LBFGS, TRON, SGD, AdsGrad, CG, Nesterov etc.) and Classifiers/Regressors (Logistic Regression, SVMs, Least Squares Regression etc.)

R/Rcpp implementation of the 'Follow-the-Regularized-Leader' algorithm