Skip to content

Latest commit

 

History

History

transformer-wikitext2

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Fromage 🧀 optimiser

Transformer training on Wikitext-2

This codebase is from the Pytorch example. To run the training script, use a command like:

python main.py --cuda --epochs 20 --model Transformer --optim fromage --lr 0.01 --p_bound 1.0

We provide the shell script batch.sh to run multiple experiments.