A simple GPT implementation (layered transformer decoder) written in PyTorch. It takes the text in the input.txt file and writes more like it to output.txt.
Put input text into input.txt, e.g.
wget https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt
Then, run python gpt.py. The hyperparameters at the beginning of gpt.py were set for GPU training, and must be adjusted down if using a CPU to train.
Thanks to Andrej Karpathy for a great video on transformer implementations!