fix the overfit single batch behavior to actually overfit batch, not …

[karpathy]

…microbatch, we do this more cleanly by simply resetting the dataloader every step

[karpathy]

This recovers our ability to reproduce our test with our train. Example:

# reproduce our test with our train
make train_gpt2cu PRECISION=FP32
./train_gpt2cu -b 4 -t 64 -d 256 -l 0.0001 -v 200 -s 200 -a 1 -x 10 -r 0 -f 0 -e "gpt2_124M.bin"

We can also use grad accum 2:

./train_gpt2cu -b 2 -t 64 -d 256 -l 0.0001 -v 200 -s 200 -a 1 -x 10 -r 0 -f 0 -e "gpt2_124M.bin"

Same for PyTorch:

python train_gpt2.py --write_tensors 0 --batch_size 4
python train_gpt2.py --write_tensors 0 --batch_size 2

Now print the same things. Grad accum kicks in to recover the total desired batch size and gives the same results.