I trained GPT-2 with full-batch gradient descent so you don't have to.

Someone trained GPT-2 using full-batch gradient descent, an approach motivated in part by Charles Renshaw-Whitman and by claims that noise is essential to neural network training.

Written by AI from LessWrong's published text. Read the original for full details.