Simon Willison’s Weblog

The most dramatic optimization to nanoGPT so far (~25% speedup) is to simply increase vocab size from 50257 to 50304 (nearest multiple of 64). This calculates added useless dimensions but goes down a different kernel path with much higher occupancy. Careful with your Powers of 2.

— Andrej Karpathy

# 12:08 am / andrej-karpathy, performance, gpt-3, generative-ai, ai, llms