Quotations tagged andrejkarpathy, gpt3 in 2023
Filters: Type: quotation × Year: 2023 × andrejkarpathy × gpt3 × Sorted by date
1 result
The most dramatic optimization to nanoGPT so far (~25% speedup) is to simply increase vocab size from 50257 to 50304 (nearest multiple of 64). This calculates added useless dimensions but goes down a different kernel path with much higher occupancy. Careful with your Powers of 2.
— Andrej Karpathy # 4th February 2023, 12:08 am