Very impressive!!
Very impressive: Boris Power praises NanoGPT Speedrun gains
AIOpenAI's Boris Power replies "Very impressive!!" to Herman Brunborg's post. Brunborg says he cut NanoGPT Speedrun from 39.9s to 6.7s, an 83.2% reduction, using retrieval-augmented n-gram hints for rare web content.
Boris PowerVerified on X
@BorisMPower
I improved NanoGPT Speedrun from 39.9s → 6.7s (-83.2%), more than every record since 2025 combined (-81.4%)! The web is full of rare, repeated content that the LM can’t memorize in time. I solve this with retrieval-augmented n-gram hints🧵