I (think) finally wrapped my head around multi-token prediction, which is a term used for two very different techniques in speculative decoding
here's an illustration of the difference
Hugging Face's Merve Noyan explains two multi-token prediction techniques
AISummary
Merve Noyan says she has finally worked out multi-token prediction, a term used for two very different techniques in speculative decoding. She posts an illustration showing how the two differ.
Post on XView on X
Source: merve · x.comPublished