When Does Better Acceptance Pay? A Controlled Study of Speculative Decoding
arXiv:2610.24993v2 Announce Type: replace Abstract: The throughput gains of speculative decoding depend on the trade-off between acceptance length and execution latency. We ask when additional acceptance is worth the cost of obtaining it. We establish a comparison protocol spanning training, inferen