Implementation · TOPLOC

Speculative decoding goes undetected

On this page

← All known flaws

SignificantTheoretical argumentOpen

The TOPLOC authors state that it cannot detect speculative decoding. In speculative decoding, a provider decodes with a cheaper model and uses the larger model only for prefill.

Sources: [1]

Search

Full search page