Synced·Added· 5 days ago65
SGLang achieves SOTA performance in parallel speculative decoding with a maximum throughput improvement of 4x, now open source.
Brief overview
SGLang achieves draft network depth causal dependency through parallel speculative decoding with a maximum throughput of 4x, now open source.
Source: Synced · jiqizhixin.com