K2-Horizon-7B-DSpark

A lossless DSpark draft model for IFM/K2-Horizon-7B, trained with speculators (training code).

• Up to 5.11× faster on math and 3.01× on code, 3.00× on average (temperature 0)
• Mean acceptance length 3.37 (block size 8)

Usage

vllm serve IFM/K2-Horizon-7B --port 8000 \
  --trust-remote-code \
  --reasoning-parser k2_horizon \
  --max-model-len 32768 \
  --speculative-config '{"model": "rasyosef/K2-Horizon-7B-DSpark", "num_speculative_tokens": 8}' \
  --override-generation-config '{"temperature": 0}'

For tool use, add --enable-auto-tool-choice --tool-call-parser k2_horizon.

Details

• Qwen3-style, 4 layers, ~0.9B params, bfloat16, block size 8
• Hidden size 4096, MLP 8192, 32 query / 8 KV heads (head dim 128)
• First 3 layers sliding-window (512), last full attention
• Draft vocab 32,000 (from 250,624); aux hidden states from verifier layers 2/10/18/26/34
• 100,000 Open PerfectBlend prompts with verifier-generated responses, seq length 8192
• 3 epochs, lr 3e-4, cosine (4% warmup), loss {"ce": 0.1, "tv": 0.9}
• Trained on a single NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB)

Evaluation

RedHatAI/speculator_benchmarks, temperature 0, single A100, concurrency 1.

Throughput (tokens/s)

Mean output throughput. Baseline is K2-Horizon-7B without speculative decoding.

subset	baseline (no drafter)	DSpark (this model)	speedup
math_reasoning	71.4	364.5	5.11×
HumanEval	71.0	213.5	3.01×
rag	70.7	214.1	3.03×
qa	70.8	217.6	3.07×
question	71.2	176.9	2.48×
writing	71.0	176.3	2.48×
translation	71.2	190.9	2.68×
tool_call	70.4	190.5	2.71×
summarization	70.7	172.8	2.44×
average	70.9	213.0	3.00×

Acceptance at temperature 0

acceptance_length is mean tokens committed per verification step, including the bonus token (max 9.0 at block size 8).

subset	acceptance_length	pos_0	pos_1	pos_2	pos_3	pos_4	pos_5	pos_6	pos_7
math_reasoning	5.350	87.8%	75.6%	65.4%	55.8%	47.7%	40.5%	34.0%	28.1%
HumanEval	3.625	78.1%	57.3%	41.4%	29.7%	21.4%	15.5%	11.1%	7.9%
rag	3.411	75.4%	54.5%	38.6%	26.8%	18.7%	12.4%	8.6%	6.0%
qa	3.326	70.9%	49.9%	35.8%	25.6%	18.8%	14.0%	10.3%	7.2%
question	3.316	69.8%	48.4%	34.0%	25.1%	19.2%	14.6%	11.5%	8.8%
writing	3.284	69.3%	47.9%	34.0%	25.0%	18.9%	14.3%	11.0%	7.9%
translation	3.050	68.6%	47.3%	32.1%	21.8%	14.8%	9.9%	6.4%	4.1%
tool_call	3.018	72.0%	48.5%	32.0%	20.4%	12.9%	8.0%	5.0%	3.0%
summarization	2.863	68.6%	44.5%	28.9%	18.9%	11.8%	7.1%	4.1%	2.4%

Weighted across all subsets: 3.370 over 298,078 verification steps.

Limitations

Only works with K2-Horizon-7B. Sampling, concurrent load, and contexts beyond 8K tokens were not evaluated.

License

Apache 2.0