RASMUS/gemma-4-E4B-it-mtp-ONNX

Gemma 4 E4B MTP drafter (ONNX, WebGPU)

Files for multi-token-prediction (speculative) decoding of Gemma 4 E4B in the browser with onnxruntime-web.

• onnx/model_chain2_q4.onnx: two draft steps of google/gemma-4-E4B-it-assistant in one graph, with the target's own 4-bit token embedding and the argmax. Weights 4-bit (MatMulNBits, block 32), math in float32.
• onnx/decoder_model_merged_q4f16_hidden.onnx: the graph of onnx-community/gemma-4-E4B-it-ONNX decoder_model_merged_q4f16.onnx with one added output, mtp_hidden (the final-norm hidden state). It has no weights: load it with the upstream decoder_model_merged_q4f16.onnx_data and decoder_model_merged_q4f16.onnx_data_1 as external data.

Build: ONNX_CONVERSION/tools/export_gemma4_target_hidden.py and export_gemma4_drafter_chain.py in the WebGPU Platform repository.

Parity: at context lengths 37, 511, 513, 514, 1500, 3000, each draft step picks the same token as the Hugging Face reference forward when both get the same step input, and the embedding equals the target's own within one float16 step.