indic-speech-mixed-4bit-onnx: Indic-Transcribe-flex at 4 bits (701 MB)

Built with Indic-Transcribe-flex from Bodhan AI / AI4Bharat.

bodhan-ai/indic-transcribe-flex at 4 bits, small enough to run in a web browser with ONNX Runtime Web. It writes Indian speech the way it is spoken: Indian words in their own script, English words in English letters ("mixed" mode). Used on keepnothing.in for Audio to text and Auto captions, on the visitor's own device. Unofficial; not affiliated with or endorsed by Bodhan AI, AI4Bharat, IIT Madras or NVIDIA.

Made from the 8-bit ONNX port by Mvkd2k06: its weights re-packed as 4-bit (ONNX Runtime MatMulNBits, 128-wide blocks), and the encoder's 1×1 convolutions made 4-bit too. Graphs, inputs, outputs and tokenizer.json are the port's own; see its card for the decode loop.

File	Size
encoder.onnx + encoder.onnx.data	477 MB
decoder.onnx + decoder.onnx.data	224 MB

Graphs

• encoder: audio [1, samples] float32, 16 kHz mono, up to 30 s → cross_k, cross_v [24, 8, frames, 128] (the mel front end is inside the graph).
• decoder: input_ids [1, S], past_k / past_v [P, 24, 8, 128], cross_k, cross_v → logits [1, 7152] for the last position, new_k / new_v [S, 24, 8, 128] to append to the cache.
• Prompt [7, 4, 18, L, L, 5, 9, 11, 13, 15], L the language token (tokenizer.json → prompts). Mixed output: position 6 = 8 (ITN). Romanized: position 7 = 10. Decode greedily until 3 (EOS) or 2 (pad).

Accuracy

FLEURS test, first 10 recordings per language, words wrong (native script, ONNX Runtime 1.31, CPU):

	Hindi	Gujarati	Marathi	Bengali	Tamil
8-bit port (1.71 GB)	7.9	19.3	14.3	15.2	38.3
This model (701 MB)	9.0	19.8	12.6	12.6	37.0

A small sample: read it as "no loss from 8 to 4 bits", not as a benchmark.

License

The Indic Open Model License v1.0, unmodified, as required for derivatives. Running it as an API or hosted service for third parties requires Bodhan AI's written approval. Base architecture nvidia/canary-1b-v2: CC-BY-4.0. Attribution and changes: NOTICE.md.