Luigi/primeminutes-nemo-litert

voxsum-nemo-litert

LiteRT (TFLite) graphs for the streaming speech engine of VoxSum: the X-ASR streaming encoder and the Nemotron-3 Sortformer diarization graph, both run on CPU through XNNPACK. They are the LiteRT counterpart of the GGUF models used by the ggml backend.

Files

File	Model	Input → output	Quantization
xasr_stream_q8_octav.tflite	X-ASR zh-en streaming Zipformer2 encoder	480 ms fbank window [61, 80] + streaming states → encoder frames + updated states	dynamic int8 weights (OCTAV), fp32 activations
sortformer_int8_f190.tflite	Nemotron-3 Sortformer diarization (encoder + head)	embeddings [1, 190, 512] fp32 + valid_frames → speaker probabilities [190, 8]	dynamic int8 weights, fixed window of 190 frames
sortformer_int8_f380.tflite	same Sortformer weights, larger window	embeddings [1, 380, 512] fp32 + valid_frames → speaker probabilities [380, 8]	dynamic int8 weights, fixed window of 380 frames: holds the trained speaker cache of 264 frames + chunk 50 + right context 12
xasr_host.bin	X-ASR host weights for the transducer decoder/joiner and front end (replaces the ggml GGUF)	packed tensors, magic XAH1	–
diar_host.bin	Sortformer host weights (replaces the ggml GGUF)	packed tensors, magic DH01	–

These are not standalone models. The fbank/mel front ends, the X-ASR transducer decoder and joiner, the speaker cache and turn decoding stay in the host code, as in the ggml engine. The Sortformer pre-encode (stacked mel → 512-d, a 512×1024 linear layer) is not part of the graph; the app applies it with a separate fp32 matrix, pre_encode_512x1024.f32, which ships inside the app.

Sources

• X-ASR: GilgameshWind/X-ASR-zh-en (480 ms chunk model), weights transplanted into a Zipformer2 graph for streaming export.
• Diarization: nvidia/Nemotron-3-Diarization, encoder and head re-implemented in PyTorch and exported to LiteRT.

Check the upstream licences before redistributing these files.

Validation

• Sortformer, int8 TFLite vs ggml Q8_0 on the same 16 windows taken from two recorded clips: 99.95 % of frame × speaker decisions (p > 0.5) agree; max |Δ probability| 0.21, mean about 1e-3.
• Padding leak (windows padded to 190 frames): at most 2e-5.
• f380 (speaker cache 264, the length the model was trained with): host-engine output identical to the reference export; on six AISHELL-4 meetings missed speaker changes 39 % → 32 % vs f190 (cache 128); real time on a Galaxy Note10+ during live recording. The app uses f380; f190 stays for older app versions.

Usage

Load with the LiteRT C API on CPU (XNNPACK, num_threads). Reference loader: app/src/main/cpp/nemo/litert_graph.cc in the VoxSum repository. Each graph has a single signature.

The .bin files are produced by tools/nemo-host/{xasr,diar}_host_pack.py and read by the ggml-free C++ host parts (xasr_host.cc, diar_host.cc), so no GGUF or ggml is needed (backend litert_host).

The app pins a revision per file set (ModelManager.kt on Android, ModelStore.swift on iOS).