Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
jiteshkprasad 
posted an update 5 days ago
Post
3,655
We just released Vemana, a 278M Telugu base model trained from scratch, live in public, in about 16 hours on one H100. It learned from 5.1B tokens of public Telugu web text (Sangraha, FineWeb-2 and HPLT 3.0), seen about 2.4 times, so 12B tokens of training, with its own Telugu tokenizer.

On 416 Telugu Wikipedia pages written in Sep-Oct 2026, after nearly all the compared models were released, it scores 0.361 bits per byte. That is the lowest of every open base model under 1B we tested: nanochat-telugu-560M 0.383, Dhee-Pravega-OG Atom 0.393, Pothana Base v2 0.426, Gemma 3 1B 0.484. Same script and pages for every model, and the win rule was fixed before the run.

Sarvam-1 (2.5B) is still level or better, and on knowledge tests like Belebele and MILU Vemana is at chance, like most other small models. I think it’s most useful as a base for Telugu fine-tunes and for studying how a small model learns a language, so we put out all 12 checkpoints also. And you can run it right in your browser, on your own device.

Model: udgamlabs/Vemana-Base
Checkpoints: udgamlabs/Vemana-Checkpoints
GGUF: udgamlabs/Vemana-Base-GGUF
Try it in your browser: https://udgamlabs.com/models/vemana
Technical report: https://udgamlabs.com/research/vemana
chronossystemic

The separation between bits-per-byte and the knowledge-task results is useful: a compression gain should not quietly become a claim about understanding. With the released checkpoints, would you consider tracking held-out BPB and a fixed downstream task side by side, including a sample from outside Wikipedia? That could show where gains transfer, remain specific to the evaluation text, or plateau. I am suggesting a comparison, not reporting a run. (AI-assisted comment.)

·
jiteshkprasad

@chronossystemic Thanks, and I agree. Bits per byte going down does not mean the model knows more. On the knowledge and reasoning tests Vemana is at chance, like most open models under 1B we tested, and we say that on the card.

All our held-out numbers are on fresh Telugu Wikipedia pages, so outside Wikipedia we have not measured it. All 12 checkpoints are public, so anyone can track bits per byte and a fixed task side by side across the run, on whatever Telugu text they care about:
https://huggingface.co/udgamlabs/Vemana-Checkpoints
In this post