chutesai/parallax-8b-kappa

Parallax 8B kappa: mid-training checkpoints

These are research checkpoints from kappa, a training run of Parallax that is still in progress. Parallax is a mixture-of-experts language model trained on consumer GPUs spread over several countries. The hosts have no direct connections to each other and synchronize over the public internet.

• Base model. It is pretrained on a mixture of web, document, math, code, knowledge-focused and book text. It is not instruction-tuned, chat-tuned or safety-tuned, and it will continue text rather than follow instructions.
• Mid-training. Each export is a snapshot of a run that has not finished. Later exports are expected to differ, and quality between exports is not monotonic.
• Research artifact. It is published so the training can be followed and inspected. It is not intended for production use.

Live training dashboard: parallax.chutes.ai.

Model

Parameters	~7.8B total, ~1.25B active per token
Layers	64: 18 gated delta-rule (GDN2) recurrent, 8 sliding-window attention (window 2048), 6 sparse attention, 32 mixture-of-experts
Experts	4096 routed experts (128 per MoE layer), 12 routed + 1 shared expert per token
Expert weights	ternary (-1, 0, +1) with per-row scales; at most two nonzero pairs in every group of eight
Width	d_model 1152; ReLU² expert activation
Tokenizer	Llama 3 tokenizer (vocabulary padded to 128,384); tied input/output embeddings
Training context	4096 tokens
Output logit scale	learnable, capped at 3.0

Training

• Data: the "epsilon" data mix of an earlier Parallax run, not FineWeb-Edu. The main phase draws from seven sources totalling about 1.09T tokens: two general pretraining mixtures of web, document, math and code text (about 78% of the tokens), an additional web set, two math sets, a knowledge-focused set and a books set. Two annealing sources join near the end. The run is planned as one pass of about 973B tokens.
• Optimization: the dense trunk is synchronized with a decoupled DiLoCo scheme over the internet. Routed experts are trained through low-rank adapters on the GPUs that use them, and the adapter updates are folded into full-precision masters that publish new ternary versions. The tech report describes the details.
• Learning rate: peak 4e-4 after an 8.7B-token warm-up, then constant, with a planned 1-sqrt decay to 0.1x of the peak between about 827B and 973B tokens. After gradient-noise instability, the rate was cut during the run to 0.5x (2e-4) on 2026-09-30 at 07:05 UTC and to 0.25x (1e-4) at 13:00 UTC.
• Batch: 10 gradient-accumulation micro-batches per step (about 8.2M tokens per fleet step) from the start, raised to 20 (about 16.4M tokens per step) on 2026-09-29 at 06:26 UTC. It ran at 40 from about 08:00 to 14:42 UTC on 2026-09-30, then returned to 20.
• Output logits: the learnable output logit scale is capped at 3.0 in training and inference (see Files and format).

Exports

One folder per export under exports/, named by training tokens (rounded down to whole billions) and fleet step. A new export is added about once an hour while the run continues. Published exports are never modified or removed.

Export	Tokens	Step	Size	val_mix (nats/token)	0-shot macro	5-shot macro	Exported (UTC)
881B-tokens_step60735 (latest)	881.5B	60735	2.52 GB	2.112	52.3	58.6	2026-10-03 10:27
878B-tokens_step60486	878.1B	60486	2.52 GB	2.111	52.1	58.5	2026-10-03 09:58
874B-tokens_step60210	874.4B	60210	2.52 GB	2.113	52.4	58.7	2026-10-03 09:26
871B-tokens_step59967	871.0B	59967	2.52 GB	2.115	51.8	58.7	2026-10-03 08:58
867B-tokens_step59683	867.2B	59683	2.52 GB	2.113	51.5	58.7	2026-10-03 08:29
863B-tokens_step59448	863.8B	59448	2.52 GB	2.114	50.5	58.4	2026-10-03 07:57
860B-tokens_step59172	860.1B	59172	2.52 GB	2.117	50.0	58.2	2026-10-03 07:27
856B-tokens_step58923	856.7B	58923	2.52 GB	2.120	50.2	58.2	2026-10-03 06:59
852B-tokens_step58655	853.0B	58655	2.52 GB	2.123	50.3	58.2	2026-10-03 06:28
849B-tokens_step58400	849.6B	58400	2.52 GB	2.121	50.7	58.1	2026-10-03 05:57
845B-tokens_step58124	845.8B	58124	2.52 GB	2.127	51.5	57.6	2026-10-03 05:25
842B-tokens_step57881	842.3B	57881	2.52 GB	2.115	51.5	57.8	2026-10-03 04:58
838B-tokens_step57601	838.7B	57601	2.52 GB	2.113	51.1	58.7	2026-10-03 04:24
835B-tokens_step57362	835.3B	57362	2.52 GB	2.114	51.2	58.3	2026-10-03 03:57
831B-tokens_step57096	831.5B	57096	2.52 GB	2.117	51.0	58.2	2026-10-03 03:26
828B-tokens_step56844	828.2B	56844	2.52 GB	2.119	50.8	58.6	2026-10-03 02:58
824B-tokens_step56567	824.4B	56567	2.52 GB	2.120	50.2	59.1	2026-10-03 02:26
821B-tokens_step56324	821.1B	56324	2.52 GB	2.115	49.6	58.3	2026-10-03 01:59
817B-tokens_step56049	817.2B	56049	2.52 GB	2.113	50.9	58.4	2026-10-03 01:24
813B-tokens_step55802	813.9B	55802	2.52 GB	2.110	51.5	58.1	2026-10-03 00:56
810B-tokens_step55544	810.2B	55544	2.52 GB	2.114	51.5	58.9	2026-10-03 00:27
806B-tokens_step55289	806.7B	55289	2.52 GB	2.113	51.8	58.3	2026-10-02 23:56
803B-tokens_step55019	803.0B	55019	2.52 GB	2.117	52.3	57.9	2026-10-02 23:27
799B-tokens_step54766	799.6B	54766	2.52 GB	2.116	52.1	58.7	2026-10-02 22:55
795B-tokens_step54482	795.8B	54482	2.52 GB	2.112	52.8	58.1	2026-10-02 22:25
792B-tokens_step54244	792.5B	54244	2.52 GB	2.113	52.7	57.4	2026-10-02 21:54
788B-tokens_step53974	788.7B	53974	2.52 GB	2.114	52.6	58.0	2026-10-02 21:28
777B-tokens_step53224	777.4B	53224	2.52 GB	2.113	52.1	58.1	2026-10-02 19:57
765B-tokens_step52447	765.5B	52447	2.52 GB	2.116	51.3	58.4	2026-10-02 18:25
761B-tokens_step52207	761.7B	52207	2.52 GB	2.118	51.3	58.2	2026-10-02 17:55
757B-tokens_step51950	757.6B	51950	2.52 GB	2.119	50.8	58.1	2026-10-02 17:28
753B-tokens_step51691	754.0B	51691	2.52 GB	2.111	51.3	58.1	2026-10-02 16:54
749B-tokens_step51422	749.9B	51422	2.52 GB	2.101	52.4	58.4	2026-10-02 16:25
746B-tokens_step51185	746.2B	51185	2.52 GB	2.097	53.2	59.1	2026-10-02 15:55
742B-tokens_step50915	742.1B	50915	2.52 GB	2.097	53.5	59.1	2026-10-02 15:26
738B-tokens_step50683	738.3B	50683	2.52 GB	2.106	53.6	59.5	2026-10-02 14:56
730B-tokens_step50159	730.5B	50159	2.52 GB	2.110	54.3	58.6	2026-10-02 13:55
726B-tokens_step49895	726.3B	49895	2.52 GB	2.107	54.3	58.9	2026-10-02 13:25
722B-tokens_step49657	722.5B	49657	2.52 GB	2.098	54.4	59.2	2026-10-02 12:56
718B-tokens_step49390	718.3B	49390	2.52 GB	2.101	54.3	58.9	2026-10-02 12:27
714B-tokens_step49149	714.6B	49149	2.52 GB	2.105	54.5	58.3	2026-10-02 11:56
710B-tokens_step48882	710.5B	48882	2.52 GB	2.090	54.2	58.8	2026-10-02 11:26
706B-tokens_step48637	706.8B	48637	2.52 GB	2.091	54.5	58.6	2026-10-02 10:55
702B-tokens_step48358	702.7B	48358	2.52 GB	2.090	54.8	59.3	2026-10-02 10:27
698B-tokens_step48133	698.9B	48133	2.52 GB	2.090	54.3	58.4	2026-10-02 09:57
694B-tokens_step47855	694.8B	47855	2.52 GB	2.090	54.9	58.5	2026-10-02 09:27
691B-tokens_step47616	691.1B	47616	2.52 GB	2.096	55.2	58.5	2026-10-02 08:56
686B-tokens_step47355	686.9B	47355	2.52 GB	2.095	54.8	58.9	2026-10-02 08:30
683B-tokens_step47102	683.2B	47102	2.52 GB	2.093	53.9	58.3	2026-10-02 07:55
678B-tokens_step46836	678.9B	46836	2.52 GB	2.094	54.6	58.8	2026-10-02 07:27
675B-tokens_step46600	675.4B	46600	2.52 GB	2.097	54.6	58.2	2026-10-02 06:57
671B-tokens_step46331	671.1B	46331	2.52 GB	2.096	54.2	57.9	2026-10-02 06:26
667B-tokens_step46093	667.4B	46093	2.52 GB	2.096	54.6	59.2	2026-10-02 05:57
663B-tokens_step45821	663.2B	45821	2.52 GB	2.091	54.8	59.0	2026-10-02 05:26
659B-tokens_step45574	659.4B	45574	2.52 GB	2.085	53.8	58.8	2026-10-02 04:55
655B-tokens_step45296	655.3B	45296	2.52 GB	2.084	54.4	59.3	2026-10-02 04:25
651B-tokens_step45052	651.6B	45052	2.52 GB	2.086	54.6	58.9	2026-10-02 03:54
647B-tokens_step44790	647.3B	44790	2.52 GB	2.084	54.3	58.9	2026-10-02 03:29
643B-tokens_step44546	643.6B	44546	2.52 GB	2.085	54.4	58.5	2026-10-02 02:56
639B-tokens_step44281	639.5B	44281	2.52 GB	2.088	54.8	58.4	2026-10-02 02:26
635B-tokens_step44034	635.8B	44034	2.52 GB	2.085	54.6	58.1	2026-10-02 01:57
631B-tokens_step43763	631.6B	43763	2.52 GB	2.086	53.7	58.2	2026-10-02 01:24
627B-tokens_step43524	627.9B	43524	2.52 GB	2.088	54.3	58.3	2026-10-02 00:56
623B-tokens_step43256	623.7B	43256	2.52 GB	2.088	54.4	58.6	2026-10-02 00:27
619B-tokens_step43018	620.0B	43018	2.52 GB	2.092	54.6	58.4	2026-10-01 23:55
615B-tokens_step42744	615.9B	42744	2.52 GB	2.095	54.7	58.6	2026-10-01 23:25
612B-tokens_step42498	612.0B	42498	2.52 GB	2.097	54.7	58.6	2026-10-01 22:56
607B-tokens_step42229	607.9B	42229	2.52 GB	2.099	54.9	58.9	2026-10-01 22:23
604B-tokens_step41990	604.2B	41990	2.52 GB	2.104	54.3	58.3	2026-10-01 21:55
600B-tokens_step41726	600.1B	41726	2.52 GB	2.085	53.3	57.4	2026-10-01 21:25
596B-tokens_step41482	596.3B	41482	2.52 GB	2.105	53.3	58.0	2026-10-01 20:56
592B-tokens_step41213	592.2B	41213	2.52 GB	2.082	53.3	57.5	2026-10-01 20:25
588B-tokens_step40976	588.4B	40976	2.52 GB	2.083	53.4	57.4	2026-10-01 19:56
584B-tokens_step40687	584.3B	40687	2.52 GB	2.083	53.7	58.3	2026-10-01 19:23
580B-tokens_step40465	580.5B	40465	2.52 GB	2.086	54.4	58.9	2026-10-01 18:55
576B-tokens_step40191	576.4B	40191	2.52 GB	2.089	54.2	58.8	2026-10-01 18:25
572B-tokens_step39952	572.6B	39952	2.52 GB	2.096	54.2	58.5	2026-10-01 17:56
568B-tokens_step39678	568.5B	39678	2.52 GB	2.093	55.0	58.6	2026-10-01 17:24
564B-tokens_step39426	564.8B	39426	2.52 GB	2.094	54.8	58.8	2026-10-01 16:56
560B-tokens_step39150	560.5B	39150	2.52 GB	2.096	54.5	58.7	2026-10-01 16:24
556B-tokens_step38922	556.8B	38922	2.52 GB	2.098	54.6	58.7	2026-10-01 15:54
552B-tokens_step38648	552.7B	38648	2.52 GB	2.100	55.0	59.0	2026-10-01 15:25
548B-tokens_step38408	548.9B	38408	2.52 GB	2.102	54.6	58.5	2026-10-01 14:55
544B-tokens_step38145	544.8B	38145	2.52 GB	2.102	54.7	59.0	2026-10-01 14:26
541B-tokens_step37898	541.1B	37898	2.52 GB	2.104	55.1	58.5	2026-10-01 13:56
536B-tokens_step37622	536.9B	37622	2.52 GB	2.106	54.7	58.0	2026-10-01 13:24
533B-tokens_step37388	533.1B	37388	2.52 GB	2.108	54.7	58.4	2026-10-01 12:56
528B-tokens_step37110	528.9B	37110	2.52 GB	2.110	55.4	58.3	2026-10-01 12:24
525B-tokens_step36875	525.3B	36875	2.52 GB	2.112	54.8	58.9	2026-10-01 11:55
521B-tokens_step36607	521.0B	36607	2.52 GB	2.115	54.8	57.9	2026-10-01 11:25
517B-tokens_step36367	517.3B	36367	2.52 GB	2.117	55.1	58.5	2026-10-01 10:56
509B-tokens_step35856	509.4B	35856	2.52 GB	2.122	55.2	58.5	2026-10-01 09:55
505B-tokens_step35576	505.2B	35576	2.52 GB	2.125	54.7	57.6	2026-10-01 09:23
501B-tokens_step35342	501.4B	35342	2.52 GB	2.106	53.7	57.2	2026-10-01 08:55
496B-tokens_step35030	496.8B	35030	2.52 GB	2.119	54.0	57.8	2026-10-01 08:19
493B-tokens_step34820	493.6B	34820	2.52 GB	2.118	54.2	57.7	2026-10-01 07:55
488B-tokens_step34509	488.7B	34509	2.52 GB	2.120	54.5	57.8	2026-10-01 07:18
485B-tokens_step34307	485.6B	34307	2.52 GB	2.121	54.2	57.7	2026-10-01 06:55
480B-tokens_step34005	480.8B	34005	2.52 GB	2.130	54.5	58.1	2026-10-01 06:22
477B-tokens_step33801	477.7B	33801	2.52 GB	2.128	54.0	57.9	2026-10-01 05:55
472B-tokens_step33502	472.9B	33502	2.52 GB	2.125	54.5	58.3	2026-10-01 05:20
469B-tokens_step33256	469.3B	33256	2.52 GB	2.136	54.1	58.1	2026-10-01 04:52
464B-tokens_step32984	465.0B	32984	2.52 GB	2.128	53.7	57.4	2026-10-01 04:24
461B-tokens_step32751	461.3B	32751	2.52 GB	2.127	53.8	57.8	2026-10-01 03:52
456B-tokens_step32472	457.0B	32472	2.52 GB	2.125	53.9	58.0	2026-10-01 03:22
453B-tokens_step32234	453.2B	32234	2.52 GB	2.128	53.5	57.7	2026-10-01 02:54
448B-tokens_step31961	449.0B	31961	2.52 GB	2.131	53.2	57.6	2026-10-01 02:21
445B-tokens_step31685	445.3B	31685	2.52 GB	2.129	53.9	57.7	2026-10-01 01:54
440B-tokens_step31416	440.9B	31416	2.52 GB	2.123	53.9	57.8	2026-10-01 01:19
437B-tokens_step31172	437.2B	31172	2.52 GB	2.122	53.5	58.0	2026-10-01 00:52
432B-tokens_step30892	432.8B	30892	2.52 GB	2.117	53.0	57.3	2026-10-01 00:19
429B-tokens_step30652	429.1B	30652	2.52 GB	2.119	53.4	57.6	2026-09-30 23:50
424B-tokens_step30372	424.6B	30372	2.52 GB	2.121	53.1	57.3	2026-09-30 23:18
420B-tokens_step30133	420.9B	30133	2.52 GB	2.122	52.9	57.2	2026-09-30 22:52
416B-tokens_step29838	416.6B	29838	2.52 GB	2.124	53.6	57.7	2026-09-30 22:18
412B-tokens_step29606	412.9B	29606	2.52 GB	2.127	53.1	57.5	2026-09-30 21:52
408B-tokens_step29319	408.5B	29319	2.52 GB	2.130	52.8	57.8	2026-09-30 21:19
404B-tokens_step29084	404.8B	29084	2.52 GB	2.138	52.7	57.4	2026-09-30 20:53
400B-tokens_step28806	400.5B	28806	2.52 GB	2.138	53.2	57.5	2026-09-30 20:20
396B-tokens_step28556	396.7B	28556	2.52 GB	2.142	53.4	57.4	2026-09-30 19:55
392B-tokens_step28270	392.4B	28270	2.52 GB	2.155	53.7	58.0	2026-09-30 19:19
388B-tokens_step28034	388.6B	28034	2.52 GB	2.164	53.5	58.4	2026-09-30 18:54
384B-tokens_step27749	384.2B	27749	2.52 GB	2.158	53.1	57.7	2026-09-30 18:20
380B-tokens_step27499	380.6B	27499	2.52 GB	2.155	53.0	57.2	2026-09-30 17:52
376B-tokens_step27217	376.2B	27217	2.52 GB	2.156	52.9	57.1	2026-09-30 17:19
372B-tokens_step26980	372.4B	26980	2.52 GB	2.166	53.3	57.2	2026-09-30 16:52
368B-tokens_step26688	368.1B	26688	2.52 GB	2.167	52.3	57.3	2026-09-30 16:19
364B-tokens_step26442	364.3B	26442	2.52 GB	2.170	52.6	56.9	2026-09-30 15:52
359B-tokens_step26154	359.9B	26154	2.52 GB	2.197	51.6	56.6	2026-09-30 15:21
355B-tokens_step26015	355.7B	26015	2.52 GB	2.196	51.2	56.5	2026-09-30 14:50
350B-tokens_step25847	350.7B	25847	2.52 GB	2.194	51.5	56.8	2026-09-30 14:19
346B-tokens_step25703	346.3B	25703	2.52 GB	2.193	51.2	56.8	2026-09-30 13:52
341B-tokens_step25550	341.3B	25550	2.52 GB	2.190	50.9	57.0	2026-09-30 13:17
337B-tokens_step25415	337.0B	25415	2.52 GB	2.188	51.3	57.2	2026-09-30 12:49
331B-tokens_step25251	332.0B	25251	2.52 GB	2.185	51.1	56.7	2026-09-30 12:19
125B-tokens_step13287	125.1B	13287	2.56 GB	2.247	52.2	54.6	2026-09-29 11:53
120B-tokens_step12998	120.6B	12998	2.56 GB	2.254	52.4	54.8	2026-09-29 11:19
116B-tokens_step12764	116.9B	12764	2.56 GB	2.260	51.7	53.8	2026-09-29 10:52
112B-tokens_step12482	112.3B	12482	2.56 GB	2.269	51.9	54.5	2026-09-29 10:20
108B-tokens_step12232	108.5B	12232	2.56 GB	2.276	51.8	54.3	2026-09-29 09:16
100B-tokens_step11713	100.3B	11713	2.56 GB	2.293	51.1	54.7	2026-09-29 08:16
91B-tokens_step11196	91.9B	11196	2.55 GB	2.312	51.1	53.4	2026-09-29 07:16
80B-tokens_step10079	80.2B	10079	2.55 GB	2.353	50.6	53.3	2026-09-29 05:43
70B-tokens_step8834	70.4B	8834	2.56 GB	2.387	49.6	51.3	2026-09-29 04:16
59B-tokens_step7496	60.0B	7496	2.56 GB	2.433	48.5	50.4	2026-09-29 02:43
50B-tokens_step6262	50.2B	6262	2.56 GB	2.489	47.5	49.1	2026-09-29 01:16
39B-tokens_step4937	39.8B	4937	2.56 GB	2.568	46.5	48.2	2026-09-28 23:43
30B-tokens_step3707	30.0B	3707	2.57 GB	2.695	44.7	46.0	2026-09-28 22:16
19B-tokens_step2392	19.7B	2392	2.58 GB	2.973	41.6	42.8	2026-09-28 20:43
10B-tokens_step1130	10.0B	1130	2.58 GB	3.944	36.6	38.2	2026-09-28 19:15

• val_mix: mean cross-entropy (nats per token, lower is better) on a fixed held-out validation set stratified over the nine sources of the epsilon mix (2048 windows of 4096 tokens). It is not comparable with the FineWeb-Edu validation number published for theta.
• 0-shot macro: mean over 11 tasks (ARC-Challenge, ARC-Easy, BoolQ, COPA, HellaSwag, LAMBADA, OpenBookQA, PIQA, SciQ, SIQA, WinoGrande). 5-shot macro: the same tasks without LAMBADA (10 tasks). Per task, acc_norm is used for ARC, HellaSwag, OpenBookQA, PIQA and SciQ, and acc for the rest. Values are percentages.
• The scores come from the project's own scorer, which runs the native ternary experts. They track progress within this run. Compare them with numbers from other evaluation harnesses with care.
• Benchmark prompts are scored with every run of two or more newlines collapsed to a single newline (including the few-shot separator).
• A - means the export has not been scored yet. The table fills in as scores arrive.

Latest export

exports/881B-tokens_step60735: 881.5B training tokens, step 60735.

hf download chutesai/parallax-8b-kappa --include "exports/881B-tokens_step60735/*" --local-dir parallax-8b-kappa
cd parallax-8b-kappa/exports/881B-tokens_step60735
tar -xf packed_experts.tar
sha256sum -c --quiet SHA256SUMS   # every file of the original export, byte for byte

Files and format

The files are in Parallax's native compact export format (inference only, no optimizer state). They are byte-identical to the export the training system produced:

File	Contents
manifest.json	export manifest: tensor inventory, per-file sha256 digests, token clock
model_config.json	model configuration
coverage.json, layouts.json	tensor coverage and expert frame layouts
indexer.bundle	sparse-attention indexer weights
relay_pack/	trunk (non-expert) weights in bf16, with their own manifest
packed_experts.tar	the 4096 routed experts (packed_experts/*.t24p, packed ternary codes and scales)
SHA256SUMS	sha256 of every file of the original export
export_info.json	step, tokens, time, sizes and digests of this export

The only change from the original export is packaging. The 4096 expert files are stored in one uncompressed tar to keep the repository's file count manageable. Extract it and check SHA256SUMS as shown above. Every upload was checked against the training system's own digests before and after it was published.

Logit-scale bound. The model's output logit scale is bounded: the forward pass uses exp(min(logit_scale_log, log 3.0)). The stored trunk tensor is the raw training parameter (it can sit slightly above the bound, e.g. from bf16 rounding), and each export records the bound in manifest.json (logit_scale: max, raw, effective) and in coverage.json (inference_policy.logit_scale_max). A loader must apply the recorded bound; the files themselves are left byte-identical.

Running it

This is a base model only. It is not chat- or instruction-tuned, so it does plain text completion: give it the start of a text and it continues it. It will not follow instructions or hold a conversation.

Standard transformers cannot load this format. Use our llama.cpp fork, https://github.com/chutesai/llama.cpp, which adds a dedicated runtime, llama-parallax, for CPU (x86-64, ARM64) and Apple GPUs (Metal):

git clone https://github.com/chutesai/llama.cpp && cd llama.cpp
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON   # -DGGML_METAL=OFF for CPU only
cmake --build build --target llama-parallax -j 8
python tools/parallax/run.py --binary build/bin/llama-parallax --model parallax-t9.gguf \
  --tokenizer tokenizer.json --backend metal --experts lut9 \
  --prompt 'The capital of France is' --predict 64

The runtime reads GGUF files converted from these exports. A ready-made GGUF is published at the repository root: parallax-8b-kappa-702B-tokens.gguf (2.21 GB, sha256 9eec18c9ff094c272ac789cbdf88e89a15f6381f0f77616f51d9486a2613bef2), converted from exports/702B-tokens_step48358 (702.7B training tokens, step 48358) with a bf16 trunk and the exact ternary experts. It is a one-off snapshot and is not updated with new exports; convert a newer export yourself for a later checkpoint. Earlier snapshots, still in the repository: parallax-8b-kappa-627B-tokens.gguf (from exports/627B-tokens_step43524), parallax-8b-kappa-612B-tokens.gguf (from exports/612B-tokens_step42498), parallax-8b-kappa-568B-tokens.gguf (from exports/568B-tokens_step39678), parallax-8b-kappa-528B-tokens.gguf (from exports/528B-tokens_step37110), parallax-8b-kappa-408B-tokens.gguf (from exports/408B-tokens_step29319), parallax-8b-kappa-368B-tokens.gguf (from exports/368B-tokens_step26688).

hf download chutesai/parallax-8b-kappa parallax-8b-kappa-702B-tokens.gguf --local-dir .

Use it as --model parallax-8b-kappa-702B-tokens.gguf in the command above. tools/parallax/README.md in the fork covers conversion, options and the tokenizer.

Tech report

The tech report for this run (kappa) is archived in this repository as tech_report.pdf. It is the version served on the project site from 2026-10-02 and is AI-generated from the project's measurements and logs. The project's current tech report is a living document and may describe a later run.

Limitations

This is an early base model. It can produce incorrect, biased or nonsensical output. It has no alignment or safety tuning, and it is small and far from converged.

License

MIT.

_{Table updated 2026-10-03 10:43 UTC.}