Instructions to use magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8")
model = AutoModelForMultimodalLM.from_pretrained("magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8

SGLang

How to use magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8 with Docker Model Runner:
```
docker model run hf.co/magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8
```

Qwen3.8-Flash-Next Quark MXFP4 + FP8

Mixed-precision derivative of amd/Qwen3.8-Flash-Next-Quark-MXFP4.

The AMD Quark checkpoint is retained as the base: language-model routed and shared MoE experts remain in OCP MXFP4, while additional high-memory BF16 components are converted to FP8 using precision/layout choices cross-checked against the official Qwen/Qwen3.8-Flash-Next-FP8 and nvidia/Qwen3.8-Flash-Next-NVFP4 checkpoints.

The objective is straightforward: reduce the remaining BF16 footprint while preserving the behavior of the high-quality AMD MXFP4 checkpoint.

Support

I’m a solo developer working full time for myself to achieve my dream. I build open source code on the side. If you like any of my work, buying me a coffee is always appreciated. Otherwise, I hope you enjoy, maybe give me a star or something—or just send good vibes.

Click here to see ways to support — BTC, PayPal, GitHub Sponsors.

Quantization Layout

The source AMD checkpoint already provides:

OCP MXFP4 static weights for language-model routed MoE experts
OCP MXFP4 static weights for the shared MoE expert
Dynamic MXFP4 expert activations
BF16 attention, PLE, MTP, router/gating paths, vision tower, hyper-connections, and lm_head

This derivative additionally converts:

Component	Source	This model
Main routed/shared MoE experts	OCP MXFP4	OCP MXFP4, preserved
PLE n-gram embedding	BF16	FP8 E4M3FN
MTP routed expert projections	BF16	Quark FP8, 128×128 block
Attention / linear attention	BF16	BF16
Router and shared-expert gates	BF16	BF16
Vision tower	BF16	BF16
`lm_head`	BF16	BF16

PLE

The 128 physical PLE embedding shards are quantized to FP8 E4M3FN using one shared FP32 scale across the complete logical embedding.

MTP

The 1,536 MTP routed-expert projections are stored using Quark-compatible 128×128 block FP8 with per-weight FP32 scale tensors.

Reference-guided layout

The additional FP8 targets were selected by comparing the corresponding tensor layouts of:

amd/Qwen3.8-Flash-Next-Quark-MXFP4
Qwen/Qwen3.8-Flash-Next-FP8
nvidia/Qwen3.8-Flash-Next-NVFP4

Weights are quantized from the AMD Quark checkpoint; weights are not copied from the FP8 or NVFP4 reference checkpoints.

Size

Approximate tensor payload:

Checkpoint	Payload
AMD Quark MXFP4 source	169.72 GiB
This model	119.69 GiB
Reduction	50.03 GiB

The original MXFP4 payload is preserved. The reduction comes primarily from the PLE BF16 → FP8 conversion, with additional savings from the MTP FP8 conversion.

Validation

AMD source checkpoint

AMD reports the following GSM8K result for the source checkpoint:

Model	GSM8K, 5-shot
`Qwen/Qwen3.8-Flash-Next`	96.8
`amd/Qwen3.8-Flash-Next-Quark-MXFP4`	96.5
Recovery	99.7%

The AMD checkpoint is a post-training Quark MXFP4 quantization of Qwen3.8-Flash-Next.

Full-vocabulary logit comparison

This model was compared directly against amd/Qwen3.8-Flash-Next-Quark-MXFP4 using teacher-forced, full-vocabulary logits.

KL direction:

KL(reference || candidate)

Metric	Result
Scored tokens	28
Mean KL divergence	0.01542 nats/token
P95 KL	0.03851
P99 KL	0.04607
Maximum KL	0.04850
Top-1 agreement	85.71%
Reference NLL	5.44837
Candidate NLL	5.41924
NLL delta	-0.02913
Perplexity ratio	0.97129

KLD was measured on the standard base-model forward; speculative MTP decoding was not exercised.

FP8 reconstruction

Converted tensors were also validated numerically after quantization. Sampled FP8 weight reconstruction produced approximately:

cosine similarity: ~0.99965
relative RMSE: ~0.026
observed clipping: 0

Model Architecture

Architecture: Qwen4ExpForConditionalGeneration
Input: Text, Image, Video
Output: Text
Base architecture: Qwen3.8-Flash-Next
Source quantization: AMD Quark OCP MXFP4
Additional quantization: FP8 E4M3FN / Quark block FP8
Format: Safetensors

Lineage

Qwen/Qwen3.8-Flash-Next

→ amd/Qwen3.8-Flash-Next-Quark-MXFP4

→ this mixed MXFP4 + FP8 checkpoint

See the upstream Qwen and AMD model cards for architecture details, original evaluation results, usage guidance, and licensing information.

License

This is a derivative checkpoint of amd/Qwen3.8-Flash-Next-Quark-MXFP4, itself derived from Qwen/Qwen3.8-Flash-Next.

Refer to the upstream AMD and Qwen repositories for the applicable licenses and terms.

Downloads last month: 19

Safetensors

Model size

119B params

Tensor type

BF16

I64

F8_E4M3

Model tree for magiccodingman/Qwen3.8-Flash-Next-Quark-MXFP4-fp8

Base model

Qwen/Qwen3.8-Flash-Next

Quantized

amd/Qwen3.8-Flash-Next-Quark-MXFP4

Quantized

(1)

this model