Instructions to use AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0 # Run inference directly in the terminal: llama cli -hf AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0 # Run inference directly in the terminal: llama cli -hf AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
Use Docker
docker model run hf.co/AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
- Llama new
- LM Studio
- Jan
- Ollama
How to use AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp with Ollama:
ollama run hf.co/AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
- Unsloth Desktop
- Docker Model Runner
How to use AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp with Docker Model Runner:
docker model run hf.co/AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
- Lemonade
How to use AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
Run and chat with the model
lemonade run user.Qwen3-ASR-1.7B-AX8860-llama.cpp-Q8_0
List all available models
lemonade list
- Atomic Chat
Commit History
7 days ago
-
static bins from llama.cpp axera-dev 0104880be (shared sequence state, media in batch, agent protocol fixes)
Verified signaturec49018e -
README: drop the kernel source line
Verified signature75f9577 -
static bins from llama.cpp axera-dev b0b968339 (shared sequence state, media in batch, agent protocol fixes)
Verified signature6c1b276
11 days ago
-
rel1109: unified static bins (axera-dev 2f81e2416)
Verified signaturea5874e1
15 days ago
-
static bins from llama.cpp axera-dev 57a30bd (one build for every model)
Verified signaturefe4a7da -
static bins from llama.cpp axera-dev 44b3529 (one build for every model)
Verified signature31aff98
17 days ago
-
decode kernel reads the prefill weight banks (one copy of the layer weights in CMM); bin (llama.cpp axera-dev 894a78a, ax-dsl-kernels 804a260); README
Verified signature1cb3135 -
bin: load host memory (GGUF pages dropped once converted, packs built layer by layer); llama.cpp axera-dev cff4752; README
Verified signature0be0b70
18 days ago
-
Update README.md
Verified signaturea07d615 -
README: the audio encoder's source type (Q8_0 in the mmproj)
Verified signature9455cb2 -
README: drop the DDR rate (a stage setting)
Verified signature9197f3d
19 days ago
-
decode group: softmax without the trailing sentinel column (ax-dsl-kernels 60dd25b)
Verified signature0f80a61
20 days ago
-
Qwen3-ASR on AX8860: audio encoder, text prefill and decode kernels, static programs
Verified signatured2f5d53 -
LFS rules
Verified signaturee23b00b -
initial commit
Verified signature4a5a92f