Instructions to use AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0 # Run inference directly in the terminal: llama cli -hf AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0 # Run inference directly in the terminal: llama cli -hf AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
Use Docker
docker model run hf.co/AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
- Llama new
- LM Studio
- Jan
- Ollama
How to use AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp with Ollama:
ollama run hf.co/AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
- Unsloth Desktop
- Docker Model Runner
How to use AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp with Docker Model Runner:
docker model run hf.co/AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
- Lemonade
How to use AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AXERA-TECH/Qwen3-ASR-1.7B-AX8860-llama.cpp:Q8_0
Run and chat with the model
lemonade run user.Qwen3-ASR-1.7B-AX8860-llama.cpp-Q8_0
List all available models
lemonade list
- Atomic Chat
README: drop the kernel source line
Browse files
README.md
CHANGED
|
@@ -60,7 +60,6 @@ The test recordings' transcripts are identical to those of the previous NPU impl
|
|
| 60 |
## 源码 / Sources
|
| 61 |
|
| 62 |
- llama.cpp:[ZHEQIUSHUI/llama.cpp](https://github.com/ZHEQIUSHUI/llama.cpp) `axera-dev` @ `b0b968339`(`ggml/src/ggml-axera/models/qwen3`、`qwen3_asr`)
|
| 63 |
-
- kernel:[ZHEQIUSHUI/ax-dsl-kernels](https://github.com/ZHEQIUSHUI/ax-dsl-kernels) `main` @ `b60fc30`(`models/qwen3_asr`)
|
| 64 |
|
| 65 |
## 更新 / Changelog
|
| 66 |
|
|
|
|
| 60 |
## 源码 / Sources
|
| 61 |
|
| 62 |
- llama.cpp:[ZHEQIUSHUI/llama.cpp](https://github.com/ZHEQIUSHUI/llama.cpp) `axera-dev` @ `b0b968339`(`ggml/src/ggml-axera/models/qwen3`、`qwen3_asr`)
|
|
|
|
| 63 |
|
| 64 |
## 更新 / Changelog
|
| 65 |
|