Sweep Next Edit
Collection
Locally running next edit autocomplete • 3 items • Updated • 5
How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sweepai/sweep-next-edit-0.5B:Q8_0# Run inference directly in the terminal:
llama cli -hf sweepai/sweep-next-edit-0.5B:Q8_0Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf sweepai/sweep-next-edit-0.5B:Q8_0# Run inference directly in the terminal:
./llama-cli -hf sweepai/sweep-next-edit-0.5B:Q8_0Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf sweepai/sweep-next-edit-0.5B:Q8_0# Run inference directly in the terminal:
./build/bin/llama-cli -hf sweepai/sweep-next-edit-0.5B:Q8_0Use Docker
docker model run hf.co/sweepai/sweep-next-edit-0.5B:Q8_0Quick Links
Sweep Next-Edit 0.5B (GGUF)
A 0.5B parameter model for next-edit autocomplete, quantized to Q8_0 GGUF format.
Model Description
Sweep Next-Edit predicts your next code edit before you make it. It runs locally on your laptop in under 500ms (with speculative decoding) and outperforms models over 4x its size on next-edit benchmarks. More details here.
Usage
Download run_model.py and the model file, then:
uv pip install llama-cpp-python huggingface_hub
python run_model.py
Model Details
- Format: GGUF (Q8_0 quantization)
- Parameters: 0.5B
- Context Length: 8192 tokens
- Base Model: Qwen2.5-Coder
Example
The model uses a specific prompt format with file context, recent diffs, and current state to predict the next edit. See run_model.py for a complete example.
Links
- Blog Post - Technical details and benchmarks
- JetBrains Plugin - Sweep AI JetBrains Plugin
- HN Thread - Discuss implementation for VSCode, Neovim & Emacs
- Twitter Post - Ask us any other questions
License
Apache 2.0
- Downloads last month
- 392
Hardware compatibility
Log In to add your hardware
8-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf sweepai/sweep-next-edit-0.5B:Q8_0# Run inference directly in the terminal: llama cli -hf sweepai/sweep-next-edit-0.5B:Q8_0