Instructions to use proofofvitalik/Sovereign-V2-MLX-Q4-Companion with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use proofofvitalik/Sovereign-V2-MLX-Q4-Companion with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("proofofvitalik/Sovereign-V2-MLX-Q4-Companion") config = load_config("proofofvitalik/Sovereign-V2-MLX-Q4-Companion") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use proofofvitalik/Sovereign-V2-MLX-Q4-Companion with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "proofofvitalik/Sovereign-V2-MLX-Q4-Companion"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "proofofvitalik/Sovereign-V2-MLX-Q4-Companion" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use proofofvitalik/Sovereign-V2-MLX-Q4-Companion with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "proofofvitalik/Sovereign-V2-MLX-Q4-Companion"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default proofofvitalik/Sovereign-V2-MLX-Q4-Companion
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use proofofvitalik/Sovereign-V2-MLX-Q4-Companion with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "proofofvitalik/Sovereign-V2-MLX-Q4-Companion"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "proofofvitalik/Sovereign-V2-MLX-Q4-Companion" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Sovereign V2 MLX + Companion
A local 27B text and vision model for Apple Silicon, with optional coding and agent tools.
Sovereign V2 MLX packages the Heretic ARA 27B model as standard MLX files for use with LM Studio's MLX engine. Companion adds optional workflows for coding, tasks and other enabled tools through a separately installed compatible Hermes host. Model inference runs locally; network tools connect to their respective services.
ARA 27B ยท MLX Q4 text + BF16 vision ยท LM Studio ยท optional MTP ยท Companion 0.3.5.4 MLX RC1
Key features
- Text and images. Ask questions, write or edit text, and reason about image content in a compatible client.
- Local coding workflows. Generate Python, validate syntax and imports, run supplied checks, and repair a copy while preserving the original.
- Optional agent tools. A compatible Hermes host and Companion can execute enabled tools and return their results to the model; the model alone does not execute code.
- Direct answers by default. The included chat template disables thinking by default; compatible clients can explicitly enable it.
- Standard MLX + optional MTP. The main text model uses 4-bit weights; the vision component is BF16. The included MTP draft can accelerate supported requests through an experimental custom LM Studio runtime or the separate local-server generator.
Companion tools
| Tool | What it does |
|---|---|
| Compact Repair | Proposes targeted Python edits, checks the candidate and writes a new copy. |
| Code Guard | Checks Python syntax and isolated imports, and can run supplied tests. |
| Task and profile controls | Organize supported workflows and select enabled tool sets in a compatible host. |
| Research integrations | Optional integrations can retrieve external information; some require credentials and send requests to their providers. |
Companion is an optional source distribution, not the Hermes desktop application. It needs a compatible separately installed host. The scoped live checks below cover a status tool call and the two code workflows, not every integration or a clean desktop installation.
Experimental native MTP for LM Studio
Use the ordinary Attach button with Sovereign text and vision in LM Studio, with no auxiliary vision model. The experimental native MTP package creates a separate custom copy of the official MLX 1.13.1 runtime, links the downloaded model weights and adds a preset. This is not an official LM Studio backend; the original runtime is preserved.
On an M4 Max / 48 GB / macOS 26.6.2 Mac, nine paired engine-level runs (three short prompts, three repetitions, 220 output tokens each) measured a median 26.85 tokens/s without MTP and 32.10 tokens/s with MTP, using full request wall time including prompt processing. The median per-pair speed ratio was 1.365ร. All nine paired token sequences matched. This is a small text-only sample measured inside the engine, not in the LM Studio GUI; results vary by workload and Mac. Method and raw results.
The Sovereign V2 MLX MTP preset uses temperature 0 and disables Repeat Penalty. MTP is used only for compatible greedy requests without additional logits processors. Tool calls with the reasoning guard, stochastic sampling, penalties and structured-output requests use ordinary generation and are not accelerated by this prototype. The included launcher uses an 8,192-token context, so the previously documented 65,536-token Companion setup is not established for this native prototype.
Install the native prototype
Download the complete model including
mtp/. Install LM Studio and itslmsCLI. With LM Studio running, obtain the exact official backend:lms runtime get 'mlx-llm@1.13.1' --yes.Download the native MTP package and verify SHA-256
66229d50853942c2ac6d0311dffdc98dc129027a96bc2a556e67cfaf03d5eaff. Fully quit LM Studio, including its menu-bar process. Extract the package and run:cd Sovereign-MLX-Native-MTP-experimental python3 install.py --model-dir '/absolute/path/to/downloaded/model'Reopen LM Studio, run
Start-Sovereign-Native-MTP.command, selectsovereign-v2-mlx-mtp-native, and select theSovereign V2 MLX MTPpreset. The model view links to the downloaded files, so leave the original model directory in place. Run the launcher again after restarting LM Studio. Detailed instructions and verification.
The native prototype and ordinary Attach were tested on the Mac above. Older macOS, a clean first installation on a different Mac, minimum memory and the full Companion workflow with this prototype remain unverified. The separate-server package below remains available for its documented workflow.
Alternative: separate-server MTP generator
The included MTP weights work with the separate experimental Sovereign MTP package. It runs an MLX-VLM server locally and connects it to LM Studio using a generator plugin; it does not enable MTP in LM Studio's built-in MLX engine. The package uses the model files already downloaded and does not include a second copy of the weights.
On the tested M4 Max / 48 GB Mac, six matched short requests had a median of 25.3 output tokens/s without MTP and 36.8 tokens/s with MTP, measured over full request wall time. Results vary by Mac, prompt and configuration. The old macOS target, clean first install on an older Mac and full 65,536-token input are not yet verified.
Set up the separate-server generator
Requires Apple Silicon, macOS 14 or later, LM Studio, the complete model including mtp/, Python 3.11, Node.js 22+ and npm. The macOS 14 wheel tags and binaries were inspected, but the package itself was tested only on macOS 26.6.2. The package's minimum RAM requirement is not established. Initial setup downloads pinned Python packages and npm dependencies.
Download the MTP package and verify its SHA-256:
f0a9204ac715179dbbdab6e71343556bdd248d91db09b2364fc9421ddb3a422c. Checksums.Unpack it and prepare the isolated environment and LM Studio plugin:
unzip Sovereign-MLX-MTP-experimental.zip cd Sovereign-MLX-MTP-experimental ./prepare.sh ./install-plugin.shThe installer refuses to replace an existing plugin of the same name. For a previous test installation, remove or update that plugin deliberately before installing the new one; see MTP-GUIDE.md.
Start the local server, supplying the actual directory of your downloaded Sovereign MLX model:
./start-mtp.sh '/absolute/path/to/Sovereign-V2-MLX-Q4-Companion'Wait for
Drafter ready; speculative decoding enabled.In LM Studio chooseproofofvitalik/sovereign-mlx-mtp-localfrom the model selector. The server listens on127.0.0.1:18027; stop it with Ctrl-C in the terminal where it was started. Avoid loading a second copy of the 27B model in LM Studio at the same time.
Images with the separate-server generator
LM Studio currently rejects the ordinary Attach button for generator plugins, even when the model supports vision. For this MTP generator, put the image's absolute path directly in your own message:
What is shown in this image? [[image:/Users/yourname/Pictures/example.png]]
PNG, JPEG, WebP, GIF and BMP are supported, up to 20 MiB per image. Spaces in paths are supported; ] and line breaks in a path are not. Only explicitly marked user-message paths are read and sent to the local loopback server. This is a workaround for the current LM Studio interface, not support for its Attach button. Both engineering-dashboard images below passed through the actual LM Studio chat using this method. Full setup, Companion and troubleshooting.
Demonstrations
The videos below are previously published Sovereign demonstrations. They illustrate the workflows and vision task, but were recorded with the Splash/Desktop release, not this MLX build. The MLX results are reported separately under Verification.
Code tools: Compact Repair + Code Guard
See a targeted repair, preservation of the original file and checks against supplied assertions.
Watch or download the code tools video ยท English subtitles ยท ะ ัััะบะธะต ััะฑัะธััั
Vision: engineering release review
The task combines a dependency map, service table, latency chart and release rules. The model must distinguish an optional degraded service from the required path and apply the p95 latency gate. This video explains the earlier Splash run; both images were independently retested with this MLX candidate.
Watch or download the vision video ยท MLX test images and results
Compatibility and setup
LM Studio lists Apple Silicon and macOS 14 or newer as its application requirements. This particular model and Companion have been tested on M4 Max, 48 GB, macOS 26.6.2, with LM Studio MLX backend 1.14.0. Older macOS, smaller-memory Macs, and a clean first installation have not been tested. LM Studio's general 16 GB recommendation is not a memory requirement for this 27B package; the minimum memory for this model is not established.
- Install LM Studio on a compatible Apple Silicon Mac and install a compatible MLX runtime.
- Download this repository's complete model files, open the model in LM Studio and select the MLX engine. LM Studio detected approximately 16.07 GB for the main model; the complete repository with the MTP weights and supporting files occupies approximately 16.9 GB. Start a chat; enable its local API if an external client needs it.
- For Companion, install the Companion source archive only into a new isolated profile with a compatible Hermes host. Follow COMPANION.md. Load the model with a 65,536-token context, identifier
proofofvitalik/sovereign-v2-mlx, and start the local API athttp://127.0.0.1:1234/v1. Hermes requires at least 64,000 context tokens. A full 65,536-token prompt has not been tested.
The host, LM Studio and their dependencies are separate downloads. The tested Companion installation reused dependencies from an existing matching installation. Installation on a clean Mac and the desktop UI remain to be validated.
Speed and acceleration
On the M4 Max / 48 GB test Mac, the ordinary LM Studio MLX backend produced a median 27.5 output tokens/s over six short text requests, with a 26.1โ28.4 tokens/s range. The metric divides output tokens by full request wall time, including prompt processing. It is not directly comparable with engine-only decode rates or the earlier Splash results. Speed and memory use depend on hardware, context, prompt and settings.
For the optional MTP package, a separate MLX-VLM server built from macOS 14-tagged wheels measured median 25.3 tok/s without MTP and 36.8 tok/s with MTP across six matched greedy text requests. Each MTP output matched its baseline on this small set. LM Studio chat uses this separate local server through a generator; the unmodified official native MLX engine was not shown to activate MTP for this model in that test. The experimental custom engine above is a separate integration. The earlier three-prompt MTP/DFlash2 experiment used a different environment and is documented separately. Older macOS compatibility and speed are unverified. Measurement details and raw results.
Verification
- Text and tools: direct-answer and tool-call smoke tests passed. A live isolated Hermes session called
sovereign_status, received the result and answered. Code Guard passed syntax, isolated import and three functional assertions; Compact Repair passed four assertions while preserving the source file. - Vision: both synthetic release dashboards passed all 10 checked fields each through LM Studio. Candidate r42 at 310 ms was correctly marked NO-GO; r43 at 220 ms was correctly marked GO, with the degraded Search dependency treated as optional. Images, responses and checks.
- Known limitation: in one request for roughly 80 lines of Python, the model returned 339 lines despite valid syntax. Use Code Guard with independent tests and line constraints when exact output matters; passing a check is not a proof of general correctness.
- Native MTP GUI: ordinary Attach worked with a dashboard image, and the Stop generating button stopped an MTP request; a subsequent request succeeded. A tool-call round trip used the safe non-MTP fallback. Scope and limitations.
- Separate-server MTP GUI and shutdown: text and Code Guard were checked in LM Studio chat; both complex images were answered correctly using the explicit image-path workaround. The repaired server completed three loaded-model stop cycles and a further stop after the GUI tests without a crash. Longer stress testing remains outstanding. MTP verification.
- Scope: these are small authored checks, not a comprehensive benchmark. Full-length context, clean installation on older hardware, and older macOS compatibility remain unverified.
Provenance and license
The base is Heretic ARA 27B, revision 2dc9b364104881cbb85e390f00195ba6b9d745e9. The standard MLX weights come from Sovereign's earlier MLX conversion; weight hashes and component details are in RELEASE-QA.md. The model and Companion source use Apache-2.0; third-party components retain their own licenses. Review generated content and code before use.
- Downloads last month
- 35
4-bit
