Instructions to use IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF
# Run inference directly in the terminal:
llama cli -hf IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF
# Run inference directly in the terminal:
llama cli -hf IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF
# Run inference directly in the terminal:
./llama-cli -hf IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF
# Run inference directly in the terminal:
./build/bin/llama-cli -hf IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF

Use Docker

docker model run hf.co/IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF

LM Studio
Jan

vLLM

How to use IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF

Ollama
How to use IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF with Ollama:
```
ollama run hf.co/IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF
```
Unsloth Desktop

How to use IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF with Docker Model Runner:
```
docker model run hf.co/IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF
```

Lemonade

How to use IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF

Run and chat with the model

lemonade run user.Iris-mini-APEX-I-MiniPlus-V1-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "IsValorum/Iris-mini-APEX-I-MiniPlus-V1-GGUF" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

ARCHITECTURE SELECTION GUIDE — MINIPLUS V1 & V2.1 EDITIONS

This repository hosts the MiniPlus V1 edition of Iris-mini. Our releases are precision-engineered for specific hardware budgets and memory topologies. V1 is NOT obsolete or inferior; it represents our leanest, most agile operating profile:

MiniPlus V1 (Lean & Agile Profile): Highly compact footprint with uncompressed F32 router gates, a fully armored Q6_K output head, Q8_0 attention gates, and IQ3_XXS core experts. Both V1 and V2.1 run flawlessly with the vast majority of the model residing in system RAM (DDR4/DDR5), thanks to linear CPU-friendly vectorization that avoids lookup stalls. V1 is dramatically superior to generic community APEX-I-Mini releases (which crush core reasoning down to 2-bit IQ2_S) and flat 3-bit quants.

MiniPlus V2.1 (System RAM Streaming Specialist with Deep Context): Specially prepared to run totally or partially in system RAM (DDR4/DDR5) across deep mathematical and reasoning contexts (up to 256k tokens). Upgrades all 40 shared foundation experts to Q5_K, armors attention gates in Q8_0, and uses linear CPU-friendly vectorization that eliminates AVX2 lookup stalls. Depending on your processor and memory bandwidth (DDR4/DDR5), streaming generation in system RAM can be almost as fast as having everything in VRAM, while supporting deep context allocating GPU VRAM to the active KV cache while model weights stream from system RAM. All for only approx. 180 MB more (approx. 13.74 GiB vs approx. 13.56 GiB)—an overhead that is completely negligible in system RAM.

Which one should you choose? (Official Recommendation: V2.1)

⭐ PRIMARY RECOMMENDATION — Iris-mini APEX-I-MiniPlus V2.1: For virtually all users and deployments, V2.1 is the strictly recommended release. V2.1 reports a Perplexity of 5.3735 ± 0.1214 (ΔPPL ≈ +0.06 from unquantized baseline (approx. 5.31)), matching the token fidelity of Q5_K / Q6_K class quantizations while weighing only approx. 14.7 GB (same footprint as Q3_K_M). Furthermore, it eliminates AVX2 CPU stalls and is optimized for system RAM offload.

MiniPlus V1 Legacy: Maintained for architectural transparency and users seeking specialized configurations for their workflow.

Both editions are handcrafted and vastly outperform flat 3-bit quants and generic community APEX-I-Mini releases. To explore or download the V2.1 edition of Iris-mini, visit: IsValorum/Iris-mini-APEX-I-MiniPlus-V2.1-GGUF

DO NOT CONFUSE APEX-I-MINIPLUS WITH GENERIC COMMUNITY APEX-I-MINI!

Regardless of release version (whether V1, V2, or V2.1), NEVER confuse handcrafted APEX-I-MiniPlus builds with generic community APEX-I-Mini releases:

Generic Community APEX-I-Mini: Uniformly compresses all core MoE experts down to aggressive 2-bit IQ2_S (dropping below the critical quality floor), leaves the sensitive token output head unarmored at 3-bit Q3_K_M, and compresses attention projections down to Q3_K. In deep reasoning models, this triggers severe perplexity spikes, syntax errors, and broken code brackets.

Handcrafted APEX-I-MiniPlus (All Editions by IsValorum): Every single MiniPlus release—from V1 and V2 to V2.1—is a custom tensor-by-tensor architecture that preserves uncompressed F32 router gates, armors the token output head in high-precision Q6_K, safeguards attention gates in Q8_0, and keeps core reasoning experts at or above calibrated 3-bit (IQ3_XXS/IQ3_S). Even our earlier builds vastly outperform generic community APEX recipes and flat 3-bit quants.

Quick Navigation Index

Model Files & Technical Specifications
Comparative Quantization Analysis (vs. Flat Quants & Generic APEX)
Independent Benchmark of the APEX-I-MiniPlus Family (Occamy V2 Reference)
Native Multi-Token Prediction (MTP) Co-Pilot
Everyday Laptop Guidance (DDR4 / DDR5 RAM)
The 24GB Miracle: Full 256K Context Runs In VRAM!
Hardware Throughput & Offload Benchmarks (RTX 30 / 40 / 50)
Recommended Generation Parameters (Creator Official)
Model Inherent Behavior vs. Quantization Fidelity Notice

🏅 Independent Benchmark of the APEX-I-MiniPlus Family (Occamy V2 Reference)

External report: zephel01 independently benchmarked Occamy V2. The benchmark below was performed on Occamy-1.0 APEX-I-MiniPlus V2, not on this specific model. It is included as independent evidence of the broader MiniPlus quantization approach.

The APEX-I-MiniPlus quantization architecture powering this model was subjected to an extensive independent evaluation by Japanese AI researcher and evaluator zephel01 (CoolZero) on an NVIDIA RTX 5090 (32GB) workstation running llama.cpp CUDA b11027 with FlashAttention (-fa on -ctk q8_0 -ctv q8_0 -ngl 99).

The evaluation tested the APEX-I hybrid MoE engine across 348 unseeded trials on SWE-bench style multi-file Python bug-fixing tasks with hidden pytest suites (llmbench):

L6 Multi-File Code Generation (60 tasks):
- Context 32,768 (32K): 93.3% Resolved (46/60 tasks passed 5/5 consecutive trials; 20/20 on Easy–Hard).
- Context 65,536 (65K): 90.0% Resolved (45/60 tasks passed 5/5 consecutive trials).
- Match with 25–28 GB Models: Matches or exceeds the resolution rate of full 25–28 GB models (such as Ornith-1.5 and Tiel-Coder 35B-A3B) while consuming over 10 GB less VRAM (14.6 GB vs approx. 26 GB).
Extreme Context VRAM Scaling (The Hybrid DeltaNet SSM Advantage):
- 32K Context: 14.6 GB total VRAM allocation.
- 65K Context: 15.1 GB total VRAM allocation (only +0.5 GB VRAM added when doubling context!).
- Architectural Explanation: Because 30 of the 40 layers utilize Linear Attention / DeltaNet SSM ($O(1)$ constant recurrence memory), only the 10 full-attention anchor layers expand the KV cache. This proves empirically that 65,536 context runs 100% in VRAM on consumer 16GB GPUs (RTX 4080 / RTX 5080) without offloading to system RAM.
Measured Real-World Throughput: Sustained single-stream generation of approx. 247 – 251 tok/s on NVIDIA RTX 5090.

Model Files & Specifications

File Name	File Size	Memory Footprint	BPW	Description
`Iris-mini-MTP.APEX-I-MiniPlus.gguf`	`14.84 GB`	`13.82 GiB`	3.42 BPW	Handcrafted language, math, reasoning & native MTP draft head

Base Architecture: qwen35moe (35B total parameters, approx. 3.2B active per token).
Speculative Decoding: Fully preserved native Multi-Token Prediction head (blk.40).
Target Precision: Armored boundaries (Q3_K), calibrated core experts (IQ3_XXS), 6-bit uncompromised output head (Q6_K).

Comparative Quantization Analysis (vs. Flat Quants & Generic APEX)

Also, don't confuse APEX-I-MiniPlus (Standard) with a generic baseline APEX-I-Mini. Traditional APEX-I-Mini drops core experts aggressively to 2-bit IQ2_S and leaves output.weight at 3-bit Q3_K_M, which creates a noticeable perplexity hit on complex reasoning tasks. Standard MiniPlus avoids that degradation floor while keeping boundary layers in linear Q3_K for single-cycle vectorized AVX2 CPU dequantization (optimized for DDR4/DDR5 laptop streaming), while protecting output in Q6_K and routers in F32.

To put the numbers in perspective: this cuts nearly 2 GB off a flat 3-bit quant (approx. 15.6 GB), and weighs only about approx. 1 GB more than a generic APEX-I-Mini (approx. 12.5 GB). For that single extra gigabyte of VRAM, you get a massive jump in reasoning and syntactic stability while maximizing CPU/RAM execution throughput.

Take a look at the tensor-by-tensor comparison table below to inspect the exact architectural differences and see why this specific allocation is optimal. That's specifically what this was built for:

Architectural Component	Generic Automated Quants (Flat `Q3_K_S` / `IQ3_S`)	Generic APEX-I-Mini (Baseline Recipe)	Our Handcrafted APEX-I-MiniPlus (Standard / IsValorum)	Perceived Quality & Real-World Impact
Output Head (`output.weight`)	Flat `IQ3_S` / `Q3_K_S` (approx. 3.44 BPW)	Inherits base type `Q3_K_M` (approx. 3.44 BPW unarmored)	`Q6_K` (approx. 6.56 BPW uncompromised)	Eliminates Syntax & Vocabulary Hallucinations: Low-bit output heads cause tokenizer classification noise, breaking code indentation, brackets (`{}`, `[]`), math symbols, and domain terms. `Q6_K` preserves near-FP16 output classification.
Expert Routers (`ffn_gate_inp.weight`)	Blindly quantized to 3-bit / unoptimized	Inherits base type `Q3_K_M` (approx. 3.44 BPW compressed)	`F32` uncompressed (32.0 BPW, 2 MB/layer)	Zero Router Drift: In micro-expert models, even minuscule quantization errors in router logits misdirect tokens to wrong experts. Retaining uncompressed `F32` guarantees 100% routing fidelity with virtually zero memory overhead (approx. 80 MB total).
Attention & Language (`attn_output`, `attn_qkv`)	Flat `IQ3_S` / `Q3_K_S`	`Q3_K` on 34 middle layers (L3–36), `Q4_K` on 6 edge layers	`Q6_K` for `attn_output`, `Q3_K` / `Q4_K` + `imatrix`	Contextual Precision & CPU Throughput: Combines uncompromised `Q6_K` for the output projection with fast vectorized linear blocks for attention, balancing retrieval accuracy with maximum token streaming speed on CPU/RAM.
Attention Gates (`attn_gate.weight`)	Blindly compressed to 3-bit	Compressed to `Q3_K` (middle) / `Q4_K` (edges)	`Q4_K` / `Q8_0` (linear high-precision)	Attention Routing Dynamics: High-precision linear gating modulating query-key projections without CPU dequantization latency.
*Shared Foundation Expert (`ffn__shexp`)**	Flat `IQ3_S` / `Q3_K_S` (3.44 BPW)	Linear `Q4_K` (middle) / `Q5_K` (edges)	Linear `Q4_K` (middle) / `Q5_K` (edges) + `imatrix`	Foundational Knowledge Stability: Keeps the universal pathway in high-fidelity linear blocks, eliminating quantization drift while maintaining rapid single-cycle dequantization.
Core MoE Layers (Middle: 10–29)	Flat `IQ3_S` / `Q3_K_S` (uniform bit-rate across all layers)	Aggressive `IQ2_S` (2.50 BPW)	`IQ3_XXS` (3.06 BPW) + calibrated `imatrix`	Above the Quality Threshold: Generic 2-bit `IQ2_S` baselines drop below the critical quality floor for 35B MoEs, resulting in perplexity spikes on reasoning tasks. Our `IQ3_XXS` with imatrix achieves deep compression (272 MiB → 98 MiB per block) without sacrificing logic.
Edge MoE Layers (Layers 0–9 & 30–39)	Flat `IQ3_S` / `Q3_K_S` (no layer-wise gradient)	`Q3_K` (limited to first/last 5 layers only: L0–4, L35–39)	`Q3_K` (expanded to 10 input & 10 output layers)	AVX2 Single-Cycle Speed: Expanded 10+10 layer protection using linear `Q3_K` blocks enables single-cycle vectorized AVX2 CPU dequantization, supporting efficient streaming on budget DDR4/DDR5 laptops.
MTP Draft Block (`blk.40`)	Stripped with `--no-mtp` or broken	Crushed to `IQ2_S` / tier precision	Preserved in `IQ3_S` / `Q3_K` & `IQ4_NL`	Speculative Decoding Speedup: Maintains 58%–65% candidate acceptance rate, yielding 1.6x–1.75x real-world token speedup without speculative rejection waste.
Normalization & Biases	Often degraded	Standard	`F32` uncompressed	Numerical Stability: Prevents cumulative floating-point underflow/overflow across deep 40-layer computation.

Native Multi-Token Prediction (MTP) Co-Pilot

Most automated community releases strip or break the native Multi-Token Prediction head using --no-mtp. APEX-I-MiniPlus fully preserves and calibrates the native prediction block (blk.40):

Zero-Cost Speculative Acceleration: Unlike external draft models that consume separate VRAM and memory bandwidth, Iris-mini's native MTP head is integrated directly into the weights.
Empirical Acceptance Rate: 58.8% to 65.5% of predicted candidate tokens are accepted on full GPU offload.
Token Yield: Delivers 1.60 to 1.75 tokens per forward step on standard text, peaking at 2.0+ tokens/step during continuous code and prose generation.
Net Speedup: Provides approx. 1.6x faster real-world generation without quality degradation.

Everyday Laptop Guidance

VRAM Allocation: 3.8 GB VRAM utilized on budget 4GB/6GB GPUs.
System Memory: 32GB DDR4 holds the remaining layers.
Prefill Speed: 300 to 410 tokens/second sustained across dense inputs.
Generation Speed: 23 to 26+ tokens/second sustained on standard DDR4 RAM!

The 24GB Miracle: Full 256K Context Runs In VRAM!

Context Length	Model Weights (Est.)	KV Cache (q8_0, 4 slots)	Compute Buffers	Total GPU VRAM (Est.)	Hardware Feasibility
32,512 (32k)	`13.82 GiB`	`0.58 GiB`	`1.80 GiB`	`16.20 GiB`	Full offload on 24GB; partial on 16GB
64,512 (64k)	`13.82 GiB`	`0.92 GiB`	`1.95 GiB`	`16.69 GiB`	Effortless fit on 24GB GPUs
128,640 (128k)	`13.82 GiB`	`1.58 GiB`	`2.22 GiB`	`17.62 GiB`	Effortless fit on 24GB GPUs
262,144 (Full 256K)	`13.82 GiB`	`2.92 GiB`	`2.80 GiB`	`19.54 GiB`	FULL 256K NATIVE IN VRAM!

Hardware Throughput Projections (RTX 30 / 40 / 50)

⚙️ Recommended Generation Parameters (AllSpark-Research Official)

Official generation guidelines specified by AllSpark-Research for deep research and search agent loops:

Workload / Agent Mode	Temperature	Top-P	Context Strategy	Output Format
Deep Search & Tool Invocation	`0.20 – 0.60`	`0.95`	`discard-all` at 131K	OpenAI function calling interface
Exploratory Query Generation	`0.70`	`0.95`	Summary retry	Final answers wrapped in `\boxed{}`

🔍 Model Inherent Behavior vs. Quantization Fidelity Notice

Any behavioral nuances, stylistic tendencies, domain-specific habits, or zero-shot edge-case oversights stem entirely from the original unquantized checkpoint weights and fine-tuning distribution, NOT from the APEX-I quantization process. Handcrafted APEX-I-MiniPlus strictly preserves mathematical tensor fidelity—keeping 100% of expert routing matrices (gate_inp) in uncompressed F32 (zero router drift), armoring the token output head in Q6_K, and safeguarding attention gates in Q8_0. Empirical verification confirms near-zero perplexity loss (ΔPPL ≈ +0.06), ensuring that token logits, routing decisions, and reasoning trajectories are mathematically faithful to the original base model.