Instructions to use unsloth/North-Mini-Code-1.0-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use unsloth/North-Mini-Code-1.0-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("unsloth/North-Mini-Code-1.0-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use unsloth/North-Mini-Code-1.0-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M
# Run inference directly in the terminal:
llama cli -hf unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M
# Run inference directly in the terminal:
llama cli -hf unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M

Use Docker

docker model run hf.co/unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M

LM Studio
Jan
Ollama
How to use unsloth/North-Mini-Code-1.0-GGUF with Ollama:
```
ollama run hf.co/unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M
```
Unsloth Desktop

How to use unsloth/North-Mini-Code-1.0-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use unsloth/North-Mini-Code-1.0-GGUF with Docker Model Runner:
```
docker model run hf.co/unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M
```

Lemonade

How to use unsloth/North-Mini-Code-1.0-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M

Run and chat with the model

lemonade run user.North-Mini-Code-1.0-GGUF-UD-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use unsloth/North-Mini-Code-1.0-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use unsloth/North-Mini-Code-1.0-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

danielhanchen commited on Jun 10

Commit

e306bb4

verified ·

1 Parent(s): e1e23cf

Fix tokenizer type

Browse files

Files changed (22) hide show

BF16/North-Mini-Code-1.0-BF16-00001-of-00002.gguf +1 -1
North-Mini-Code-1.0-MXFP4_MOE.gguf +1 -1
North-Mini-Code-1.0-Q8_0.gguf +1 -1
North-Mini-Code-1.0-UD-IQ1_M.gguf +1 -1
North-Mini-Code-1.0-UD-IQ2_M.gguf +1 -1
North-Mini-Code-1.0-UD-IQ2_XXS.gguf +1 -1
North-Mini-Code-1.0-UD-IQ3_S.gguf +1 -1
North-Mini-Code-1.0-UD-IQ3_XXS.gguf +1 -1
North-Mini-Code-1.0-UD-IQ4_NL.gguf +1 -1
North-Mini-Code-1.0-UD-IQ4_XS.gguf +1 -1
North-Mini-Code-1.0-UD-Q2_K_XL.gguf +1 -1
North-Mini-Code-1.0-UD-Q3_K_M.gguf +1 -1
North-Mini-Code-1.0-UD-Q3_K_XL.gguf +1 -1
North-Mini-Code-1.0-UD-Q4_K_M.gguf +1 -1
North-Mini-Code-1.0-UD-Q4_K_S.gguf +1 -1
North-Mini-Code-1.0-UD-Q4_K_XL.gguf +1 -1
North-Mini-Code-1.0-UD-Q5_K_M.gguf +1 -1
North-Mini-Code-1.0-UD-Q5_K_S.gguf +1 -1
North-Mini-Code-1.0-UD-Q5_K_XL.gguf +1 -1
North-Mini-Code-1.0-UD-Q6_K.gguf +1 -1
North-Mini-Code-1.0-UD-Q6_K_XL.gguf +1 -1
North-Mini-Code-1.0-UD-Q8_K_XL.gguf +1 -1

BF16/North-Mini-Code-1.0-BF16-00001-of-00002.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:28a587c45038cc2c99b70881b8f472e4ee96f28e0dbf13bfed9395932c1896f6
 size 49783515904

 version https://git-lfs.github.com/spec/v1
+oid sha256:f417602e8bda9fb783f9ebea5860332e0b36e1e1b996ccef9607d788b88086c9
 size 49783515904

North-Mini-Code-1.0-MXFP4_MOE.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:284445909efedf64289d6a2a7f34a602fb9ca427fd0f233992723fe3aeecc6cb
 size 18662121568

 version https://git-lfs.github.com/spec/v1
+oid sha256:d4cd6070cbab02f9f3066d3b3e2d955ecad2b8ffcd5332cbc87c4c548b3feadd
 size 18662121568

North-Mini-Code-1.0-Q8_0.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4f0b3b8e9bb874d0b51019c9fa4fce1e0eaaa8b5ff30656940ff7b9d13c1aa1e
 size 32437264480

 version https://git-lfs.github.com/spec/v1
+oid sha256:652eeb9932e8020ef7b35374189411848187fa58796ded0ceee258bd9d36426b
 size 32437264480

North-Mini-Code-1.0-UD-IQ1_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cd65e4fafb2116b1a8c7365af9ffb920328419a665d4d0c65e5856d352bc9b99
 size 9379799136

 version https://git-lfs.github.com/spec/v1
+oid sha256:3f6d7808b940ad798a837607daea5364dbea32933ad484303821308992b293db
 size 9379799136

North-Mini-Code-1.0-UD-IQ2_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6fe750ecf59efdf71aa7d5e8c06e62889b610298eae5f35f2323ac2fdd570e09
 size 9863438432

 version https://git-lfs.github.com/spec/v1
+oid sha256:24675158d1920ac3a41bc84f4cc13f280571185825733a49bbec5233ad05eecf
 size 9863438432

North-Mini-Code-1.0-UD-IQ2_XXS.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d42139ebbee6c674f24796384c12b13ef2335a1e2e93109c75956c88d09dc6ff
 size 9782452320

 version https://git-lfs.github.com/spec/v1
+oid sha256:cf840071270657f32839398cd03745a6fd901671d2d4574cb799da104cfbdcc8
 size 9782452320

North-Mini-Code-1.0-UD-IQ3_S.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:45af75445c6976c4db703e97c3c40bb39a781878ec5925987fadf4ea94c3880a
 size 12765339744

 version https://git-lfs.github.com/spec/v1
+oid sha256:4d0f6bde22b5c784d01f3f160d4a99d763fd44f56ba5deddb1108d58302672c3
 size 12765339744

North-Mini-Code-1.0-UD-IQ3_XXS.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:215204ca4bff73ce1b4a5af1235d761ceff15f16284f79b54aa73fc544189d04
 size 11708375136

 version https://git-lfs.github.com/spec/v1
+oid sha256:029b51f93570c4295680bf6fb138bbc51bba7c377d613f75ac4eaa6913597140
 size 11708375136

North-Mini-Code-1.0-UD-IQ4_NL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:36c27d3a4d74293e70e911997cacad2c221f5d87303378f38b7e550d45157444
 size 15532122208

 version https://git-lfs.github.com/spec/v1
+oid sha256:dcf37cbe54875cf18ad0b472c916127614fa9338e2f4412c672632683def46bd
 size 15532122208

North-Mini-Code-1.0-UD-IQ4_XS.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9a0f2259bb2b7d159d2e3524c58f51d194d2b78915229c145c5494576dbe9a5e
 size 15230132320

 version https://git-lfs.github.com/spec/v1
+oid sha256:34565d2cbce8202e7baacce74845e34852038f95c0f612398c3b794524a945c0
 size 15230132320

North-Mini-Code-1.0-UD-Q2_K_XL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1fc72a94b2293c6a7e5dcf400a62418d5122931e5109c84bddab12991fa3df9e
 size 10480001120

 version https://git-lfs.github.com/spec/v1
+oid sha256:3d4687064b2309d79094481ca638b8556d5fa1b720b54d9a3b09cea2f8130ef5
 size 10480001120

North-Mini-Code-1.0-UD-Q3_K_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:247bb9f144cae06459293d279186adfc71081f67e54ab15f8ab39e11019c7ec8
 size 14213013600

 version https://git-lfs.github.com/spec/v1
+oid sha256:145e185430e31c43221995ac52aed7efbd91f043dbfcdcf297a986f06f7dcee7
 size 14213013600

North-Mini-Code-1.0-UD-Q3_K_XL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c871275d526c3f24341dce9f5d8d5c2bccb5f1577e9fcf31686eb7abb54e76d2
 size 14343037024

 version https://git-lfs.github.com/spec/v1
+oid sha256:6e3a3d769c9b0e369c09fd9fcecd3aace57eafbbfd787f817d47712718801aba
 size 14343037024

North-Mini-Code-1.0-UD-Q4_K_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:78cd1d31c6b7b50cba278ff3d9fdcba9583a95c8e3ffd87d1e76ca4a018f1165
 size 19203186784

 version https://git-lfs.github.com/spec/v1
+oid sha256:d59f4cb3abec1bd3e5e185b99cdd811e3be3402cbc2bab8223081a460d5724bf
 size 19203186784

North-Mini-Code-1.0-UD-Q4_K_S.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:422e59059c0ae68a6124c95136ccd5ce6789fa78e491f77692a525f09861aaca
 size 18045558880

 version https://git-lfs.github.com/spec/v1
+oid sha256:b62eee03ca42a69401bc5b17e22ce78cf34469628a3cf7b4cf6ac2406078a28a
 size 18045558880

North-Mini-Code-1.0-UD-Q4_K_XL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2de1a9fecbe90cd9bf1f614a0098ad73a2f5a7f17b142dacf9d412b0b0f71903
 size 19253518432

 version https://git-lfs.github.com/spec/v1
+oid sha256:86ba9cf324a1cf53687872abdda3861e3304bfd25d1f606f92a6641017ecbc6e
 size 19253518432

North-Mini-Code-1.0-UD-Q5_K_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dab35597817281ac1a7d5b9c746d48a247be50f74b8dd2d2ff3db9e5a5db663a
 size 22946603104

 version https://git-lfs.github.com/spec/v1
+oid sha256:77d04849d0cd1c725db22c560a2c0a39c700f2f6939d125f2419676c9dd3956e
 size 22946603104

North-Mini-Code-1.0-UD-Q5_K_S.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2cf6fa9a260d3d2988bc7c077cbcec3cc6788c8b31bbf2c2550f656edac682d2
 size 21619105888

 version https://git-lfs.github.com/spec/v1
+oid sha256:c6bfe3b270a9a50bd24d7cfa248921bd6a5f4a08d35b8aff92b07df24d22e1a0
 size 21619105888

North-Mini-Code-1.0-UD-Q5_K_XL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:930671299b10a2f082d5cec3822d4d953d728cd4f77f3b235248241390b46ef9
 size 23000080480

 version https://git-lfs.github.com/spec/v1
+oid sha256:2a0b6a47228a29c8a61e58d2c0e0c89fc09f0d3f46ecc8890f4e48e8ad4f6b77
 size 23000080480

North-Mini-Code-1.0-UD-Q6_K.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3f26f58d2b50d4438a821aafa8d61cda643a9b563ee255976ab7abf1f7e8eea1
 size 25513517152

 version https://git-lfs.github.com/spec/v1
+oid sha256:dd2d9ff23cf4b2930cba116de47d91a711f3b21dffafbcf3ae52013396a37b1a
 size 25513517152

North-Mini-Code-1.0-UD-Q6_K_XL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5525e4b1434f96072078ecdc9737a1cb2df2bc8be17882d0753c75267665069d
 size 27853938784

 version https://git-lfs.github.com/spec/v1
+oid sha256:c745f23f0c802bc07a9f3f067cf2ac80f58b415bfc78d7620624bd16eef8d44c
 size 27853938784

North-Mini-Code-1.0-UD-Q8_K_XL.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fe7402bc8bf249262a6b42a87e159ed01912379e834bb2811c57be5fb8f055ec
 size 33209933920

 version https://git-lfs.github.com/spec/v1
+oid sha256:c047252b9f4b969034dd28cf4928cef94479a56d5753166ae836a5685b588ad5
 size 33209933920