Instructions to use mlx-community/Mistral-7B-Instruct-v0.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/Mistral-7B-Instruct-v0.2 with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # if on a CUDA device, also pip install mlx[cuda] # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("mlx-community/Mistral-7B-Instruct-v0.2") prompt = "Once upon a time in" text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use mlx-community/Mistral-7B-Instruct-v0.2 with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Generate some text mlx_lm.generate --model "mlx-community/Mistral-7B-Instruct-v0.2" --prompt "Once upon a time"
Script that converted this model?
#5
by nheagy - opened
I'd love to see the script used to convert this model for MLX. Unlike the other Mistral models, this one isn't available as pth, and (afaik) only available from HF in transformer-compatible formats. There are quite a few models that are similarly only available on HF, and sharing how this one was converted would help provide an example of how to do such conversions for MLX.