Instructions to use LucasInsight/Meta-Llama-3-8B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use LucasInsight/Meta-Llama-3-8B-Instruct with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("LucasInsight/Meta-Llama-3-8B-Instruct", dtype="auto") - llama-cpp-python
How to use LucasInsight/Meta-Llama-3-8B-Instruct with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="LucasInsight/Meta-Llama-3-8B-Instruct", filename="unsloth.F16.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use LucasInsight/Meta-Llama-3-8B-Instruct with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf LucasInsight/Meta-Llama-3-8B-Instruct:Q4_K_M # Run inference directly in the terminal: llama-cli -hf LucasInsight/Meta-Llama-3-8B-Instruct:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf LucasInsight/Meta-Llama-3-8B-Instruct:Q4_K_M # Run inference directly in the terminal: llama-cli -hf LucasInsight/Meta-Llama-3-8B-Instruct:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf LucasInsight/Meta-Llama-3-8B-Instruct:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf LucasInsight/Meta-Llama-3-8B-Instruct:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf LucasInsight/Meta-Llama-3-8B-Instruct:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf LucasInsight/Meta-Llama-3-8B-Instruct:Q4_K_M
Use Docker
docker model run hf.co/LucasInsight/Meta-Llama-3-8B-Instruct:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use LucasInsight/Meta-Llama-3-8B-Instruct with Ollama:
ollama run hf.co/LucasInsight/Meta-Llama-3-8B-Instruct:Q4_K_M
- Unsloth Studio new
How to use LucasInsight/Meta-Llama-3-8B-Instruct with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for LucasInsight/Meta-Llama-3-8B-Instruct to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for LucasInsight/Meta-Llama-3-8B-Instruct to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for LucasInsight/Meta-Llama-3-8B-Instruct to start chatting
- Docker Model Runner
How to use LucasInsight/Meta-Llama-3-8B-Instruct with Docker Model Runner:
docker model run hf.co/LucasInsight/Meta-Llama-3-8B-Instruct:Q4_K_M
- Lemonade
How to use LucasInsight/Meta-Llama-3-8B-Instruct with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull LucasInsight/Meta-Llama-3-8B-Instruct:Q4_K_M
Run and chat with the model
lemonade run user.Meta-Llama-3-8B-Instruct-Q4_K_M
List all available models
lemonade list
LucasInsight/Meta-Llama-3-8B-Instruct Model Card
Model Overview
The LucasInsight/Meta-Llama-3-8B-Instruct model is an enhanced version of the Meta-Llama3 project, incorporating the alpaca-gpt4-data-zh Chinese dataset. The model has been fine-tuned using Unsloth with 4-bit QLoRA and generates GGUF model files compatible with the Ollama inference engine.
👋Join our WeChat
模型概述
LucasInsight/Meta-Llama-3-8B-Instruct 模型是在 Meta-Llama3 工程的基础上,增加了 alpaca-gpt4-data-zh 中文数据集。该模型通过使用 Unsloth 的 4-bit QLoRA 进行微调,生成的 GGUF 模型文件支持 Ollama 推理引擎。
👋加入我们的微信群
License Information
This project is governed by the licenses of the integrated components:
Meta-Llama3 Project
- Project URL: https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct
- License: Llama 3 Community License Agreement
Citation:
@article{llama3modelcard, title={Llama 3 Model Card}, author={AI@Meta}, year={2024}, url = {https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md} }Unsloth Project
- License: Apache-2.0 License
- Project URL: https://huggingface.co/unsloth/llama-3-8b-Instruct-bnb-4bit
Chinese Dataset Integration
- Dataset: alpaca-gpt4-data-zh
- License: CC BY NC 4.0 (for non-commercial research use only)
- Dataset URL: https://huggingface.co/datasets/llm-wizard/alpaca-gpt4-data-zh
Usage and License Notices:
The data is intended and licensed for research use only. The dataset is CC BY NC 4.0, allowing only non-commercial use. Models trained using this dataset should not be used outside of research purposes.Citation:
@article{peng2023gpt4llm, title={Instruction Tuning with GPT-4}, author={Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, Jianfeng Gao}, journal={arXiv preprint arXiv:2304.03277}, year={2023} }
许可证信息
本项目的许可证由各集成工程的许可证构成:
Meta-Llama3 项目
- 项目地址:https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct
- 许可证:Llama 3 Community License Agreement
引用说明:
@article{llama3modelcard, title={Llama 3 Model Card}, author={AI@Meta}, year={2024}, url = {https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md} }Unsloth 项目
- 许可证:Apache-2.0 许可证
- 项目地址:https://huggingface.co/unsloth/llama-3-8b-Instruct-bnb-4bit
中文数据集集成
- 数据集:alpaca-gpt4-data-zh
- 许可证:CC BY NC 4.0(仅用于非商业的研究用途)
- 数据集地址:https://huggingface.co/datasets/llm-wizard/alpaca-gpt4-data-zh
使用和许可证通知:
该数据仅限于研究使用,且基于 CC BY NC 4.0 许可证,只允许非商业用途。使用此数据集训练的模型不得用于研究用途以外的场合。引用说明:
@article{peng2023gpt4llm, title={Instruction Tuning with GPT-4}, author={Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, Jianfeng Gao}, journal={arXiv preprint arXiv:2304.03277}, year={2023} }
- Downloads last month
- 103
4-bit
8-bit
16-bit
