Text Generation
Transformers
Safetensors
llama
llama-factory
full
Generated from Trainer
conversational
text-generation-inference
Instructions to use hlillemark/llama3_8b_sft_mc with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hlillemark/llama3_8b_sft_mc with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="hlillemark/llama3_8b_sft_mc") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("hlillemark/llama3_8b_sft_mc") model = AutoModelForCausalLM.from_pretrained("hlillemark/llama3_8b_sft_mc", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use hlillemark/llama3_8b_sft_mc with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "hlillemark/llama3_8b_sft_mc" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hlillemark/llama3_8b_sft_mc", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/hlillemark/llama3_8b_sft_mc
- SGLang
How to use hlillemark/llama3_8b_sft_mc with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "hlillemark/llama3_8b_sft_mc" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hlillemark/llama3_8b_sft_mc", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "hlillemark/llama3_8b_sft_mc" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hlillemark/llama3_8b_sft_mc", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use hlillemark/llama3_8b_sft_mc with Docker Model Runner:
docker model run hf.co/hlillemark/llama3_8b_sft_mc
Training in progress, step 670
Browse files
model-00001-of-00004.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4976698672
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:90bc2cb43ef69f683577064c90eab93a6187d058d056d6216eb49e0d28fbcdde
|
| 3 |
size 4976698672
|
model-00002-of-00004.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4999802720
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0d782789c466ff853e5d061730ee0850433aaa60d4568054ea0cdbdfa5ff88e5
|
| 3 |
size 4999802720
|
model-00003-of-00004.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4915916176
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0c8fb2b1f224c7bf76645e9e27afcecfd02845fed45c3f3138e07be0ae4a11c6
|
| 3 |
size 4915916176
|
model-00004-of-00004.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 1168138808
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:710e856f7645c4cbc62939c8216d3d9867cf902080a41977bca2e66268c02484
|
| 3 |
size 1168138808
|
trainer_log.jsonl
CHANGED
|
@@ -65,3 +65,17 @@
|
|
| 65 |
{"current_steps": 550, "total_steps": 670, "loss": 0.0091, "lr": 9.457478323545749e-07, "epoch": 8.208955223880597, "percentage": 82.09, "elapsed_time": "0:32:22", "remaining_time": "0:07:03"}
|
| 66 |
{"current_steps": 550, "total_steps": 670, "eval_loss": 2.2086477279663086, "epoch": 8.208955223880597, "percentage": 82.09, "elapsed_time": "0:32:23", "remaining_time": "0:07:03"}
|
| 67 |
{"current_steps": 560, "total_steps": 670, "loss": 0.0065, "lr": 7.988611635181099e-07, "epoch": 8.35820895522388, "percentage": 83.58, "elapsed_time": "0:32:50", "remaining_time": "0:06:27"}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
{"current_steps": 550, "total_steps": 670, "loss": 0.0091, "lr": 9.457478323545749e-07, "epoch": 8.208955223880597, "percentage": 82.09, "elapsed_time": "0:32:22", "remaining_time": "0:07:03"}
|
| 66 |
{"current_steps": 550, "total_steps": 670, "eval_loss": 2.2086477279663086, "epoch": 8.208955223880597, "percentage": 82.09, "elapsed_time": "0:32:23", "remaining_time": "0:07:03"}
|
| 67 |
{"current_steps": 560, "total_steps": 670, "loss": 0.0065, "lr": 7.988611635181099e-07, "epoch": 8.35820895522388, "percentage": 83.58, "elapsed_time": "0:32:50", "remaining_time": "0:06:27"}
|
| 68 |
+
{"current_steps": 570, "total_steps": 670, "loss": 0.0104, "lr": 6.633752576786251e-07, "epoch": 8.507462686567164, "percentage": 85.07, "elapsed_time": "0:33:18", "remaining_time": "0:05:50"}
|
| 69 |
+
{"current_steps": 580, "total_steps": 670, "loss": 0.006, "lr": 5.396577872130676e-07, "epoch": 8.656716417910447, "percentage": 86.57, "elapsed_time": "0:33:45", "remaining_time": "0:05:14"}
|
| 70 |
+
{"current_steps": 590, "total_steps": 670, "loss": 0.006, "lr": 4.2804448812404754e-07, "epoch": 8.805970149253731, "percentage": 88.06, "elapsed_time": "0:34:13", "remaining_time": "0:04:38"}
|
| 71 |
+
{"current_steps": 600, "total_steps": 670, "loss": 0.0054, "lr": 3.288382489424502e-07, "epoch": 8.955223880597014, "percentage": 89.55, "elapsed_time": "0:34:41", "remaining_time": "0:04:02"}
|
| 72 |
+
{"current_steps": 600, "total_steps": 670, "eval_loss": 2.2865116596221924, "epoch": 8.955223880597014, "percentage": 89.55, "elapsed_time": "0:34:42", "remaining_time": "0:04:02"}
|
| 73 |
+
{"current_steps": 610, "total_steps": 670, "loss": 0.006, "lr": 2.4230828876927293e-07, "epoch": 9.104477611940299, "percentage": 91.04, "elapsed_time": "0:35:09", "remaining_time": "0:03:27"}
|
| 74 |
+
{"current_steps": 620, "total_steps": 670, "loss": 0.0033, "lr": 1.6868942668726408e-07, "epoch": 9.253731343283581, "percentage": 92.54, "elapsed_time": "0:35:37", "remaining_time": "0:02:52"}
|
| 75 |
+
{"current_steps": 630, "total_steps": 670, "loss": 0.0064, "lr": 1.0818144452496293e-07, "epoch": 9.402985074626866, "percentage": 94.03, "elapsed_time": "0:36:04", "remaining_time": "0:02:17"}
|
| 76 |
+
{"current_steps": 640, "total_steps": 670, "loss": 0.0023, "lr": 6.094854470245326e-08, "epoch": 9.552238805970148, "percentage": 95.52, "elapsed_time": "0:36:31", "remaining_time": "0:01:42"}
|
| 77 |
+
{"current_steps": 650, "total_steps": 670, "loss": 0.0038, "lr": 2.711890463007405e-08, "epoch": 9.701492537313433, "percentage": 97.01, "elapsed_time": "0:36:58", "remaining_time": "0:01:08"}
|
| 78 |
+
{"current_steps": 650, "total_steps": 670, "eval_loss": 2.301581621170044, "epoch": 9.701492537313433, "percentage": 97.01, "elapsed_time": "0:36:59", "remaining_time": "0:01:08"}
|
| 79 |
+
{"current_steps": 660, "total_steps": 670, "loss": 0.0037, "lr": 6.784328869339218e-09, "epoch": 9.850746268656717, "percentage": 98.51, "elapsed_time": "0:37:27", "remaining_time": "0:00:34"}
|
| 80 |
+
{"current_steps": 670, "total_steps": 670, "loss": 0.0034, "lr": 0.0, "epoch": 10.0, "percentage": 100.0, "elapsed_time": "0:37:55", "remaining_time": "0:00:00"}
|
| 81 |
+
{"current_steps": 670, "total_steps": 670, "epoch": 10.0, "percentage": 100.0, "elapsed_time": "0:44:38", "remaining_time": "0:00:00"}
|