Instructions to use DevQuasar/llama3_8b_chat_brainstorm-v2.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DevQuasar/llama3_8b_chat_brainstorm-v2.1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DevQuasar/llama3_8b_chat_brainstorm-v2.1")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DevQuasar/llama3_8b_chat_brainstorm-v2.1")
model = AutoModelForCausalLM.from_pretrained("DevQuasar/llama3_8b_chat_brainstorm-v2.1")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DevQuasar/llama3_8b_chat_brainstorm-v2.1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DevQuasar/llama3_8b_chat_brainstorm-v2.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DevQuasar/llama3_8b_chat_brainstorm-v2.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/DevQuasar/llama3_8b_chat_brainstorm-v2.1

SGLang

How to use DevQuasar/llama3_8b_chat_brainstorm-v2.1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DevQuasar/llama3_8b_chat_brainstorm-v2.1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DevQuasar/llama3_8b_chat_brainstorm-v2.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DevQuasar/llama3_8b_chat_brainstorm-v2.1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DevQuasar/llama3_8b_chat_brainstorm-v2.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use DevQuasar/llama3_8b_chat_brainstorm-v2.1 with Docker Model Runner:
```
docker model run hf.co/DevQuasar/llama3_8b_chat_brainstorm-v2.1
```

'Make knowledge free for everyone'

Brainstorm-v2.1

Model intention

Brainstorm facilitates idea exploration through interaction with a Language Model (LLM). Rather than providing direct answers, the model engages in a dialogue with users, offering probing questions aimed at fostering deeper contemplation and consideration of various facets of their ideas. Model can summarize the discussed ideas in Markdown format.

Examples

Inference code

https://github.com/csabakecskemeti/ai_utils/blob/main/brainstorm_inference.py

Example generations

[USER] (Ctrl+C to exit): I'd like to organize my vintage computer collection

[CHAT]: That sounds like a great idea! What kind of organization system are you thinking of using?

[USER] (Ctrl+C to exit): Make it sure that they are available for use any time, but I don't want to see all the cables hanging around

[CHAT]: So you want to make it easy to access and tidy up the cables. Have you considered using a cable management system or tying up loose ends with ribbon cutters?

[USER] (Ctrl+C to exit): That makes sense

[CHAT]: What about labeling each device so you can find what you need quickly?

[USER] (Ctrl+C to exit): Can you please summaryze what we've been talked so far?

[CHAT]: Here's a summary:

Organizing Vintage Computer Collection

Make sure devices are easily accessible for use
Tidy up cables using a cable management system or ribbon cutters
Label each device to find it quickly

Let me know if you'd like to add or modify anything!

[USER] (Ctrl+C to exit): Ok let's continue: I also want to use a single monoytor an pheripherias for all computer

[CHAT]: That could get expensive fast. Are you thinking of using a hub or a switch? Or maybe something more sophisticated like a KVM switch?

[USER] (Ctrl+C to exit): yest that's make sense, can you please add it to the summary?

[CHAT]: Here's the updated summary:

Organizing Vintage Computer Collection

Make sure devices are easily accessible for use
Tidy up cables using a cable management system or ribbon cutters
Label each device to find it quickly
Use a single monitor and peripherals for all computers (hub, switch, KVM switch)

Let me know if there's anything else you'd like to add or change!

[USER] (Ctrl+C to exit): I also want to protect them from dust, not sure how to do that

[CHAT]: For protecting against dust, one option could be using enclosures or cabinets. Another approach might be setting up a dedicated workspace with appropriate ventilation. Which of these ideas resonates with you?