Instructions to use xxxllz/ChemVLR-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use xxxllz/ChemVLR-7B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="xxxllz/ChemVLR-7B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForImageTextToText processor = AutoProcessor.from_pretrained("xxxllz/ChemVLR-7B") model = AutoModelForImageTextToText.from_pretrained("xxxllz/ChemVLR-7B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use xxxllz/ChemVLR-7B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "xxxllz/ChemVLR-7B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "xxxllz/ChemVLR-7B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/xxxllz/ChemVLR-7B
- SGLang
How to use xxxllz/ChemVLR-7B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "xxxllz/ChemVLR-7B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "xxxllz/ChemVLR-7B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "xxxllz/ChemVLR-7B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "xxxllz/ChemVLR-7B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use xxxllz/ChemVLR-7B with Docker Model Runner:
docker model run hf.co/xxxllz/ChemVLR-7B
ChemVLR-7B
ChemVLR is a chemical Vision-Language Model (VLM) designed to prioritize reasoning within the perception process. Unlike conventional chemical VLMs that often function as "black-box" systems, ChemVLR analyzes visual inputs in a fine-grained manner by explicitly identifying granular chemical descriptors, such as functional groups, prior to generating answers. This approach ensures the production of explicit and interpretable reasoning paths for complex visual chemical problems.
Model Description
ChemVLR-7B is built upon the Qwen2.5-VL-7B backbone and trained using a three-stage framework to systemically build perception and reasoning capacity. It utilizes a curated dataset of 760k high-quality samples across molecular and reaction tasks.
- Paper: ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understanding
- Repository: https://github.com/xxlllz/ChemVLR
Model Highlights
- Reasoning-Prioritized Perception: Explicitly identifies chemical components before answering to provide interpretable outputs.
- Large-Scale Dataset: Trained on 760k high-quality reasoning-and-captioning samples.
- State-of-the-Art Performance: Surpasses leading proprietary models and domain-specific open-source baselines in chemical understanding benchmarks.
Citation
@misc{zhao2026chemvlrprioritizingreasoningperception,
title={ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understanding},
author={Xuanle Zhao and Xinyuan Cai and Xiang Cheng and Xiuyi Chen and Bo Xu},
year={2026},
eprint={2604.06685},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2604.06685},
}
Acknowledgement
ChemVLR is built upon Qwen2.5-VL and Qwen3-VL. We thank these teams for open-sourcing their work!
- Downloads last month
- 6