ParamMute
Collection
The official Collections of ParamMute • 5 items • Updated • 2
How to use chengpingan/ParamMute-8B-SFT with Transformers:
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("chengpingan/ParamMute-8B-SFT", device_map="auto")This is the supervised fine-tuning (SFT) checkpoint released with ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation. It is based on Llama 3 8B Instruct and uses the ParamMute activation-suppression setting for layers 19–26. ParamMute studies how to make a retrieval-augmented model follow accurate retrieved evidence when it conflicts with the model’s memorized answer.
To evaluate contextual faithfulness, use the normalization and recall metrics defined in Section 5.
@inproceedings{huang2025parammute,
title = {{ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation}},
author = {Pengcheng Huang and Zhenghao Liu and Yukun Yan and Haiyan Zhao and Xiaoyuan Yi and Hao Chen and Zhiyuan Liu and Maosong Sun and Tong Xiao and Ge Yu and Chenyan Xiong},
year = {2025},
doi = {10.52202/085713-3356},
url = {https://proceedings.neurips.cc/paper_files/paper/2025/hash/90d1e515d74686ad148b469a3cecbbaa-Abstract-Conference.html},
booktitle = {Advances in Neural Information Processing Systems 38},
publisher = {Curran Associates, Inc.}
}
Base model
meta-llama/Meta-Llama-3-8B