DSLM-LST-9B

DSLM-LST-9B is a Qwen3.5-9B derivative refined with our in-house Language Selection Tuning (LST) technique. The goal is to suppress unwanted Chinese-character generation when the model is used to serve non-Chinese (English / Korean / Japanese etc.) users. The adjustment is intentionally minimal in scope; most of the network — including vision and multimodal components — is preserved bit-identical to the base model. Vision and multimodal capabilities are preserved unchanged.

Why LST?

Multilingual LLMs trained on heavily skewed corpora (e.g., Qwen on Chinese-rich data) tend to leak the dominant training language regardless of prompt language. This phenomenon is known as language confusion. For Korean users, this means Chinese characters sometimes appear in the middle of an otherwise Korean answer. This hurts both readability and user trust.

Language Selection Tuning (LST) addresses this problem in a learning-based manner. Unlike post-hoc decoding tricks (vocabulary masking, banned-token lists, etc.), LST adjusts the model's internal language-selection behavior, so the effect tends to persist through downstream full-parameter SFT / RLHF stages rather than being washed out by further fine-tuning. (The exact algorithm and training configuration are proprietary and not disclosed in this release.)

Key Properties

• Most of the network is preserved bit-identical to the base model — including the tokenizer, chat template, and vision tower — so existing integrations remain compatible.
• Reasoning performance is preserved: KMMLU / HumanEval / GSM8K scores remain on par with — and in some configurations slightly above — the base model.
• Selectivity is preserved: when the user explicitly asks for Chinese, the model still produces fluent Chinese. This is not blanket suppression.
• Persistence through SFT: after a downstream full-parameter SFT stage, the Chinese-leak suppression effect remains almost unchanged (SRR ≈ 1.0).

Quickstart

The recommended serving path is vLLM, which is also what we used in our evaluation pipeline.

vllm serve dataslab/DSLM-LST-9B \
    --port 8000 \
    --dtype bfloat16 \
    --gpu-memory-utilization 0.90 \
    --reasoning-parser qwen3        # exposes <think> trace via OpenAI API
    # --enable-reasoning            # auto-on with --reasoning-parser (vLLM >= 0.7)
    # --max-model-len 16384         # cap context to shrink KV cache (default: 262,144)

Use with transformers

Non-Thinking mode (recommended for fast chat)

import torch
from transformers import AutoTokenizer, AutoModelForImageTextToText

REPO = "dataslab/DSLM-LST-9B"

tokenizer = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForImageTextToText.from_pretrained(
    REPO,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "한반도 주변에 가장 흔한 점토광물은?"},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

out = model.generate(**inputs, max_new_tokens=256)
text = tokenizer.decode(out[0][inputs.input_ids.shape[-1]:],
                       skip_special_tokens=True)
print(text)

Thinking mode (recommended for complex reasoning)

Either use thinking_budget (e.g., vLLM's --reasoning-parser qwen3) or give max_new_tokens enough headroom (e.g., 8,192 + 256 = 8,448). Caveat: without a thinking_budget cap, a too-small max_new_tokens can be fully consumed inside <think> and the answer never gets emitted.

# ... tokenizer / model loaded as above ...

THINKING_BUDGET = 8192   # max tokens inside <think>
ANSWER_TOKENS   = 256    # tokens after </think>

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

out = model.generate(**inputs, max_new_tokens=THINKING_BUDGET + ANSWER_TOKENS)
text = tokenizer.decode(out[0][inputs.input_ids.shape[-1]:],
                       skip_special_tokens=True)
print(text)

Why AutoModelForImageTextToText? Qwen3.5-9B's declared architecture is Qwen3_5ForConditionalGeneration, a composite class that wraps both the text decoder and the vision tower. Loading via AutoModelForCausalLM works for text-only inference but strips the vision submodule and may produce a config that downstream tools (e.g., vLLM) reject. If you need a pure text causal-LM handle, use model.language_model after loading.

Benchmark Results

Evaluation Metrics

(1) Selectivity

Refusal rate on explicit Chinese requests — the fraction of cases where the model fails to produce Chinese even though the user explicitly asked for it. Lower is better (respects user intent).

• Lower better (~0): produces Chinese when asked (respects user intent).
• Higher worse (~1): refuses Chinese even when asked (blanket suppression).

Metric	Benchmark Dataset
chin_refusal ↓	In-house 1,000-prompt Chinese elicitation set (e.g., How do you say '사랑' in Chinese? or the Python + Chinese-comment prompt)

(2) Chinese-leak suppression

Korean prompts → Korean answers expected; any Chinese token leaked into the answer is a failure. Metric is the clean-Korean response ratio.

• Higher better (~1): Korean answers stay fully Korean (no Chinese tokens leaked).
• Lower worse (~0): Chinese tokens leak into otherwise-Korean answers.

Metric	Benchmark Dataset
chin_cs ↑	KMMLU Computer Science subjects (free-form Korean generation)
chin_ie ↑	KMMLU Industrial Engineering subjects (free-form Korean generation)
chin_total ↑	KMMLU (free-form Korean generation)

(3) Reasoning / task performance

Metric	Benchmark Dataset
acc_cs ↑	KMMLU Computer Science subjects (multiple-choice log-likelihood comparison)
acc_ie ↑	KMMLU Industrial Engineering subjects (multiple-choice log-likelihood comparison)
acc_total ↑	KMMLU (multiple-choice log-likelihood comparison)
HumanEval ↑	HumanEval (pass@1)
GSM8K ↑	GSM8K (exact-match accuracy)

(4) Full-parameter SFT-persistence

Metric	Details
SRR (Suppression Retention Rate) ↑	Ratio of chin_total after SFT to before SFT. Closer to 1.0 = SFT did not erode the leak-suppression effect. Built on chin_total (not chin_refusal) so direction stays higher-is-better.
|Δ_selectivity| ↓	Absolute change in chin_refusal. Smaller = SFT barely shifted selectivity.

Chinese Suppression (Thinking mode)

Evaluated with enable_thinking=True. The DSLM-LST-9B column is calibrated with thinking enabled.

Metric	Qwen3.5-9B (base)	LST-L1	LST-L2	DSLM-LST-9B
(1) Selectivity
chin_refusal ↓	0.029	0.993	0.992	0.065
(2) Chinese-leak suppression
chin_cs ↑	0.985	1.000	1.000	0.999
chin_ie ↑	0.978	1.000	0.999	0.997
chin_total ↑	0.9717	0.9988	0.9988	0.9927
(3) Reasoning / Task performance
acc_cs ↑	0.811	0.811	0.811	0.811
acc_ie ↑	0.618	0.618	0.620	0.620
acc_total ↑	0.5897	0.5897	0.5893	0.5893
HumanEval ↑	0.6646	0.6768	0.6585	0.6646
GSM8K ↑	0.8749	0.8749	0.8749	0.8749

DSLM-LST-9B keeps chin_refusal at 0.065. It preserves the ability to generate Chinese when the user explicitly asks for it, while still cutting unintended Chinese leakage to the level of chin_total ≈ 0.99. Downstream reasoning (acc_*, HumanEval, GSM8K) is comparable to, or in some cases even better than, the base model.

Chinese Suppression (Non-Thinking mode)

Evaluated with enable_thinking=False. The DSLM-LST-9B column here is a separate think-OFF-calibrated checkpoint (not this release).

Metric	Qwen3.5-9B (base)	LST-L1	LST-L2	DSLM-LST-9B
(1) Selectivity
chin_refusal ↓	0.037	0.966	0.963	0.080
(2) Chinese-leak suppression
chin_cs ↑	0.964	0.999	1.000	0.990
chin_ie ↑	0.934	0.997	0.999	0.983
chin_total ↑	0.9405	0.9974	0.9975	0.9830
(3) Reasoning / Task performance
acc_cs ↑	0.811	0.811	0.811	0.811
acc_ie ↑	0.615	0.614	0.614	0.614
acc_total ↑	0.5900	0.5897	0.5897	0.5897
HumanEval ↑	0.6707	0.6768	0.6707	0.6707
GSM8K ↑	0.8757	0.8749	0.8741	0.8787

Suppression Persistence after SFT-stage (Non-Thinking mode)

Each pipeline was fine-tuned via full-parameter SFT (all weights trainable, no PEFT / LoRA) on the beomi/KoAlpaca-v1.1a dataset. After the SFT stage, DSLM-LST-9B keeps both its Chinese-leak suppression (SRR ≈ 1.000) and its selectivity (|Δ_selectivity| ≈ 0.08) almost unchanged.

Metric	Qwen3.5-9B → SFT	DSLM-LST-9B → SFT
(1) Selectivity
chin_refusal before ↓	0.037	0.080
chin_refusal after ↓	0.128	0.155
|Δ_selectivity| ↓	0.091	0.075
(2) Chinese-leak suppression
chin_total before ↑	0.9405	0.9830
chin_total after ↑	0.9927	0.9926
SRR ↑	1.0555	1.0098

Metric	Qwen3.5-9B (base)	Qwen3.5-9B → SFT	DSLM-LST-9B → SFT
(1) Selectivity
chin_refusal ↓	0.037	0.128	0.155
(2) Chinese-leak suppression
chin_cs ↑	0.964	0.998	0.998
chin_ie ↑	0.934	0.993	0.994
chin_total ↑	0.9405	0.9927	0.9926
(3) Reasoning / Task performance
acc_cs ↑	0.811	0.748	0.751
acc_ie ↑	0.615	0.505	0.509
acc_total ↑	0.5900	0.5202	0.5217
HumanEval ↑	0.6707	0.6037	0.6402
GSM8K ↑	0.8757	0.8211	0.8226

The base model's selectivity shifts substantially after full-parameter SFT (chin_refusal 0.037 → 0.128), while DSLM-LST-9B's suppression behavior remains nearly invariant before and after full-parameter SFT. This shows that LST does not act as a thin surface patch — its effect is encoded in a way that survives downstream fine-tuning.

English Suppression (Non-Thinking mode) — generalization check

To confirm LST is not tied to a specific language pair, we applied the same approach to Llama-3.1-8B-Instruct for English leakage suppression. The DSLM-LST configuration is the only variant that keeps coding (HumanEval) and math (GSM8K) usable while still meaningfully reducing leakage.

Metric	Llama-3.1-8B-Instruct (base)	DSLM-LST (Llama-3.1-8B)
(1) Selectivity
eng_refusal ↓	0.018	0.067
(2) English-leak suppression
eng_cs ↑	0.241	0.365
eng_ie ↑	0.483	0.551
eng_total ↑	0.5391	0.6067
(3) Reasoning / Task performance
acc_cs ↑	0.698	0.727
acc_ie ↑	0.439	0.475
acc_total ↑	0.4417	0.4580
HumanEval ↑	0.6341	0.5671
GSM8K ↑	0.6967	0.6088

Example Outputs

Asked in Korean about the most common clay mineral on the Korean Peninsula, Qwen3.5-9B leaks 9 Chinese / mixed-script tokens (伊利石, кao린, 的) into its answer. DSLM-LST-9B answers the same prompt entirely in Korean (0 Chinese tokens).

Domain-wise Leakage Comparisons

Computer Science

Prompt: TCP 와 UDP 프로토콜의 차이점을 자세히 설명해 주세요.

Qwen3.5-9B (leaks 才开始)	DSLM-LST-9B (clean Korean)
TCP (Transmission Control Protocol) 와 UDP (User Datagram Protocol) 는 인터넷의 가장 기본이 되는 전송 계층 (Transport Layer) 프로토콜로, … 1. 연결 방식 (Connection) • TCP: 연결 지향적 (Connection-oriented)입니다. … 이는 전화통화처럼 상대방이 준비된 상태에서才开始하는 것과 같습니다. • UDP: 비연결 지향적 (Connectionless)입니다. …	TCP (Transmission Control Protocol) 와 UDP (User Datagram Protocol) 는 인터넷과 네트워크 통신의 기반이 되는 전송 계층 (Transport Layer) 의 두 가지 핵심 프로토콜입니다. … 1. 연결 방식 (Connection-Oriented vs Connectionless) • TCP: 연결 지향적입니다. … 마치 우편물을 보낼 때 먼저 주소 확인과 발송 절차를 거치는 것과 같습니다. …

Industrial Engineering

Prompt: 린 (Lean) 제조 방식에서 정의하는 7가지 낭비 (muda) 의 종류를 한국어로 설명해 주세요.

Qwen3.5-9B (leaks 积压)	DSLM-LST-9B (clean Korean)
린 (Lean) 제조 방식에서 7 가지 낭비 (7 Muda) 는 … 핵심 개념입니다. … 1. 과잉 생산 (Overproduction) … 과잉 생산은 재고积压을 유발하고, 보관 비용을 증가시키며, …	린 (Lean) 제조 방식에서 7 가지 낭비 (7 Muda) 는 가치 없는 활동을 제거하여 효율성을 극대화하기 위한 핵심 개념입니다. … 1. 과잉 생산 (Overproduction) … 이는 재고 증가, 보관 비용 상승, 그리고 나중에 발견될 결함 수리 비용을 초래합니다. …

Law

Prompt: 민법상 채권자대위권의 성립요건과 행사 방법을 자세히 설명해 주세요.

Qwen3.5-9B (leaks 享有的)	DSLM-LST-9B (clean Korean)
민법상 채권자대위권은 채권자가 채무자가 타인 (제 3 자) 에게享有的 채권을 행사하지 않을 때, …	민법상 채권자대위권은 채무자가 그 재산에 속하는 권리를 행사하지 않음으로써 채권자의 채권이 불이행될 우려가 있는 경우, 채권자가 채무자의 지위에 서서 그 권리를 직접 행사할 수 있는 제도입니다. …

Cross-lingual Selectivity

When the user explicitly asks for Chinese, DSLM-LST-9B readily produces it. The previous examples showed the model avoiding unwanted Chinese inside an otherwise-Korean answer; the example below shows it emitting Chinese fluently when the user's instruction calls for it.

Prompt: 피보나치 수열의 n번째 항을 반환하는 파이썬 함수를 작성해주세요. 설명은 중국어로 해주세요.

DSLM-LST-9B (code in Python, explanation in Chinese):

다음은 파이썬을 사용하여 피보나치 수열의 n 번째 항을 계산하는 함수입니다.
이 코드는 **반복 (Iterative)** 방식을 사용하여 효율적으로 구현되었습니다.

python
def fibonacci(n):
    """피보나치 수열의 n 번째 항을 반환합니다."""
    if n <= 0: return 0
    elif n == 1: return 1
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b


### 코드 설명 (中文):
1.  **函数定义**: `fibonacci(n)` 接收 `n` 并返回该位置的斐波那契数。
2.  **边界处理**: `n <= 0` 返回 0; `n == 1` 返回 1。
3.  **迭代计算**: 使用 `a, b` 两个变量滚动累加 ...

Thinking-trace Suppression

Qwen3.5-9B's <think> block leaks Chinese even more severely than its final answer, often slipping into Chinese once the reasoning gets stuck. DSLM-LST-9B suppresses that leakage inside the thinking block too.

Prompt: 업무 협조 요청을 받은 기관이 협조 요청 문서에 흠이 있음을 발견한 때에는 접수한 날부터 몇 일 이내에 보완을 요구하여야 하는가? (사무관리규정 개정으로 제외된 문제입니다. 정답은 3번 입니다.)

Chinese-character counts (thinking budget = 8,192):

Metric	Qwen3.5-9B	DSLM-LST-9B
<think> block, Chinese characters	3,472	0
<answer> block, Chinese characters	leaks Chinese (正如您所说, 该规定已被修订/删除)	0 (clean Korean)
"should write in Korean" → Chinese events	484	0

In the base model's trace, every cycle ends with (Wait, I need to write in Korean). Okay, I will write in Korean. — yet the very next token is Chinese again, and the trace slides right back into the same fragment. This loop fires 484 times before the token budget runs out. DSLM-LST-9B targets exactly this failure: Chinese tokens being chosen even right after the model says they should not be. On the same prompt, DSLM-LST-9B's <think> block contains 0 Chinese characters and terminates naturally, and the final user-facing answer is in clean Korean.

Limitations

• Not an instruction-tuned chat model. The adjustment scope is intentionally minimal, so conversational behaviour, instruction-following style, and reasoning patterns are inherited from the base model — only the unintended Chinese-token leakage is mitigated.
• Degraded Chinese generation. Tasks that require Chinese output — Chinese translation, Chinese code comments, bilingual Q&A — will see noticeably lower quality. Use the base Qwen3.5-9B instead for such workloads.
• Multimodal not re-benchmarked. The vision tower weights are bit-identical to the base, so multimodal performance should be unchanged. We have not, however, re-benchmarked the vision pipeline in this release.
• Out-of-distribution robustness. Suppression strength on contexts very different from typical Korean-assistant usage — e.g., highly unusual domains, much longer generations, or atypical prompting styles — has not been separately verified.

License

This model is released under the Apache 2.0 License.

Contact

For questions, feedback, or collaboration inquiries, feel free to reach out via our website.