Llama Email Fraud Detector (bf16)

A fine-tuned Llama-3.2-3B-Instruct model for structured email fraud/phishing analysis. Given an email, the model outputs a detailed JSON verdict including 11 threat type labels, a 0-100 risk score, human-readable reasoning, and a suggested action.

This is the explanation layer of a dual-model anti-fraud pipeline. The discriminative model (cunxin/roberta-email-fraud-detector, 99.5% accuracy, <50ms) provides a fast binary pre-screen; its result is passed to this generative model as a [CLASSIFIER HINT] prior. The final verdict is reconciled by the backend service.

For GPUs with limited VRAM (< 12 GB), use the AWQ 4-bit quantized version: cunxin/llama-email-fraud-detector-awq.


ๅŸบไบŽ Llama-3.2-3B-Instruct ๅพฎ่ฐƒ็š„็ป“ๆž„ๅŒ–้‚ฎไปถๆฌบ่ฏˆ/้’“้ฑผๅˆ†ๆžๆจกๅž‹ใ€‚่พ“ๅ…ฅไธ€ๅฐ้‚ฎไปถ๏ผŒๆจกๅž‹่พ“ๅ‡บๅŒ…ๅซ 11 ็งๅจ่ƒ็ฑปๅž‹ๆ ‡็ญพใ€0-100 ้ฃŽ้™ฉๅˆ†ๆ•ฐใ€ๅฏ่ฏปๆŽจ็†่ฏดๆ˜Žๅ’Œๅปบ่ฎฎๆ“ไฝœ็š„่ฏฆ็ป† JSON ๅˆคๅ†ณใ€‚

ๆœฌๆจกๅž‹ๆ˜ฏๅŒๆจกๅž‹ๅๆฌบ่ฏˆๆตๆฐด็บฟ็š„่งฃ้‡Šๅฑ‚ใ€‚ๅˆคๅˆซๆจกๅž‹๏ผˆcunxin/roberta-email-fraud-detector๏ผŒ99.5% ๅ‡†็กฎ็އ๏ผŒ<50ms๏ผ‰ๆไพ›ๅฟซ้€ŸไบŒๅ…ƒ้ข„็ญ›๏ผŒๅ…ถ็ป“ๆžœไปฅ [CLASSIFIER HINT] ๅ…ˆ้ชŒไผ ๅ…ฅๆœฌ็”Ÿๆˆๅผๆจกๅž‹ใ€‚ๆœ€็ปˆๅˆคๅ†ณ็”ฑๅŽ็ซฏๆœๅŠก่žๅˆใ€‚

ไฝŽๆ˜พๅญ˜ GPU๏ผˆ< 12 GB๏ผ‰่ฏทไฝฟ็”จ AWQ 4-bit ้‡ๅŒ–็‰ˆ๏ผšcunxin/llama-email-fraud-detector-awqใ€‚

Model Details / ๆจกๅž‹่ฏฆๆƒ…

Architecture LlamaForCausalLM (Decoder-only Transformer with RoPE, GQA, SwiGLU)
Base Model meta-llama/Llama-3.2-3B-Instruct
Parameters 3,237,063,680 (3.2B)
Fine-Tuning LoRA (r=16, alpha=32, dropout=0.1) merged into base weights
Trainable Parameters 24,313,856 (0.75% of total)
Precision bfloat16
Model Size 6.4 GB
Context Window 4,096 tokens (inference) / 2,048 tokens (training)
Vocabulary 128,256 tokens

Lineage / ๆจกๅž‹่ก€็ปŸ

meta-llama/Llama-3.2-3B-Instruct (Meta AI โ€” instruction-tuned on 3T tokens)
    โ”‚
    โ”œโ”€โ”€ LoRA fine-tuning (r=16, alpha=32, 7 target modules)
    โ”‚     Training: ~12K email conversations with structured JSON labels
    โ”‚     Hint injection: 75% correct / 15% adversarial / 10% no hint
    โ”‚
    โ”œโ”€โ”€ Merge LoRA adapters into base weights
    โ”‚
    โ”œโ”€โ–บ cunxin/llama-email-fraud-detector (this model, bf16, 6.4 GB)
    โ”‚
    โ””โ”€โ–บ cunxin/llama-email-fraud-detector-awq (AWQ 4-bit, 2.2 GB)

Output Format / ่พ“ๅ‡บๆ ผๅผ

The model outputs structured JSON with 6 fields:

ๆจกๅž‹่พ“ๅ‡บๅŒ…ๅซ 6 ไธชๅญ—ๆฎต็š„็ป“ๆž„ๅŒ– JSON๏ผš

{
  "is_fraud": true,
  "risk_score": 95,
  "confidence_level": 0.97,
  "detected_threats": ["DOMAIN_MISMATCH", "CREDENTIAL_REQUEST", "URGENCY_FEAR"],
  "reason": "The sender domain 'amaz0n-verify.com' typosquats amazon.com. The email requests account credentials via a suspicious URL and uses urgency tactics to pressure immediate action.",
  "suggestion": "Do not click any links. Do not enter any credentials. Report this email as phishing to your IT department."
}

Threat Types & Scoring / ๅจ่ƒ็ฑปๅž‹ไธŽ่ฏ„ๅˆ†

The model detects 11 threat categories. Risk score = sum of triggered threat points (capped at 100).

ๆจกๅž‹ๆฃ€ๆต‹ 11 ็งๅจ่ƒ็ฑปๅˆซใ€‚้ฃŽ้™ฉๅˆ†ๆ•ฐ = ่งฆๅ‘็š„ๅจ่ƒ็‚นๆ•ฐไน‹ๅ’Œ๏ผˆไธŠ้™ 100๏ผ‰ใ€‚

Label / ๆ ‡็ญพ Points / ๅˆ†ๅ€ผ Description / ๆ่ฟฐ
CREDENTIAL_REQUEST 35 Asks for passwords, SSN, credit card numbers / ่ฏทๆฑ‚ๅฏ†็ ใ€่บซไปฝ่ฏๅทใ€ไฟก็”จๅกๅท
DOMAIN_MISMATCH 30 Sender domain does not match claimed organization / ๅ‘ไปถไบบๅŸŸๅไธŽๅฃฐ็งฐ็š„็ป„็ป‡ไธๅŒน้…
URL_DISCREPANCY 30 Links point to suspicious or mismatched domains / ้“พๆŽฅๆŒ‡ๅ‘ๅฏ็–‘ๆˆ–ไธๅŒน้…็š„ๅŸŸๅ
TOO_GOOD_TO_BE_TRUE 30 Unrealistic promises (lottery wins, free money) / ไธๅˆ‡ๅฎž้™…็š„ๆ‰ฟ่ฏบ๏ผˆไธญๅฅ–ใ€ๅ…่ดน่ต„้‡‘๏ผ‰
PROMPT_INJECTION 30 Attempts to manipulate AI analysis / ่ฏ•ๅ›พๆ“็บต AI ๅˆ†ๆž
URGENCY_FEAR 15 Pressure tactics ("act now", "account suspended") / ๆ–ฝๅŽ‹็ญ–็•ฅ๏ผˆ"็ซ‹ๅณ่กŒๅŠจ"ใ€"่ดฆๅทๅทฒๅ†ป็ป“"๏ผ‰
REPLY_TO_MISMATCH 15 Reply-To address differs from sender / ๅ›žๅคๅœฐๅ€ไธŽๅ‘ไปถไบบไธๅŒ
GENERIC_SALUTATION 8 Impersonal greeting ("Dear Customer") / ้žไธชไบบๅŒ–็งฐๅ‘ผ๏ผˆ"ๅฐŠๆ•ฌ็š„ๅฎขๆˆท"๏ผ‰
ANOMALOUS_TIMING 8 Sent at unusual hours for the timezone / ๅœจไธๅฏปๅธธ็š„ๆ—ถ้—ดๅ‘้€
MISSING_SIGNATURE 8 No professional email signature / ็ผบๅฐ‘ไธ“ไธš้‚ฎไปถ็ญพๅ
GRAMMAR_ANOMALY 5 Unusual grammar or spelling patterns / ๅผ‚ๅธธ็š„่ฏญๆณ•ๆˆ–ๆ‹ผๅ†™ๆจกๅผ

RULE D: Any high-weight threat (>=30 pts) forces is_fraud = true.

่ง„ๅˆ™ D๏ผšไปปไฝ•้ซ˜ๆƒ้‡ๅจ่ƒ๏ผˆ>=30 ๅˆ†๏ผ‰ๅผบๅˆถ is_fraud = trueใ€‚

Dual-Model Pipeline / ๅŒๆจกๅž‹ๆตๆฐด็บฟ

This model is designed to work with cunxin/roberta-email-fraud-detector in a reconciled pipeline:

ๆœฌๆจกๅž‹่ฎพ่ฎกไธบไธŽ cunxin/roberta-email-fraud-detector ้…ๅˆไฝฟ็”จ๏ผš

Email Input
    โ”‚
    โ”œโ”€โ”€โ–บ RoBERTa (discriminative, <50ms)
    โ”‚         โ”‚
    โ”‚         โ–ผ
    โ”‚     is_fraud=True/False, confidence=0.97, risk_score=99
    โ”‚         โ”‚
    โ”‚         โ–ผ  [CLASSIFIER HINT]
    โ”œโ”€โ”€โ–บ Llama (generative, ~1-3s) โ—„โ”€โ”€ this model
    โ”‚         โ”‚
    โ”‚         โ–ผ
    โ”‚     Full JSON analysis (11 threat types, reasoning, suggestion)
    โ”‚
    โ–ผ
Reconciliation (backend)
    โ”‚
    โ–ผ
Final verdict

STEP 4 Hint Rules / ๆ็คบ่ง„ๅˆ™

When a [CLASSIFIER HINT] is provided, the model applies these rules after its own independent analysis:

Scenario / ๅœบๆ™ฏ Action / ๆ“ไฝœ
Both agree / ไธค่€…ไธ€่‡ด Keep generative result / ไฟ็•™็”Ÿๆˆๅผ็ป“ๆžœ
Hint=FRAUD, gen=safe, risk < 40 Follow hint (classifier caught something subtle) / ้ตๅพชๆ็คบ
Hint=FRAUD, gen=safe, RULE D triggered Override hint (generative has hard evidence) / ๆŽจ็ฟปๆ็คบ
Hint=NOT FRAUD, gen=fraud, RULE D triggered Override hint (generative has hard evidence) / ๆŽจ็ฟปๆ็คบ
Hint=NOT FRAUD, gen=fraud, risk < 60 Follow hint (only weak signals) / ้ตๅพชๆ็คบ

Usage / ไฝฟ็”จๆ–นๆณ•

With vLLM (Recommended) / ไฝฟ็”จ vLLM๏ผˆๆŽจ่๏ผ‰

# Set in .env
MODEL_PATH=cunxin/llama-email-fraud-detector

# Start service
docker compose --profile gpu up -d

# Test
curl -X POST http://localhost:8000/predict_generative \
  -H "Content-Type: application/json" \
  -d '{"sender":"security@amaz0n-verify.com","subject":"URGENT: Account locked","content":"Click to verify: http://amaz0n-secure.xyz"}'

With Transformers / ไฝฟ็”จ Transformers

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch, json

model_name = "cunxin/llama-email-fraud-detector"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto")

email = json.dumps({
    "date": "2026-02-25T10:00:00Z",
    "sender": "security@amaz0n-verify.com",
    "recipient": "you@example.com",
    "subject": "URGENT: Your account has been locked",
    "content": "Click here to verify: http://amaz0n-secure.xyz/verify"
})

messages = [
    {"role": "system", "content": "You are an email fraud analyst. Analyze the email and return a JSON verdict."},
    {"role": "user", "content": f"Analyze the following email:\n{email}"}
]

input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
output = model.generate(input_ids, max_new_tokens=512, temperature=0.1)
response = tokenizer.decode(output[0][input_ids.shape[-1]:], skip_special_tokens=True)
print(response)

Training / ่ฎญ็ปƒ่ฏฆๆƒ…

Fine-Tuning Method / ๅพฎ่ฐƒๆ–นๆณ•

Method LoRA (Low-Rank Adaptation) via peft + SFT via trl.SFTTrainer
LoRA Rank 16
LoRA Alpha 32 (effective scale = 2x)
LoRA Dropout 0.1
Target Modules q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Optimizer AdamW (weight_decay=0.05, label_smoothing=0.05)
LR Schedule Cosine with 10% linear warmup
Learning Rate 2e-4
Epochs 3
Batch Size 2 (grad_accum=8, effective=16)
Max Sequence Length 2,048 tokens
Mixed Precision FP16 (CUDA)
Loss Causal LM (next-token prediction on assistant turn only)

Training Data / ่ฎญ็ปƒๆ•ฐๆฎ

~12,000 email conversations generated from the Enron corpus and synthetic sources:

็บฆ 12,000 ไธช้‚ฎไปถๅฏน่ฏ๏ผŒไปŽ Enron ่ฏญๆ–™ๅบ“ๅ’Œๅˆๆˆๆ•ฐๆฎ็”Ÿๆˆ๏ผš

Source / ๆฅๆบ Count / ๆ•ฐ้‡ Role / ็”จ้€”
Fraud emails (train) ~500 per class Primary training / ไธป่ฆ่ฎญ็ปƒ้›†
Normal emails (train) ~500 per class Primary training / ไธป่ฆ่ฎญ็ปƒ้›†
Correction pool (optional) ~14K fraud + ~15K normal Extra training via --include-correction
AI-generated modern emails ~800 per class Extra training via --include-aigen

Each email is converted into a 3-turn chat conversation (system prompt -> user email -> assistant JSON verdict) with:

  • 75% include a correct [CLASSIFIER HINT] (teach trust of accurate priors)
  • 15% include an adversarial wrong hint (teach RULE D override)
  • 10% have no hint (teach independent reasoning)
  • 60% of fraud examples include [HEURISTIC ANALYSIS] context

Hardware Requirements / ็กฌไปถ่ฆๆฑ‚

Configuration / ้…็ฝฎ VRAM / ๆ˜พๅญ˜
bf16 (this model) >= 12 GB (RTX 3060 desktop, RTX 4070+)
AWQ 4-bit (llama-email-fraud-detector-awq) >= 6 GB (RTX 3050, 3060 laptop)

Intended Use / ้ข„ๆœŸ็”จ้€”

  • Email fraud/phishing detection with detailed threat analysis and human-readable reasoning / ๆไพ›่ฏฆ็ป†ๅจ่ƒๅˆ†ๆžๅ’Œๅฏ่ฏปๆŽจ็†็š„้‚ฎไปถๆฌบ่ฏˆ/้’“้ฑผๆฃ€ๆต‹
  • Explanation layer for automated email security systems / ่‡ชๅŠจๅŒ–้‚ฎไปถๅฎ‰ๅ…จ็ณป็ปŸ็š„่งฃ้‡Šๅฑ‚
  • Part of a multi-model pipeline (discriminative pre-screen + generative analysis + reconciliation) / ๅคšๆจกๅž‹ๆตๆฐด็บฟ็š„็ป„ๆˆ้ƒจๅˆ†
  • Microsoft Office Add-in integration for Outlook/Word / Microsoft Office ๆ’ไปถ้›†ๆˆ

Limitations / ๅฑ€้™ๆ€ง

  • Primarily trained on English emails; may underperform on other languages / ไธป่ฆๅœจ่‹ฑๆ–‡้‚ฎไปถไธŠ่ฎญ็ปƒ
  • Training data includes Enron corpus (early 2000s); modern attack patterns partially covered by AI-generated synthetic data / ่ฎญ็ปƒๆ•ฐๆฎๅŒ…ๅซ Enron ่ฏญๆ–™ๅบ“๏ผŒ็Žฐไปฃๆ”ปๅ‡ปๆจกๅผ้ƒจๅˆ†็”ฑ AI ็”Ÿๆˆๅˆๆˆๆ•ฐๆฎ่ฆ†็›–
  • Inference latency ~1-3s per email (use RoBERTa for real-time filtering) / ๆŽจ็†ๅปถ่ฟŸ็บฆ 1-3 ็ง’/ๅฐ
  • Structured JSON output depends on prompt engineering; edge cases may produce malformed JSON / ่พน็ผ˜ๆƒ…ๅ†ตๅฏ่ƒฝไบง็”Ÿๆ ผๅผ้”™่ฏฏ็š„ JSON

Related Models / ็›ธๅ…ณๆจกๅž‹

Model / ๆจกๅž‹ Type / ็ฑปๅž‹ Size / ๅคงๅฐ Speed / ้€Ÿๅบฆ Use Case / ็”จ้€”
cunxin/roberta-email-fraud-detector Discriminative 475 MB <50ms Fast binary pre-screen / ๅฟซ้€ŸไบŒๅ…ƒ้ข„็ญ›
cunxin/llama-email-fraud-detector (this) Generative 6.4 GB ~1-3s Detailed threat analysis / ่ฏฆ็ป†ๅจ่ƒๅˆ†ๆž
cunxin/llama-email-fraud-detector-awq Generative (quantized) 2.2 GB ~1-3s Same as above, for low VRAM / ๅŒไธŠ๏ผŒไฝŽๆ˜พๅญ˜็‰ˆ

Citation / ๅผ•็”จ

@misc{cunxin2025llama-email-fraud,
  title={Llama Email Fraud Detector},
  author={cunxin},
  year={2025},
  url={https://huggingface.co/cunxin/llama-email-fraud-detector}
}
Downloads last month
15
Safetensors
Model size
3B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for cunxin/llama-email-fraud-detector

Adapter
(896)
this model
Quantizations
1 model

Dataset used to train cunxin/llama-email-fraud-detector

Evaluation results

  • Threat Types on Enron Email + Synthetic (held-out test set)
    self-reported
    11.000