- Llama Email Fraud Detector (bf16)
Llama Email Fraud Detector (bf16)
A fine-tuned Llama-3.2-3B-Instruct model for structured email fraud/phishing analysis. Given an email, the model outputs a detailed JSON verdict including 11 threat type labels, a 0-100 risk score, human-readable reasoning, and a suggested action.
This is the explanation layer of a dual-model anti-fraud pipeline. The discriminative model (cunxin/roberta-email-fraud-detector, 99.5% accuracy, <50ms) provides a fast binary pre-screen; its result is passed to this generative model as a [CLASSIFIER HINT] prior. The final verdict is reconciled by the backend service.
For GPUs with limited VRAM (< 12 GB), use the AWQ 4-bit quantized version: cunxin/llama-email-fraud-detector-awq.
ๅบไบ Llama-3.2-3B-Instruct ๅพฎ่ฐ็็ปๆๅ้ฎไปถๆฌบ่ฏ/้้ฑผๅๆๆจกๅใ่พๅ ฅไธๅฐ้ฎไปถ๏ผๆจกๅ่พๅบๅ ๅซ 11 ็งๅจ่็ฑปๅๆ ็ญพใ0-100 ้ฃ้ฉๅๆฐใๅฏ่ฏปๆจ็่ฏดๆๅๅปบ่ฎฎๆไฝ็่ฏฆ็ป JSON ๅคๅณใ
ๆฌๆจกๅๆฏๅๆจกๅๅๆฌบ่ฏๆตๆฐด็บฟ็่งฃ้ๅฑใๅคๅซๆจกๅ๏ผcunxin/roberta-email-fraud-detector๏ผ99.5% ๅ็กฎ็๏ผ<50ms๏ผๆไพๅฟซ้ไบๅ
้ข็ญ๏ผๅ
ถ็ปๆไปฅ [CLASSIFIER HINT] ๅ
้ชไผ ๅ
ฅๆฌ็ๆๅผๆจกๅใๆ็ปๅคๅณ็ฑๅ็ซฏๆๅก่ๅใ
ไฝๆพๅญ GPU๏ผ< 12 GB๏ผ่ฏทไฝฟ็จ AWQ 4-bit ้ๅ็๏ผcunxin/llama-email-fraud-detector-awqใ
Model Details / ๆจกๅ่ฏฆๆ
| Architecture | LlamaForCausalLM (Decoder-only Transformer with RoPE, GQA, SwiGLU) |
| Base Model | meta-llama/Llama-3.2-3B-Instruct |
| Parameters | 3,237,063,680 (3.2B) |
| Fine-Tuning | LoRA (r=16, alpha=32, dropout=0.1) merged into base weights |
| Trainable Parameters | 24,313,856 (0.75% of total) |
| Precision | bfloat16 |
| Model Size | 6.4 GB |
| Context Window | 4,096 tokens (inference) / 2,048 tokens (training) |
| Vocabulary | 128,256 tokens |
Lineage / ๆจกๅ่ก็ป
meta-llama/Llama-3.2-3B-Instruct (Meta AI โ instruction-tuned on 3T tokens)
โ
โโโ LoRA fine-tuning (r=16, alpha=32, 7 target modules)
โ Training: ~12K email conversations with structured JSON labels
โ Hint injection: 75% correct / 15% adversarial / 10% no hint
โ
โโโ Merge LoRA adapters into base weights
โ
โโโบ cunxin/llama-email-fraud-detector (this model, bf16, 6.4 GB)
โ
โโโบ cunxin/llama-email-fraud-detector-awq (AWQ 4-bit, 2.2 GB)
Output Format / ่พๅบๆ ผๅผ
The model outputs structured JSON with 6 fields:
ๆจกๅ่พๅบๅ ๅซ 6 ไธชๅญๆฎต็็ปๆๅ JSON๏ผ
{
"is_fraud": true,
"risk_score": 95,
"confidence_level": 0.97,
"detected_threats": ["DOMAIN_MISMATCH", "CREDENTIAL_REQUEST", "URGENCY_FEAR"],
"reason": "The sender domain 'amaz0n-verify.com' typosquats amazon.com. The email requests account credentials via a suspicious URL and uses urgency tactics to pressure immediate action.",
"suggestion": "Do not click any links. Do not enter any credentials. Report this email as phishing to your IT department."
}
Threat Types & Scoring / ๅจ่็ฑปๅไธ่ฏๅ
The model detects 11 threat categories. Risk score = sum of triggered threat points (capped at 100).
ๆจกๅๆฃๆต 11 ็งๅจ่็ฑปๅซใ้ฃ้ฉๅๆฐ = ่งฆๅ็ๅจ่็นๆฐไนๅ๏ผไธ้ 100๏ผใ
| Label / ๆ ็ญพ | Points / ๅๅผ | Description / ๆ่ฟฐ |
|---|---|---|
CREDENTIAL_REQUEST |
35 | Asks for passwords, SSN, credit card numbers / ่ฏทๆฑๅฏ็ ใ่บซไปฝ่ฏๅทใไฟก็จๅกๅท |
DOMAIN_MISMATCH |
30 | Sender domain does not match claimed organization / ๅไปถไบบๅๅไธๅฃฐ็งฐ็็ป็ปไธๅน้ |
URL_DISCREPANCY |
30 | Links point to suspicious or mismatched domains / ้พๆฅๆๅๅฏ็ๆไธๅน้ ็ๅๅ |
TOO_GOOD_TO_BE_TRUE |
30 | Unrealistic promises (lottery wins, free money) / ไธๅๅฎ้ ็ๆฟ่ฏบ๏ผไธญๅฅใๅ ่ดน่ต้๏ผ |
PROMPT_INJECTION |
30 | Attempts to manipulate AI analysis / ่ฏๅพๆ็บต AI ๅๆ |
URGENCY_FEAR |
15 | Pressure tactics ("act now", "account suspended") / ๆฝๅ็ญ็ฅ๏ผ"็ซๅณ่กๅจ"ใ"่ดฆๅทๅทฒๅป็ป"๏ผ |
REPLY_TO_MISMATCH |
15 | Reply-To address differs from sender / ๅๅคๅฐๅไธๅไปถไบบไธๅ |
GENERIC_SALUTATION |
8 | Impersonal greeting ("Dear Customer") / ้ไธชไบบๅ็งฐๅผ๏ผ"ๅฐๆฌ็ๅฎขๆท"๏ผ |
ANOMALOUS_TIMING |
8 | Sent at unusual hours for the timezone / ๅจไธๅฏปๅธธ็ๆถ้ดๅ้ |
MISSING_SIGNATURE |
8 | No professional email signature / ็ผบๅฐไธไธ้ฎไปถ็ญพๅ |
GRAMMAR_ANOMALY |
5 | Unusual grammar or spelling patterns / ๅผๅธธ็่ฏญๆณๆๆผๅๆจกๅผ |
RULE D: Any high-weight threat (>=30 pts) forces is_fraud = true.
่งๅ D๏ผไปปไฝ้ซๆ้ๅจ่๏ผ>=30 ๅ๏ผๅผบๅถ is_fraud = trueใ
Dual-Model Pipeline / ๅๆจกๅๆตๆฐด็บฟ
This model is designed to work with cunxin/roberta-email-fraud-detector in a reconciled pipeline:
ๆฌๆจกๅ่ฎพ่ฎกไธบไธ cunxin/roberta-email-fraud-detector ้ ๅไฝฟ็จ๏ผ
Email Input
โ
โโโโบ RoBERTa (discriminative, <50ms)
โ โ
โ โผ
โ is_fraud=True/False, confidence=0.97, risk_score=99
โ โ
โ โผ [CLASSIFIER HINT]
โโโโบ Llama (generative, ~1-3s) โโโ this model
โ โ
โ โผ
โ Full JSON analysis (11 threat types, reasoning, suggestion)
โ
โผ
Reconciliation (backend)
โ
โผ
Final verdict
STEP 4 Hint Rules / ๆ็คบ่งๅ
When a [CLASSIFIER HINT] is provided, the model applies these rules after its own independent analysis:
| Scenario / ๅบๆฏ | Action / ๆไฝ |
|---|---|
| Both agree / ไธค่ ไธ่ด | Keep generative result / ไฟ็็ๆๅผ็ปๆ |
| Hint=FRAUD, gen=safe, risk < 40 | Follow hint (classifier caught something subtle) / ้ตๅพชๆ็คบ |
| Hint=FRAUD, gen=safe, RULE D triggered | Override hint (generative has hard evidence) / ๆจ็ฟปๆ็คบ |
| Hint=NOT FRAUD, gen=fraud, RULE D triggered | Override hint (generative has hard evidence) / ๆจ็ฟปๆ็คบ |
| Hint=NOT FRAUD, gen=fraud, risk < 60 | Follow hint (only weak signals) / ้ตๅพชๆ็คบ |
Usage / ไฝฟ็จๆนๆณ
With vLLM (Recommended) / ไฝฟ็จ vLLM๏ผๆจ่๏ผ
# Set in .env
MODEL_PATH=cunxin/llama-email-fraud-detector
# Start service
docker compose --profile gpu up -d
# Test
curl -X POST http://localhost:8000/predict_generative \
-H "Content-Type: application/json" \
-d '{"sender":"security@amaz0n-verify.com","subject":"URGENT: Account locked","content":"Click to verify: http://amaz0n-secure.xyz"}'
With Transformers / ไฝฟ็จ Transformers
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch, json
model_name = "cunxin/llama-email-fraud-detector"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto")
email = json.dumps({
"date": "2026-02-25T10:00:00Z",
"sender": "security@amaz0n-verify.com",
"recipient": "you@example.com",
"subject": "URGENT: Your account has been locked",
"content": "Click here to verify: http://amaz0n-secure.xyz/verify"
})
messages = [
{"role": "system", "content": "You are an email fraud analyst. Analyze the email and return a JSON verdict."},
{"role": "user", "content": f"Analyze the following email:\n{email}"}
]
input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
output = model.generate(input_ids, max_new_tokens=512, temperature=0.1)
response = tokenizer.decode(output[0][input_ids.shape[-1]:], skip_special_tokens=True)
print(response)
Training / ่ฎญ็ป่ฏฆๆ
Fine-Tuning Method / ๅพฎ่ฐๆนๆณ
| Method | LoRA (Low-Rank Adaptation) via peft + SFT via trl.SFTTrainer |
| LoRA Rank | 16 |
| LoRA Alpha | 32 (effective scale = 2x) |
| LoRA Dropout | 0.1 |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Optimizer | AdamW (weight_decay=0.05, label_smoothing=0.05) |
| LR Schedule | Cosine with 10% linear warmup |
| Learning Rate | 2e-4 |
| Epochs | 3 |
| Batch Size | 2 (grad_accum=8, effective=16) |
| Max Sequence Length | 2,048 tokens |
| Mixed Precision | FP16 (CUDA) |
| Loss | Causal LM (next-token prediction on assistant turn only) |
Training Data / ่ฎญ็ปๆฐๆฎ
~12,000 email conversations generated from the Enron corpus and synthetic sources:
็บฆ 12,000 ไธช้ฎไปถๅฏน่ฏ๏ผไป Enron ่ฏญๆๅบๅๅๆๆฐๆฎ็ๆ๏ผ
| Source / ๆฅๆบ | Count / ๆฐ้ | Role / ็จ้ |
|---|---|---|
| Fraud emails (train) | ~500 per class | Primary training / ไธป่ฆ่ฎญ็ป้ |
| Normal emails (train) | ~500 per class | Primary training / ไธป่ฆ่ฎญ็ป้ |
| Correction pool (optional) | ~14K fraud + ~15K normal | Extra training via --include-correction |
| AI-generated modern emails | ~800 per class | Extra training via --include-aigen |
Each email is converted into a 3-turn chat conversation (system prompt -> user email -> assistant JSON verdict) with:
- 75% include a correct
[CLASSIFIER HINT](teach trust of accurate priors) - 15% include an adversarial wrong hint (teach RULE D override)
- 10% have no hint (teach independent reasoning)
- 60% of fraud examples include
[HEURISTIC ANALYSIS]context
Hardware Requirements / ็กฌไปถ่ฆๆฑ
| Configuration / ้ ็ฝฎ | VRAM / ๆพๅญ |
|---|---|
| bf16 (this model) | >= 12 GB (RTX 3060 desktop, RTX 4070+) |
| AWQ 4-bit (llama-email-fraud-detector-awq) | >= 6 GB (RTX 3050, 3060 laptop) |
Intended Use / ้ขๆ็จ้
- Email fraud/phishing detection with detailed threat analysis and human-readable reasoning / ๆไพ่ฏฆ็ปๅจ่ๅๆๅๅฏ่ฏปๆจ็็้ฎไปถๆฌบ่ฏ/้้ฑผๆฃๆต
- Explanation layer for automated email security systems / ่ชๅจๅ้ฎไปถๅฎๅ จ็ณป็ป็่งฃ้ๅฑ
- Part of a multi-model pipeline (discriminative pre-screen + generative analysis + reconciliation) / ๅคๆจกๅๆตๆฐด็บฟ็็ปๆ้จๅ
- Microsoft Office Add-in integration for Outlook/Word / Microsoft Office ๆไปถ้ๆ
Limitations / ๅฑ้ๆง
- Primarily trained on English emails; may underperform on other languages / ไธป่ฆๅจ่ฑๆ้ฎไปถไธ่ฎญ็ป
- Training data includes Enron corpus (early 2000s); modern attack patterns partially covered by AI-generated synthetic data / ่ฎญ็ปๆฐๆฎๅ ๅซ Enron ่ฏญๆๅบ๏ผ็ฐไปฃๆปๅปๆจกๅผ้จๅ็ฑ AI ็ๆๅๆๆฐๆฎ่ฆ็
- Inference latency ~1-3s per email (use RoBERTa for real-time filtering) / ๆจ็ๅปถ่ฟ็บฆ 1-3 ็ง/ๅฐ
- Structured JSON output depends on prompt engineering; edge cases may produce malformed JSON / ่พน็ผๆ ๅตๅฏ่ฝไบง็ๆ ผๅผ้่ฏฏ็ JSON
Related Models / ็ธๅ ณๆจกๅ
| Model / ๆจกๅ | Type / ็ฑปๅ | Size / ๅคงๅฐ | Speed / ้ๅบฆ | Use Case / ็จ้ |
|---|---|---|---|---|
| cunxin/roberta-email-fraud-detector | Discriminative | 475 MB | <50ms | Fast binary pre-screen / ๅฟซ้ไบๅ ้ข็ญ |
| cunxin/llama-email-fraud-detector (this) | Generative | 6.4 GB | ~1-3s | Detailed threat analysis / ่ฏฆ็ปๅจ่ๅๆ |
| cunxin/llama-email-fraud-detector-awq | Generative (quantized) | 2.2 GB | ~1-3s | Same as above, for low VRAM / ๅไธ๏ผไฝๆพๅญ็ |
Citation / ๅผ็จ
@misc{cunxin2025llama-email-fraud,
title={Llama Email Fraud Detector},
author={cunxin},
year={2025},
url={https://huggingface.co/cunxin/llama-email-fraud-detector}
}
- Downloads last month
- 15
Model tree for cunxin/llama-email-fraud-detector
Dataset used to train cunxin/llama-email-fraud-detector
Evaluation results
- Threat Types on Enron Email + Synthetic (held-out test set)self-reported11.000