Spaces:
Sleeping
Sleeping
Commit History
-
Replace mermaid block in README.md with ASCII diagram (renders in HF blob viewer)
Verified signature576b083 -
Replace mermaid block in BLOG.md with ASCII diagram (renders in HF blob viewer)
Verified signature659254f -
Update models.py: humanise voice, narrative, captions
Verified signaturecf2195b -
Update model_cards/sft_adapter_README.md: humanise voice, narrative, captions
Verified signaturef9f862c -
Update gradio_app.py: humanise voice, narrative, captions
Verified signature918032b -
Update docs/architecture.md: humanise voice, narrative, captions
Verified signature83e5379 -
Update devops_pipeline_gym_colab.ipynb: humanise voice, narrative, captions
Verified signatured9d5941 -
Update README.md: humanise voice, narrative, captions
Verified signature8f2f28b -
Update BLOG.md: humanise voice, narrative, captions
Verified signature28ac483 -
BLOG: add header rows to TL;DR + SFT-setup tables (was rendering empty bars)
Verified signature1d23f7f -
frontier_results: add 5 frontier baseline JSONs (evidence behind comparison numbers)
Verified signature2ee41ce -
Notebook: strip PREVIOUS HANDOFF NOTES line; remove broken Kaggle badge
Verified signature094e300 -
grpo_train: strip PREVIOUS HANDOFF NOTES from prompt template
Verified signaturec603d39 -
video_script: fill placeholders, fix framing line
Verified signature1c58eb9 -
README: inline architecture mermaid; +200 steps in GRPO; drop broken Kaggle badge
Verified signatureaf9094b -
gradio_app docstring: stale +3.225 -> +1.156
Verified signatureae1bf4f -
BLOG: rebuild as full artifact (mermaid arch, tables, results, GRPO chart, code, artifacts list)
Verified signature79f7353 -
Honest reframe: baseline is Qwen2.5-7B (not Qwen3-1.7B); trained 1.7B beats every untrained baseline tested
Verified signature4083da2 -
Honest reframe: baseline is Qwen2.5-7B (not Qwen3-1.7B); trained 1.7B beats every untrained baseline tested
Verified signaturef33d61d -
Replace 30-step GRPO curve with 200-step retry curve
Verified signature843984e -
README: footnote on single-seed vs n=3 methodology
Verified signature639e1d6 -
Colab: drop Track-IO link, add concrete +1.156 number in wrap-up
Verified signatureda8ba22 -
Colab notebook with executed output cells visible to judges
Verified signatured287b59 -
Lock in real numbers: +1.156 same-model delta, beats all 70B+ frontier (BLOG.md)
Verified signature3c01c86 -
Lock in real numbers: +1.156 same-model delta, beats all 70B+ frontier (README.md)
Verified signaturecaa23ec -
README: add Gradio Demo Space badge
Verified signature54cf1f5 -
Drop dead Track-IO link, add Kaggle badge
Verified signaturebabdc78 -
Colab: align baseline+trained temp to 0.3 (apples-to-apples)
Verified signature814072d -
Colab: add Kaggle badge, honest GRPO cell, cleaner wrap-up
Verified signature32e0543 -
Hackathon submission: new README (3-5 min read), BLOG.md narrative, frontier baselines, design-principles framing
Verified signature40de84e
-
README: add Colab/Kaggle badges + judge re-run instructions + adapter links
Surfaces the Colab notebook prominently with the standard badges (Colab + Kaggle) directly below the tagline. Adds a one-line "judges can re-run in 15 min" instruction so reviewers see the path to verification before scrolling past the hero section. Also adds direct Hub links for: - Trained adapter (yashash045/devops-pipeline-gym-trained) - SFT warmup adapter (yashash045/devops-pipeline-gym-sft-adapter) - Track-IO live training dashboard (yashash045/dpg-trackio) Per Help Guide: "A reviewer should be able to read your README in 3-5 minutes and want to try your environment." Badges + adapter links at the top maximize the chance of that try-it path being taken.20bea6e -
Submission deliverables: Colab notebook + per-step CSV + trainer log upload
Adds three concrete artifacts to satisfy hackathon non-negotiable requirements (per [External] Meta OpenEnv Hackathon Participant Help Guide and FAQ): 1. devops_pipeline_gym_colab.ipynb (NEW, 21 cells, 539 lines) Required by FAQ: "A working training script using Unsloth or HF TRL, ideally as a Colab notebook so judges can re-run it." The notebook: - Connects to the live HF Space env (no local install needed) - Runs an untrained baseline episode via HF Inference Router - Runs a trained-adapter episode loading our published adapter (yashash045/devops-pipeline-gym-trained) via Unsloth + PEFT 4-bit - Renders side-by-side comparison (reward, steps, success) - Optional: tiny 5-step GRPO demo on Colab/Kaggle T4 to prove the training pipeline runs end-to-end on free-tier GPU Compatible with both Colab T4 and Kaggle T4. Auto-pulls HF_TOKEN from Colab secrets, Kaggle secrets, or env var. 2. training/grpo_train.py — _RewardCSVCallback (NEW) Per-ste…cc203b9 -
Phase M Stage B: align fully with kube-sre-gym (sid-rp) patterns + A100 prep
Studied https://github.com/sid-rp/kube-sre-gym and lifted every config delta. They won 1st place in the prior OpenEnv hackathon ($15K, PyTorch + Cerebral Valley) on the same TRL+vLLM+colocate stack we're trying to make work, so their config is the proven reference for our stack class. Also bumping --max-steps to 200 (full Stage B) and switching to A100 flavor at launch — yashash045 has 5+ hours of training window remaining, hardware constraints removed, time to ship the production run. == changes to scripts/phase_m_grpo_job.py == [5/7] subprocess args: - --max-steps 30 -> 200 (full Stage B, A100 makes this affordable) - --learning-rate 5e-6 -> 2e-6 (kube-sre-gym pattern; lower for stability with cosine schedule) - + --grad-accum 8 (effective batch = 4×8 = 32 prompts per gradient; kube-sre-gym uses 8 for "stable GRPO signal") - [5/7] print updated to "200 ste…9f7a631 -
Phase M Option 3 v3: 3 fixes lifted from kube-sre-gym (sid-rp) GRPO patterns
After studying https://github.com/sid-rp/kube-sre-gym/blob/main/train.py (the other OpenEnv hackathon submission, also vLLM colocate + GRPO), identified 3 differences from our stack. Two are immediately actionable. FIX 1 — PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True scripts/phase_m_grpo_job.py [5/7] subprocess env dict Their inline comment: "critical for TRL+vLLM colocate on 80GB" Lets PyTorch's CUDA allocator reuse fragmented GPU memory between trainer state and vLLM's serving cache. Without this, allocations can stall waiting for contiguous memory regions that never free. Plausibly the root cause of our silent hang at "Will smartly offload gradients to save VRAM!" — we have memory available in aggregate but not contiguous. This env var enables segment expansion. FIX 2 — Switch from Unsloth's "smart" gradient offload to standard PyTorch checkpointing when --use-vllm is set training/grpo_train.py FastLanguageModel.get_peft_model() Old: use_gradient_ch…a598d78 -
Phase M Option 3 retry: vLLM debug logging + reduce gpu_memory + max-steps 30
Previous Option 3 attempt (commit 7a266c9) hung at: Unsloth: Will smartly offload gradients to save VRAM! No further log lines. No GPU heartbeat in Track-IO. HF Job stayed in "Running" status — Python process alive but blocked. Most likely cause: GPU memory contention deadlock between Unsloth's gradient offload and vLLM's first weight transfer in colocate mode at gpu_memory_utilization=0.6 (14.4 GB on L4's 24 GB). Three changes: 1. VLLM_LOGGING_LEVEL=DEBUG env var (scripts/phase_m_grpo_job.py) vLLM is silent after Unsloth's last log. DEBUG verbosity will show what vLLM is actually doing during weight load + first generation, so the next attempt's logs reveal the hang point. 2. gpu_memory_utilization 0.6 -> 0.4 (training/grpo_train.py) Both in fpt_kwargs (FastLanguageModel.from_pretrained) and in GRPOConfig probe (vllm_gpu_memory_utilization). Reduces vLLM's memory footprint from ~14.4 GB to ~9.6 GB, leaving ~14 GB for trainer state (gradients, optimizer, acti…f2a191c -
Phase M Option 3 hotfix: re-add vllm to PEP 723 deps
Stage A relaunch failed at FastLanguageModel.from_pretrained with: ImportError: Unsloth: Please install vLLM before enabling `fast_inference`! Root cause: vllm>=0.6.0 was in PEP 723 during the vLLM debug detour (commits 22623bb..1a604b3) but got removed when we reverted to the last-known-good non-vLLM state (commit bf1c711, "revert wrapper + trainer to 4940a15"). 4940a15 predates the vLLM work, so it never had vllm in deps. Option 3 (commit fcd962c) re-enabled --use-vllm in the subprocess args + fast_inference=True / enforce_eager=True in fpt_kwargs, but forgot to restore the corresponding PEP 723 dependency. UV venv ships unsloth without vllm, Unsloth's loader.py:358 raises ImportError when fast_inference=True is passed to a venv that doesn't have vllm. Fix: re-add `"vllm>=0.6.0"` to the PEP 723 dependencies header, right after unsloth (matches the original placement from the vLLM debug session). Side effect: [2/7] install will be ~1-2 min slower because vllm brings in a chunk…7a266c9 -
Phase M Option 3: vLLM single LoRA, no SFT adapter
Stacked LoRA (frozen sft_warmup + trainable GRPO) + Unsloth vLLM = AttributeError on Unsloth's load_lora method (proven yesterday at commit 1a604b3). Single LoRA on raw base + vLLM colocate WORKS (proven at commit c701cb1 — vLLM init succeeded, KV cache loaded; the failure was specifically in stacked-LoRA load_lora). Drop the SFT adapter path. Train GRPO directly on raw Qwen3-1.7B-bnb-4bit with a single trainable LoRA via FastLanguageModel.get_peft_model. vLLM colocate with enforce_eager (proven working). Trade-off: lose SFT warmup (78 trajectories of demonstration data) that would have given the model a head-start on JSON action format. Gain: vLLM ~3-5x speedup. 100 steps × ~30 sec = ~50 min training. Changes to training/grpo_train.py: - FastLanguageModel.from_pretrained: add fpt_kwargs dict; when args.use_vllm is set, also pass fast_inference=True, max_lora_rank=16, gpu_memory_utilization=0.6, enforce_eager=True. - SFT adapter loading: wrap in `if args.sft_adapter_path and a…fcd962c -
Phase M: handle string config_edits + remove debug logger
DEBUG logging (1866fe2) revealed root cause: the model emits config_edits as a STRING ("" or "none") for actions that don't need config edits (view_pipeline, deploy, approve, abort, etc.) instead of omitting the field or sending null. Sample observations from training stdout: - 'config_edits': '' <- empty string - 'config_edits': 'none' <- literal string "none" - 'config_edits': [] <- valid empty list (this case worked) Pydantic schema is Optional[List[ConfigEdit]] which accepts None or list, but rejects any string with "Input should be a valid list". Why the model does this: SFT trajectories where config_edits is absent simply don't include the key in the assistant JSON. But under generation drift (slightly OOD prompts even with the c6c5f7f strip-fix), the model occasionally fills in the field with a string-typed placeholder. Fix in training/grpo_train.py parse_completion: extend the existing Option A coercion block to handle strings — drop the field entirely via …e196bc3 -
Phase M debug: log raw parse_completion data to see config_edits shape
Stage B (aed29b5) hit same validation error after Option A coercion was applied. Two possible causes: 1. Coercion placement is wrong (verified: it IS in correct position — inside parse_completion success try block, after json.loads, before lowercase normalization and return) 2. Model is emitting config_edits in a shape Option A didn't anticipate (multi-key flat dict, list of strings, nested object, etc.) Adding temporary debug log to dump the raw `data` dict at the very end of parse_completion (after coercion, after lowercase normalization, right before returning to reward_function which calls PipelineAction). This will reveal exactly what shape config_edits has when Pydantic rejects it, plus any other surprising structure in the dict. Will revert this debug log after diagnosis.1866fe2 -
Phase M Stage B: Option A coerce dict->list + bump to 200 steps
Smoke test (4600f3c) confirmed 3-fix patch (c6c5f7f) works for Fixes 1, 2, 3: - Fix 1 (--max-completion-length 512): generation no longer clipped - Fix 2 (build_prompt byte-identical to SFT): valid JSON, correct action_type / role / service_name - Fix 3 (Qwen3 EOS): generation terminates at <|im_end|> naturally New issue isolated during smoke: model occasionally emits config_edits as a bare {"key", "value"} dict instead of [{"key", "value"}] list. Pydantic's List[ConfigEdit] rejects non-list, raising "config_edits Input should be a valid list". Diagnosis: SFT data is 60/60 list-typed for config_edits (zero dict exceptions). Pruning the in-prompt SCHEMA + EXAMPLES blocks (commit c6c5f7f) was necessary to fix the OOD problem but removed the only in-prompt cue distinguishing list-of-dict vs bare-dict for this field. The SFT-trained model still mostly emits the list form, but drifts to bare-dict on a fraction of edit_config generations. Option A fix (training/grpo_train.py:parse_compl…aed29b5 -
Phase M smoke test wrapper: 20 steps for fast validation
Smoke test variant of phase_m_grpo_job.py with --max-steps 20 instead of 100. Validates the 3-fix patch (c6c5f7f) before committing to full Stage B 200-step run. Pass criteria (visible in first 5 step metrics): - reward_std > 0.05 (was 0.003 in broken run) - clipped_ratio < 0.2 (was 1.0) - loss > 0 (was ~0) If smoke passes: launch full Stage B via phase_m_grpo_job.py with --max-steps 200. If smoke fails: diagnose 4th issue not yet identified.4600f3c -
Phase M Stage A v2: fix 3 compound bugs preventing GRPO learning
Diagnosis from previous Stage A run (canceled): all 8 generations collapsed to identical fallback action -> reward_std ~0.003 -> no gradient signal -> loss ~0. Three compounding causes: 1. --max-completion-length default 256 caused clipped_ratio=1.0 (all completions hit token cap mid-output). Wrapper didn't pass the flag at all; CLI default kicked in. training/README.md and SATURDAY_PLAYBOOK.md both warn this is the T4 ceiling, 512 is the correct value for L4/H100. 2. GRPO build_prompt was 2.3x longer than SFT prompt (~1400 vs ~600 chars) with extra OUTPUT SCHEMA + 4 EXAMPLE JSON bodies that the SFT model NEVER saw in training. Plus textwrap.dedent + interpolated multi-line service_lines produced mixed-indent output that compounded the OOD problem (SFT prompts are flush-left). 3. parse_completion fallback returned a default view_pipeline action on parse failure -> all 8 candidates identical -> GRPO advantage collapses. (Not changed this commit; fixes 1+…c6c5f7f -
Phase M Stage A: revert wrapper + trainer to 4940a15 (last known-working non-vLLM state)
Path B (vLLM debug, 4 attempts) is concluded — Unsloth GRPO+vLLM expects a single trainable LoRA on the raw base model. We have stacked PEFT adapters (frozen sft_warmup prior + trainable GRPO LoRA on top) which hides Unsloth's load_lora method behind multiple wrapper layers. Rolling wrapper + trainer back to commit 4940a15 — the last commit before vLLM was introduced. This is byte-identical to the code path that ran successfully for 2h before being canceled to chase the vLLM speedup. Differences vs the canceled run are positive (Track-IO wired, --max-steps 100 fits the timeout, env-server log capture from earlier M-prep commits is preserved). Files restored to 4940a15: - scripts/phase_m_grpo_job.py: drop vllm dep, drop peft<0.19 pin, drop --use-vllm flag, drop VLLM_ENFORCE_EAGER env var, drop try/except wrapper around subprocess.run with stale vLLM diagnostic message. - training/grpo_train.py: drop conditional fast_inference / max_lora_rank / gpu_memory_utilization / enforce_ea…bf1c711 -
Phase M Stage A: revert --use-vllm, ship non-vLLM path
After 4 vLLM debug attempts (vllm==0.6.3 pin, enforce_eager, peft<0.19 pin, etc.), hit a fundamental architectural conflict: AttributeError: 'Qwen3ForCausalLM' object has no attribute 'load_lora' Unsloth's GRPO+vLLM colocate path expects a single trainable LoRA on the raw base model — its load_lora method lives on the FastLanguageModel wrapper. We have a stacked setup: PeftModel.from_pretrained loads the frozen sft_warmup adapter as a named prior, then FastLanguageModel.get_peft_model adds the trainable GRPO LoRA on top. After the PeftModel wrap, vLLM can no longer reach Unsloth's load_lora — that method is buried under multiple wrapper layers and the resolved object exposes the underlying HF model directly. Reverting to non-vLLM for Stage A. Other scaffolding kept for record: - enforce_eager env var in subprocess env (no-op without --use-vllm) - peft>=0.18.0,<0.19 pin (safe regardless of vLLM — keeps us off the PEFT 0.19 TP-aware code path that requires unreleased Transformers)…1a604b3 -
Phase M debug Approach 2.5: pin peft>=0.18.0,<0.19 for SFT adapter loading
Approach 2 (enforce_eager) successfully bypassed the Qwen3 vLLM compilation bug. vLLM model loads + KV cache initializes cleanly. New error in PEFT.from_pretrained path during SFT adapter load: ImportError: cannot import name 'EmbeddingParallel' from 'transformers.integrations.tensor_parallel' in peft/utils/save_and_load.py:500 _maybe_shard_state_dict_for_tp Investigation: - PEFT v0.19.0 (released 2026-04-14, just 11 days ago) added the EmbeddingParallel import for tensor-parallel adapter sharding. Earlier versions through v0.18.1 don't have this code path at all. - Transformers v5.5.0+ (released 2026-04-13) added the EmbeddingParallel symbol to integrations.tensor_parallel. Earlier versions don't have it. - Unsloth pins transformers>=4.51.3,...,<=5.5.0 (with skips) and peft>=0.18.0. The UV resolver picked PEFT 0.19.1 (latest allowed) but pulled a Transformers in the 4.x range to satisfy other deps. PEFT's brand-new TP-aware code path then fails on missing symbol.…8d77eb7 -
Phase M debug Approach 2: revert vllm pin + enforce_eager=True
Approach 1 (vllm==0.6.3) created cascading dep mismatch — torchao needed torch.int1 attribute that older torch doesn't have: AttributeError: module 'torch' has no attribute 'int1' in torchao/quantization/quant_primitives.py:191 We never reached vLLM init. Reverted to vllm>=0.6.0 to restore the working dep tree from the previous run (which got past [2/7] cleanly). Then applied actual Approach 2 fix for the original Qwen3 graph-erase compilation bug: RuntimeError: Tried to erase Node size_1 but it still had 2 users in the graph: {getitem_3: None, getitem_4: None}! at vllm/compilation/backends.py:528 _decompose_size_nodes Changes: - scripts/phase_m_grpo_job.py PEP 723 header: vllm==0.6.3 -> vllm>=0.6.0 - scripts/phase_m_grpo_job.py [5/7] subprocess.run env dict: + VLLM_ENFORCE_EAGER=1 (env var path — belt) - training/grpo_train.py FastLanguageModel.from_pretrained fpt_kwargs: + enforce_eager=True when args.use_vllm (suspenders — Unsloth forwards this kwarg to vLLM's LL…c701cb1 -
Phase M Stage A vLLM Approach 1: pin vllm==0.6.3 (avoid v1-engine graph bug)
Stage A relaunch with vLLM hit: RuntimeError: Tried to erase Node size_1 but it still had 2 users in the graph: {getitem_3: None, getitem_4: None}! at vllm/compilation/backends.py:528 _decompose_size_nodes This is the torch._dynamo + vLLM v0.19.x V1-engine + Qwen3 architecture compilation bug. v0.6.x uses the older engine path which doesn't run this graph-decomposition pass at all, so it sidesteps the bug entirely. Trade-off: v0.6.3 is missing some recent perf optimizations (FA3, prefix-caching improvements). Should still beat non-vLLM by ~2-3x for GRPO rollouts on L4. If this also fails: Approach 2 = enforce_eager=True (skips torch.compile). Approach 3 = lower gpu_memory_utilization + max_seq_length. Approach 4 = full version-pin sweep matching Unsloth's reference Qwen3 GRPO notebook. Constraint: Yashash decided to keep Qwen3-1.7B as the base — SFT adapter at yashash045/devops-pipeline-gym-sft-adapter is Qwen3-trained, switching architectures would invalidate the warm-start.2599fd4 -
Phase M Stage A vLLM: enable use_vllm + fast_inference for ~3x speedup
Switching to vLLM-accelerated GRPO. Expected speedup: ~80s/step -> ~25s/step. Stage A 100 steps: ~2.5h -> ~50min. Stage B 200 steps: ~5h -> ~100min. Changes: scripts/phase_m_grpo_job.py: - Added vllm>=0.6.0 to PEP 723 dependencies header - Pass --use-vllm to grpo_train.py - Wrapped GRPO subprocess in try/except CalledProcessError with clear diagnostic message: "vLLM may have failed to initialize (Unsloth + LoRA + vLLM colocate is a known-finicky combo on L4). To fall back to default generation, remove the '--use-vllm' arg from scripts/phase_m_grpo_job.py and relaunch." training/grpo_train.py: - FastLanguageModel.from_pretrained: when args.use_vllm is set, also pass fast_inference=True, max_lora_rank=16 (matches LoRA r below), gpu_memory_utilization=0.6 (leaves ~10GB on L4 for trainer state). - GRPOConfig probe loop: when args.use_vllm is set, also probe vllm_mode="colocate" (in-process vLLM, no external server) and vllm_gpu_memory_utilization=0.6 (matches the from_pr…22623bb -
Phase M Stage A: --max-steps 100, fix Track-IO wiring
Stage A is a 100-step test run to validate the GRPO pipeline + Track-IO logging before the production 200-step Stage B run. The previous 200-step attempt was cancelled because (a) Track-IO was completely silent — the trainer never streamed metrics anywhere, so we'd be training blind; and (b) 200 × ~80s/step would have exceeded the 4h job timeout. Root causes for silent Track-IO: 1. grpo_train.py never set report_to in GRPOConfig, so TRL/transformers defaulted to "none" and the built-in TrackioCallback was never registered. 2. The wrapper set env var TRACKIO_SPACE_NAME, but trackio reads TRACKIO_SPACE_ID. The package code in trackio/utils.py only looks at TRACKIO_SPACE_ID — TRACKIO_SPACE_NAME was a fabricated name. 3. The Track-IO Space yashash045/dpg-trackio existed but was an empty gradio shell (NO_APP_FILE, only README + .gitattributes). Even if 1+2 were fixed, the Space had no app to render the dashboard. Fixes: - training/grpo_train.py: add report_to=["trackio"] to…4940a15 -
Phase M-prep fix: append /final to sft_path in GRPO wrapper
Stage 2 [5/7] failed with PEFT 'Can't find adapter_config.json at /workspace/sft_adapter'. Root cause: Stage 1 SFT wrapper uploaded the entire /workspace/sft_output folder (which contains final/, checkpoint-10/, checkpoint-20/ subdirs) to the Hub. snapshot_download returns the parent folder path. PEFT.from_pretrained needs the path that DIRECTLY contains adapter_config.json, which is the final/ subfolder. Stage 1's own training log explicitly hinted at this: Use in GRPO: python training/grpo_train.py --sft-adapter-path /workspace/sft_output/final ... Changes to scripts/phase_m_grpo_job.py: - After snapshot_download, verify adapter_config.json exists at <sft_path>/final with debug-listing fallback if not found - Pass --sft-adapter-path <sft_path>/final (not just <sft_path>) to grpo_train.py - Updated print statements to clarify the root vs resolved adapter paths3cf8472
