LFG: Learning to Drive is a Free Gift — Large-Scale Label-Free Autonomy Pretraining from Unposed In-The-Wild Videos

LFG turns ordinary driving video into a unified pseudo-4D representation — 3D point maps, camera poses, semantic layouts, confidence and motion masks — learned from unposed, unlabelled dashcam footage with no human annotation. Given three observed frames it predicts all of these for those frames and for three frames into the future.

⚠️ Research release. This model is released for research purposes under the terms below. It is not a production system and does not include any Applied Intuition proprietary data, product code, or production checkpoints.

Model Details

Developed by	Applied Intuition — AI Research
Model type	Autoregressive transformer over a DINOv2 encoder, with future-frame prediction
Paper	Learning to Drive is a Free Gift: Large-Scale Label-Free Autonomy Pretraining from Unposed In-The-Wild Videos
Conference / venue	CVPR 2026
Code	https://github.com/Applied-Intuition-Open-Source/LFG
Project page	https://lfg-ai.github.io/
License (weights)	CC BY-NC 4.0
Contact	GitHub Issues on the code repo

Checkpoints

Checkpoint	Description	Size	Link
lfg_seg_motion_m3n3.pt	1.22B params. 3 observed frames in, 3 observed + 3 future out. Heads: depth/points, camera pose, confidence, segmentation (7 classes), motion.	4.87 GB	lfg_seg_motion_m3n3.pt

The file holds model_state_dict (inference weights), config (architecture settings) and global_step, and loads with torch.load(..., weights_only=True).

Intended Use & Limitations

Intended use: research on label-free pretraining for driving perception, reproducing the paper's KITTI-360 and Waymo benchmarks, and using the representation as a frozen backbone for downstream tasks.

Out of scope / limitations:

• Point maps are predicted up to one unknown scale and shift. Align them against a metric reference before reporting metric error.
• Trained on forward-facing dashcam video; other camera placements, viewpoints and sensor rigs are untested.
• Segmentation covers seven coarse classes, not a full urban taxonomy.

Commercial use is not permitted under CC BY-NC 4.0.

How to Use

import numpy as np
from PIL import Image
from huggingface_hub import hf_hub_download

from lfg.checkpoint import load_model_from_checkpoint
from lfg.inference import predict_window
from lfg.io import Frame

path = hf_hub_download("AppliedIntuitionResearch/LFG", "lfg_seg_motion_m3n3.pt")
model, config, _, _ = load_model_from_checkpoint(path, device="cuda")

# three consecutive RGB frames from a forward-facing camera
frames = [Frame(rgb=np.asarray(Image.open(p).convert("RGB")), source=p, frame_index=i)
          for i, p in enumerate(["000.png", "001.png", "002.png"])]

out = predict_window(model, frames, config, device="cuda",
                     target_size=518, resize_mode="crop", keep_ratio=False)

print(out["local_points"].shape)   # (1, 6, H, W, 3) — 3 observed + 3 predicted
print(out["segmentation"].shape)   # (1, 6, H, W, 7)

Full installation and usage instructions: see the GitHub repository.

Citation

@inproceedings{strong2026lfg,
  title     = {Learning to Drive is a Free Gift: Large-Scale Label-Free Autonomy Pretraining from Unposed In-The-Wild Videos},
  author    = {Strong, Matthew and Chang, Wei-Jer and Herau, Quentin and Yang, Jiezhi and Hu, Yihan and Peng, Chensheng and Zhan, Wei},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2026}
}

Acknowledgments

This model builds on Pi3 (BSD-3-Clause), whose model code is bundled in the inference repo, and which in turn builds on DINOv2 (Meta Platforms, Apache-2.0). Evaluation baselines use VGGT, Depth Anything 3, SegFormer and MaskFormer, and the benchmarks use KITTI-360 and the Waymo Open Dataset. We thank the authors for open-sourcing their work.