README.md

---
license: other
pipeline_tag: other
tags:
- 3d-tracking
- video-understanding
- 4d-reconstruction
- computer-vision
---

# Track4World: Feedforward World-centric Dense 3D Tracking of All Pixels

Track4World is a feedforward model for efficient holistic 3D tracking of every pixel in a world-centric coordinate system from a monocular video. Built on a global 3D scene representation, Track4World applies a novel 3D correlation scheme to simultaneously estimate the pixel-wise 2D and 3D dense flow between arbitrary frame pairs.

*   **Paper:** [Track4World: Feedforward World-centric Dense 3D Tracking of All Pixels](https://huggingface.co/papers/2603.02573)
*   **Project Page:** [jiah-cloud.github.io/Track4World](https://jiah-cloud.github.io/Track4World.github.io/)
*   **Repository:** [GitHub Repository](https://github.com/TencentARC/Track4World)

---

### 🖼️ Framework

Track4World estimates dense 3D scene flow of every pixel between arbitrary frame pairs from a monocular video in a global feedforward manner, enabling efficient and dense 3D tracking of every pixel in the world-centric coordinate system.

---

## ⚙️ Setup and Installation

```bash
# Clone the repository with submodules
git clone --recursive https://github.com/TencentARC/Track4World.git
cd Track4World

# Create and activate environment
conda create -n track4world python=3.11
conda activate track4world

# Install PyTorch
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu121

# Install dependencies
pip install -r requirements.txt
```

Please refer to the [official GitHub README](https://github.com/TencentARC/Track4World) for detailed instructions on installing third-party modules and downloading weights.

---

## 🚀 Sample Usage

You can perform tracking and reconstruction on the provided demo video using the following commands:

### First Frame 3D Tracking (`3d_ff`)

```bash
python demo.py \
    --mp4_path demo_data/cat.mp4 \
    --mode 3d_ff \
    --Ts -1 \
    --save_base_dir results/cat
```

### Dense Tracking: Every Pixel, Every Frame (`3d_efep`)

```bash
python demo.py \
    --mp4_path demo_data/cat.mp4 \
    --coordinate world_depthanythingv3 \
    --mode 3d_efep \
    --Ts -1 \
    --ckpt_init checkpoints/track4world_da3.pth \
    --save_base_dir results/cat
```

---

## Citation

If you find Track4World useful for your research, please cite:

```bibtex
@article{lu2026track4world,
  title   = {Track4World: Feedforward World-Centric Dense 3D Tracking of All Pixels},
  author  = {Jiahao Lu and Jiayi Xu and Wenbo Hu and Ruijie Zhu and Chengfeng Zhao and Sai-Kit Yeung and Ying Shan and Yuan Liu},
  journal = {arXiv preprint arXiv:2603.02573},
  year    = {2026}
}
```