---
license: mit
task_categories:
- question-answering
size_categories:
- n<1K
---

# MMVP Benchmark Datacard

## Basic Information

**Title:** MMVP Benchmark

**Description:** The MMVP (Multimodal Visual Patterns) Benchmark focuses on identifying “CLIP-blind pairs” – images that are perceived as similar by CLIP despite having clear visual differences. MMVP benchmarks the performance of state-of-the-art systems, including GPT-4V, across nine basic visual patterns. It highlights the challenges these systems face in answering straightforward questions, often leading to incorrect responses and hallucinated explanations.

## Dataset Details

- **Content Types:** Images (CLIP-blind pairs)
- **Volume:** 300 images
- **Source of Data:** Derived from ImageNet-1k and LAION-Aesthetics
- **Data Collection Method:** Identification of CLIP-blind pairs through comparative analysis
