# Artifacts

## Introduction

An **artifact** is a versioned, named bundle of files attached to a project — a trained model, a dataset, a set of evaluation outputs. Logging files as artifacts (rather than leaving them on disk) gives you:

- **Versioning** — each time you log under the same name, Trackio records a new version (`v0`, `v1`, ...). Identical content is de-duplicated, so re-logging unchanged files reuses the existing version.
- **Aliases** — moving pointers such as `latest` (assigned automatically) or your own (`prod`, `best`) that always resolve to a single version.
- **Lineage** — artifacts are linked to the runs that produced and consumed them, so you can trace which run created a model and which runs used it.

Artifacts work fully offline against local storage, and sync to a Hugging Face Space (or self-hosted server) when one is configured, exactly like metrics.

## Logging an artifact

The quickest way is to log a file or directory directly. This creates a new artifact, adds the path, and records it as an output of the current run:

```python
import trackio

trackio.init(project="my_project")

# ... training ...

trackio.log_artifact("checkpoints/model.safetensors", type="model")
```

When you pass a path, the artifact `name` defaults to `run-<run_id>-<basename>` and `type` defaults to `"unspecified"`. Pass `name` and `type` to set them explicitly:

```python
trackio.log_artifact("checkpoints/", name="my-model", type="model")
```

### Artifact types

Artifact `type` is a free-form category string. Trackio does not enforce a fixed list of types, but common conventions are:

| Type | Use it for |
| --- | --- |
| `"model"` | Model checkpoints, adapter weights, tokenizers, model configs |
| `"dataset"` | Training, validation, test, or generated datasets |
| `"evaluation"` | Evaluation outputs, predictions, benchmark results, score files |
| `"report"` | Figures, plots, tables, notebooks, or human-readable analysis bundles |
| `"unspecified"` | The default when no type is provided |

Use stable type names within a project so you can filter artifacts consistently and use `trackio.use_artifact(..., type="...")` to assert that a run is consuming the expected kind of artifact.

### Building an artifact explicitly

For finer control — multiple files, custom logical paths, a description, or metadata — construct an [Artifact](/docs/trackio/v0.34.1/en/api#trackio.Artifact), add files to it, then log it:

```python
artifact = trackio.Artifact(
    name="my-model",
    type="model",
    description="Fine-tuned on the v2 dataset",
    metadata={"base_model": "bert-base", "epochs": 3},
)
artifact.add_file("checkpoints/model.safetensors")
artifact.add_file("checkpoints/config.json")
artifact.add_dir("tokenizer/")

logged = trackio.log_artifact(artifact)
```

After logging, the artifact is frozen and its `version`, `aliases`, `size`, and `manifest` are populated:

```python
print(logged.version)   # "v0"
print(logged.aliases)   # ("latest",)
```

### Versions and aliases

Each log under an existing name creates the next integer version and moves the `latest` alias onto it. Assign your own aliases with `aliases=`:

```python
trackio.log_artifact(artifact, aliases=["prod", "best"])
```

Aliases are moving pointers: logging a new version with the same alias re-points it. Version specifiers (`v0`, `v1`, ...) are reserved and cannot be used as aliases.

If you log content identical to an existing version, Trackio reuses that version instead of creating a new one. Any explicit `aliases` you pass still rotate onto it, but `latest` is left where it is — re-logging identical (or older) content never moves it backward.

## Using an artifact

[use_artifact()](/docs/trackio/v0.34.1/en/api#trackio.use_artifact) fetches an artifact and records it as an **input** to the current run, which is what builds lineage between a consuming run and the artifact it used:

```python
trackio.init(project="my_project")

artifact = trackio.use_artifact("my-model")          # resolves to :latest
artifact = trackio.use_artifact("my-model:v2")       # pin a version
artifact = trackio.use_artifact("my-model:prod")     # resolve an alias
```

Pass `type` to assert the artifact's type, raising if it doesn't match:

```python
artifact = trackio.use_artifact("my-model", type="model")
```

### Downloading files

`use_artifact` returns an [Artifact](/docs/trackio/v0.34.1/en/api#trackio.Artifact) whose files you materialize with `download()`:

```python
artifact = trackio.use_artifact("my-model:latest")
path = artifact.download()
# files are now under ./.trackio/artifact-downloads/my_project/my-model_v2/
```

By default, files are written to `./.trackio/artifact-downloads/<project>/<name>_v<version>/`, keyed by project so same-named artifacts from different projects never collide; pass `root` to choose another directory. `download()` is idempotent — files already present are skipped — and when the run is backed by a Space, any file missing locally is fetched from the remote.

## Referencing external data

Some data is too large to copy, or already lives in durable storage you don't want duplicated. `add_reference` records such data by **URI** in the manifest *without staging any bytes* into Trackio's storage, so you still get versioning, de-duplication, aliases, and lineage over data that never moves:

```python
artifact = trackio.Artifact(name="training-set", type="dataset")
artifact.add_reference("file:///mnt/nvme/datasets/corpus.parquet")
artifact.add_reference("file:///mnt/nvme/datasets/shards/")            # a directory — expands to one entry per file
artifact.add_reference("https://example.com/data/eval.json")
artifact.add_reference("hf://datasets/username/my-dataset/train.parquet")
trackio.log_artifact(artifact)
```

Built-in schemes are `file://`, `http(s)://`, and `hf://` — all served by Trackio's existing dependencies. Any other scheme (`s3://`, `gs://`, Azure blob URLs, ...) is supported by registering a custom handler; see [Referencing other stores](#referencing-other-stores-s3-gcs-azure) below for complete S3, GCS, and Azure examples. With `checksum=True` (the default), Trackio probes the source to fill in `size` and a checksum:

- `file://` is stream-hashed to a sha256; its size is read from disk.
- `hf://` uses the LFS sha256 (or the git blob id); `http(s)://` uses the server `ETag`.

Every checksum is stored in a single `digest` field — the sha256 for local/LFS references, or the provider's ETag for HTTP or object-store references.

A URI that denotes a directory (`file://`) or a repo path (`hf://`) is **expanded** into one reference entry per object, each named by its path relative to the prefix (nested under `name=` when given). `max_objects` (default 10,000) caps the expansion and raises if exceeded.

When no checksum can be obtained — for example a server with no `ETag` — the reference is still recorded, using **the URI itself as the `digest`**, and a warning is emitted. A local `file://` path that doesn't exist is the exception: it raises rather than being recorded, since a missing local file is almost always a mistake — use `add_file` to stage a local file. Pass `checksum=False` to skip the per-object checksum — a single object then records just its URI — and `name=` to set the logical path.

A reference's version identity comes from its URI, size, and `digest`, so re-logging identical references de-duplicates to one version, while a changed URI, size, or checksum creates a new one — exactly like file content. When the URI is used as the `digest`, identity therefore tracks the URI and not the content.

URIs are recorded verbatim, and manifests can be synced to a Hugging Face Dataset. A URI that embeds a credential — a presigned `https://` URL or an Azure SAS token — would expose that credential to anyone who can read the Dataset, so `add_reference` warns when a URI's query string looks signed; prefer the object's canonical unsigned URI (e.g. `s3://bucket/key`).

Reference bytes are never uploaded to a Space or bucket, but they can be fetched on demand. `download()` downloads each referenced object into the directory (via `httpx`, `huggingface_hub`, or your registered handler), which can transfer a large amount of data. Before fetching, each object's source is re-probed and its current checksum compared against the recorded one — a mismatch warns that the source may have changed but the download proceeds. Each object is written atomically, and one already present on disk is skipped without re-checking the source, so repeated calls are cheap and idempotent:

```python
artifact = trackio.use_artifact("training-set:latest")
path = artifact.download()   # downloads staged files AND referenced objects
```

To work with the pointers *without* downloading, read them from the artifact instead:

```python
print(artifact.references)                  # [{'path': ..., 'ref': 's3://...', 'size': ..., 'digest': ...}, ...]
print(artifact.get_entry_uri("eval.json"))  # "s3://my-bucket/eval.json"
```

An unfetchable reference (a missing local file, an unreachable URL, or a scheme with no registered handler) raises when `download()` reaches it.

## Referencing other stores (S3, GCS, Azure)

Trackio deliberately ships no cloud-vendor SDKs. To reference objects in a store it doesn't handle natively, subclass `trackio.ReferenceHandler` and register an instance with `trackio.register_reference_handler`. A handler implements four methods:

- `matches(scheme, uri)` — whether this handler owns the URI. Registered handlers are consulted *before* the built-ins, so a handler may also claim specific `https://` hosts (as the Azure example below does) ahead of the generic HTTP handler.
- `resolve(uri, checksum, max_objects)` — probe the source and return a list of `trackio.ResolvedReference` entries: one entry with `relkey=None` for a single object, or one entry per object (with `relkey` set to the object's path relative to the prefix) when the URI denotes a prefix. Set `size` and `digest` when the source can be probed (the provider's ETag works fine as a digest — Trackio treats reference digests as opaque tokens); return them as `None` to record the reference un-checksummed, in which case Trackio falls back to the URI as the digest and warns.
- `fetch(uri, dest)` — download the single object at `uri` to the local path `dest` (called by `Artifact.download()`).
- `hint()` — a one-line actionable message shown in the warning when `resolve` produced no checksum.

Keys discovered during prefix expansion should be percent-encoded when synthesizing each entry's `uri` (and decoded in `fetch`), since manifest URIs may not contain spaces.

### S3 (`s3://`, via `boto3`)

```python
from urllib.parse import quote, unquote, urlsplit

import boto3

import trackio

class S3Handler(trackio.ReferenceHandler):
    def matches(self, scheme, uri):
        return scheme == "s3"

    def resolve(self, uri, checksum, max_objects):
        parts = urlsplit(uri)
        bucket, key = parts.netloc, unquote(parts.path.lstrip("/"))
        client = boto3.client("s3")
        if key and not key.endswith("/"):
            head = client.head_object(Bucket=bucket, Key=key)
            return [
                trackio.ResolvedReference(
                    relkey=None,
                    uri=uri,
                    size=int(head["ContentLength"]),
                    digest=head["ETag"].strip('"') if checksum else None,
                )
            ]
        base = key[: key.rfind("/") + 1]
        entries = []
        pages = client.get_paginator("list_objects_v2").paginate(
            Bucket=bucket, Prefix=key
        )
        for page in pages:
            for obj in page.get("Contents", []):
                if obj["Key"].endswith("/"):
                    continue
                if len(entries) >= max_objects:
                    raise ValueError(
                        f"{uri} expands to more than {max_objects} objects; "
                        "pass max_objects= to raise the limit."
                    )
                entries.append(
                    trackio.ResolvedReference(
                        relkey=obj["Key"][len(base):],
                        uri=f"s3://{bucket}/{quote(obj['Key'], safe='/')}",
                        size=int(obj["Size"]),
                        digest=obj["ETag"].strip('"') if checksum else None,
                    )
                )
        return entries

    def fetch(self, uri, dest):
        parts = urlsplit(uri)
        with open(dest, "wb") as f:
            boto3.client("s3").download_fileobj(
                parts.netloc, unquote(parts.path.lstrip("/")), f
            )

    def hint(self):
        return "Configure AWS credentials so the object can be probed for an ETag."

trackio.register_reference_handler(S3Handler())

artifact = trackio.Artifact(name="training-set", type="dataset")
artifact.add_reference("s3://my-bucket/shards/")   # expands to one entry per object
trackio.log_artifact(artifact)
```

### Google Cloud Storage (`gs://`, via `google-cloud-storage`)

```python
from urllib.parse import quote, unquote, urlsplit

from google.cloud import storage

import trackio

class GcsHandler(trackio.ReferenceHandler):
    def matches(self, scheme, uri):
        return scheme == "gs"

    def resolve(self, uri, checksum, max_objects):
        parts = urlsplit(uri)
        bucket, key = parts.netloc, unquote(parts.path.lstrip("/"))
        client = storage.Client()
        if key and not key.endswith("/"):
            blob = client.bucket(bucket).get_blob(key)
            if blob is None:
                raise ValueError(f"No such GCS object: {uri}")
            return [
                trackio.ResolvedReference(
                    relkey=None,
                    uri=uri,
                    size=blob.size,
                    digest=blob.etag.strip('"') if checksum else None,
                )
            ]
        base = key[: key.rfind("/") + 1]
        entries = []
        for blob in client.list_blobs(bucket, prefix=key):
            if blob.name.endswith("/"):
                continue
            if len(entries) >= max_objects:
                raise ValueError(
                    f"{uri} expands to more than {max_objects} objects; "
                    "pass max_objects= to raise the limit."
                )
            entries.append(
                trackio.ResolvedReference(
                    relkey=blob.name[len(base):],
                    uri=f"gs://{bucket}/{quote(blob.name, safe='/')}",
                    size=blob.size,
                    digest=blob.etag.strip('"') if checksum else None,
                )
            )
        return entries

    def fetch(self, uri, dest):
        parts = urlsplit(uri)
        key = unquote(parts.path.lstrip("/"))
        storage.Client().bucket(parts.netloc).blob(key).download_to_filename(
            str(dest)
        )

    def hint(self):
        return "Configure GCP credentials so the object can be probed for an ETag."

trackio.register_reference_handler(GcsHandler())
```

### Azure Blob Storage (via `azure-storage-blob` + `azure-identity`)

Azure blobs are addressed by plain `https://` URLs (`https://<account>.blob.core.windows.net/<container>/<blob>`), so this handler claims them by host suffix — registering it makes it win over Trackio's generic HTTP handler:

```python
from urllib.parse import quote, unquote, urlsplit

from azure.identity import DefaultAzureCredential
from azure.storage.blob import BlobServiceClient

import trackio

class AzureHandler(trackio.ReferenceHandler):
    def matches(self, scheme, uri):
        return scheme in ("http", "https") and urlsplit(
            uri
        ).netloc.lower().endswith(".blob.core.windows.net")

    def _split(self, uri):
        parts = urlsplit(uri)
        container, _, blob_name = parts.path.lstrip("/").partition("/")
        return f"{parts.scheme}://{parts.netloc}", container, unquote(blob_name)

    def _service(self, account_url):
        return BlobServiceClient(account_url, credential=DefaultAzureCredential())

    def resolve(self, uri, checksum, max_objects):
        account_url, container, blob_name = self._split(uri)
        service = self._service(account_url)
        if blob_name and not blob_name.endswith("/"):
            props = service.get_blob_client(container, blob_name).get_blob_properties()
            return [
                trackio.ResolvedReference(
                    relkey=None,
                    uri=uri,
                    size=props.size,
                    digest=props.etag.strip('"') if checksum else None,
                )
            ]
        base = blob_name[: blob_name.rfind("/") + 1]
        entries = []
        container_client = service.get_container_client(container)
        for blob in container_client.list_blobs(name_starts_with=blob_name):
            if blob.name.endswith("/"):
                continue
            if len(entries) >= max_objects:
                raise ValueError(
                    f"{uri} expands to more than {max_objects} objects; "
                    "pass max_objects= to raise the limit."
                )
            entries.append(
                trackio.ResolvedReference(
                    relkey=blob.name[len(base):],
                    uri=f"{account_url}/{container}/{quote(blob.name, safe='/')}",
                    size=blob.size,
                    digest=blob.etag.strip('"') if checksum else None,
                )
            )
        return entries

    def fetch(self, uri, dest):
        account_url, container, blob_name = self._split(uri)
        downloader = (
            self._service(account_url)
            .get_blob_client(container, blob_name)
            .download_blob()
        )
        with open(dest, "wb") as f:
            downloader.readinto(f)

    def hint(self):
        return "Configure Azure credentials so the blob can be probed for an ETag."

trackio.register_reference_handler(AzureHandler())
```

These handlers raise when the source can't be probed; if you'd rather record an un-checksummed reference in that case (for example, when cataloging data behind auth you can't reach at log time), catch the provider error inside `resolve` and return `[trackio.ResolvedReference(relkey=None, uri=uri)]` — Trackio will fall back to the URI as the digest and warn, using your `hint()`.

Register handlers in every process that adds *or downloads* such references: `Artifact.download()` needs the handler to fetch the bytes, and a URI whose scheme has no registered handler raises.

## Browsing artifacts in the dashboard

The dashboard's **Artifacts** tab (shown once a project has artifacts) lists artifacts grouped by type, with each version's aliases, size, and file count. Selecting a version shows its file manifest — click a file to download it — plus its metadata and lineage: the run that produced it and the runs that used it, each linking to the run's detail page.

Each version also has a **Lineage** section showing an interactive graph of the version's lineage: artifact versions and runs as nodes, with arrows from a run to the artifacts it logged and from an artifact to the runs that used it. The selected version is highlighted. To keep large graphs readable, five or more similar nodes with identical connections (for example, the many checkpoints a single run logged) are grouped into a single **cluster** node showing the count — click the cluster to search its members and extract individual ones back into the graph. Drag to pan, use Ctrl/Cmd + scroll (or the +/− buttons) to zoom, and click any node to open a summary with a link to that run or artifact version.

Run detail pages show the same lineage from the other direction, with **Output artifacts** and **Input artifacts** sections linking back to the Artifacts tab.

## Remote storage

When your run targets a Hugging Face Space or self-hosted server (see [Track](track)), artifact files are content-addressed and uploaded once: Trackio skips blobs the server already has, so re-logging shared files is cheap. Artifact metadata and blobs are persisted alongside your other run data to the configured HF Dataset or bucket.