Open source · in active development

Xet storage on infrastructure you own.

openxet is the open-source server for the Xet protocol, the chunk-deduplicated storage behind Hugging Face. The official clients work against it unmodified. Your bucket, your identity, and only changed chunks move.

Star on GitHub Read the design
one command, one variable
$ openxet serve
  ready · export HF_ENDPOINT=http://127.0.0.1:8080 HF_TOKEN=oxt_…
$ export HF_ENDPOINT=http://127.0.0.1:8080 HF_TOKEN=oxt_…
$ hf upload acme/llm-ft ./checkpoint
  10.0 GB file, 1.0 MB changed since last revision
  ✓ uploaded 1.9 MB · deduplicated 99.98%

The target workflow. openxet serve and the CAS work today; stock hf upload lands with the Hub layer in M2.

How it works

A 1 MB edit to a 10 GB checkpoint moves megabytes, not gigabytes.

Clients split files into content-defined chunks of about 64 KB and hash them. openxet answers which chunks it already stores, across every file and revision, so only new chunks are packed into xorbs and uploaded.

revision 1 · 10 GB, every chunk newuploaded once
revision 2 · 1 MB changedonly the green chunks move
file size
10 GB
bytes changed
1 MB
bytes uploaded (target)
≤ 2 MB

Targets are measured by the benchmark and compatibility suites, with results published per release.

Why openxet

Plug-and-play infrastructure, not a Hugging Face copy.

A small surface with clear seams. It runs on the storage and identity systems you already have and forces no opinions on you.

The tools you already use

hf, huggingface_hub, hf_xet, git-xet and huggingface.js work unmodified. Formats match xet-core byte for byte, in both directions.

Your bucket is the truth

S3, GCS, Azure or local disk. The database is a rebuildable index; lose it and reindex restores everything from the bucket.

Your identity provider

OIDC, webhooks or static tokens. openxet never owns users: no passwords, no profiles, no sign-up database.

Git anywhere

Keep repos on GitHub, GitLab or Gitea and large files in openxet through git-xet. Or host the Git repos in openxet itself.

Laptop to petabytes

One binary with zero-config dev mode. Stateless nodes: scale by adding more, with no redesign.

For humans and agents

Readable output and safe defaults for people. --json with versioned schemas, stable exit codes, structured errors and MCP for agents.

Non-negotiables

Boring to run, by design.

These hold for every change, and each is enforced by a test. A change that weakens one is a release blocker.

  1. Protocol exactnessData written by openxet reads with stock clients and huggingface.co tooling, and the reverse.
  2. The bucket is the source of truthLosing the database never loses data.
  3. Nothing is deleted outside garbage collectionExplicit, logged, with a grace period.
  4. openxet never owns usersEvery token is bound to an external identity.
  5. No telemetryNot by default, and never without explicit opt-in. This site has no tracking either.

openxet.com · coming

Don't want to run anything? We'll host it.

The hosted service runs the released open-source build. There is no private fork and no hosted-only core feature.

  • Landing page to a working hf upload in under 60 seconds
  • Free tier with no time limit and no card to start
  • Bring your own S3, GCS or Azure bucket
  • EU region from day one
  • Dedup savings shown on every page, in bytes and money
  • Support from every page, replies within one business day
Watch the repo for launch Become a design partner

Roadmap

Built in the open, straight through to 1.0.

Every milestone ships behind the same bar: the compatibility suite against real clients, golden vectors and published benchmarks.

  1. M0done

    Foundation

    Protocol crate on xet-core, CI, compatibility harness, spikes.

  2. M1in progress

    CAS core

    Xorbs, shards, global dedup, reconstruction, openxet serve.

  3. M2next

    Hub API and Git

    hf upload and download unmodified, Git hosting, LFS for git-xet.

  4. M3planned

    Production backends

    Postgres, S3, GCS, Azure, OIDC, multi-node.

  5. M4planned

    Operations

    GC, reindex, verify, metering, Git over SSH.

  6. M5planned

    Reach

    Cache tier, pull-through mirror, pools and data residency.

  7. M6planned

    Client and web UI

    sync, mount, MCP server, self-service UI.

  8. M7planned

    Distribution

    Releases, images, Helm, Homebrew, PyPI, docs site.