Competitive matrix

Where openxet stands against other Xet-protocol servers, re-evaluated every minor release (design §16). Every entry states how it was verified. Unverified claims are marked as such.

2026-10-23: initial evaluation

Method:

• a code audit of each project
• a hands-on run of every project against stock clients: huggingface_hub 2.2.0, hf_xet 1.7.0, git-xet 0.2.1, git-lfs 3.5.1, git 2.39.5. Each run follows the same protocol of 18 checks (setup, hf and Python API, Xet transfer, Range, dedup, Git, crash, authorization, storage, throughput, code, maintenance), each graded with the labels in the legend below, with the command and its output as evidence.

openxet has no server release yet, so this records the bar to beat and the mistakes to avoid.

Legend: PASS, FAIL, PARTIAL as observed; N/A the project doesn't claim the feature; NOT RUN not covered by that run; Info recorded for context, not graded.

Authorization is not graded or published for any project.

Shardline: observed against stock clients

ggoggam/openxet: observed against stock clients

Tested fc8cd642 (2026-10-01), native cargo build --release, filesystem backend, SQLite index. Clients: hf_xet 1.7.0 and huggingface_hub 2.2.0; hf_xet 1.5.2 and 1.6.0 to find which versions work. It is a CAS-only server, so checks that need a Hub API, Git or LFS are N/A. openxet is not affiliated with ggoggam/openxet; the names collide.

The protocol core is the strongest of the projects evaluated: HF's own crates, real reference tests, lean storage, crash safety and speed. It is a bare CAS, not a Hub, and is about one hf_xet release behind.

Security, disclosed with the maintainer's agreement. We reported these privately on 2026-10-23. The maintainer reports all of them fixed on main at ccc3591 (merged 2026-10-23) and agreed to public mention. At fc8cd642 we found:

• With auth on and the filesystem backend, GET /v1/xorbs/default/{hash} served a xorb without a credential [observed].
• Any valid OIDC token mapped to write access on all data, so one user could read another user's files [observed].
• Any writer could run POST /v1/gc with a zero grace period [observed].

The maintainer's fix:

• Xorb reads need a signed, range-bound, expiring URL or an admin token.
• OIDC roles map to read, write and admin scopes, and a file is readable only by its owners and admins.
• GC is admin-only with a minimum 1-hour grace period.
• Shard registration now requires verification entries.

Our re-test of ccc3591 is pending; this note changes to observed once it runs.

kebyn/xet-server: observed against stock clients

Tested e9c4624f (2026-10-10) in a Linux container (rust:1.94.1-bookworm), README configuration and default limits. Clients: huggingface_hub 2.2.0, hf_xet 1.7.0, git 2.39.5, git-lfs 3.5.1, git-xet 0.2.1.

The Hub layer is careful work: clean errors, snapshot commits, crash recovery. The Xet layer isn't wire-compatible with stock hf_xet 1.7.0 in either direction.

guilt/xet-server and Taytay's branches: observed against stock clients

Tested guilt main 22c4aa1 (v1.1.0) and Taytay's gc branch dc82516, which contains every other Taytay branch (dedup-*, multi-client-tests, git-lfs-bridge, signed-xorb-urls). Native go build (Go 1.27.1). Clients: huggingface_hub 2.2.0, hf_xet 1.7.0, git 2.39.5, git-lfs 3.5.1, git-xet 0.2.1. Taytay was re-run on checks 5, 8 and 10, with spot checks of 2, 6 and 9; "–" means not re-run.

xet-proxyd (guilt main's HF pull-through cache), run against huggingface.co on a public repo: hf download of hf-internal-testing/tiny-random-gpt2 works with matching sha256, but all three Xet files were relayed live (file not cacheable (partial xorb citation)), and after a restart with an unreachable upstream even config.json returned 404, so offline serving did not hold for this repo (the README hedges that claim).

guilt main moves large files over real Xet quickly and with little memory, with auth off and on one machine. Taytay's branches fix exactly what they target: cross-machine dedup, Xet downloads with auth on and a working git-xet push.

Cross-project comparison

Rows are the hands-on protocol's checks, with the lesson each one tests. Shardline's column comes from its run above; NOT RUN marks checks that run didn't cover. Labels follow the legend at the top. Row 5 grades function only: stock hf_xet downloads with no extra headers. Row 8 is graded on bytes sent from a fresh cache for a 1 MB edit. Maintenance is our judgement, not a measurement: Active (several maintainers, recent code), Single-maintainer (one author, code changes within 60 days), or Inactive (one author, no code change for over 60 days).

* kebyn's kill landed during Hub staging, before the CAS write, so the run didn't exercise a crash inside the CAS.

2026-10-24: openxet at M1

openxet's first measured column, at the end of milestone M1. M1 is the storage-only CAS, so the checks that need the Hub API, Git or LFS name the milestone that brings them instead of a grade, and a check that only part of M1 can answer is PARTIAL. Every grade is measured, by hand or by a test that runs in CI.

Method: release build of openxet serve on local disk, dev mode (auth on), Apple M3 laptop with 8 cores, macOS. Stock hf_xet 1.7.0 through its XetSession API, each client run with a fresh Xet cache unless noted. Rows 6, 11, 15 and 16 come from three runs of scripts/measure-storage-only.py --size-mib 1024 (1 GiB of os.urandom, so incompressible), which prints each figure below; the others cite the test that asserts them.

Maintenance (check 18) is a judgement we don't make about ourselves.

Lessons we turn into tests

Each item becomes a case in our compatibility suite or a unit test. The plans reference this list.

1. preupload must route large files to Xet; hf upload of a ≥ 1 GiB file must use the Xet path. Check this via hf_xet activity, not just success, with a file name that matches no .gitattributes pattern, so the test proves routing by size (S3).
2. whoami-v2 returns every field huggingface_hub reads, including orgs.
3. resolve returns X-Xet-Hash, X-Repo-Commit, X-Linked-Size, X-Linked-Etag and an xet-auth link, with absolute redirect URLs.
4. Xet transfer URLs work for stock hf_xet with no extra headers: signed, self-authorizing URLs.
5. Range reads return only the requested range, at chunk granularity, on both the reconstruction API and the HTTP gateway.
6. Cross-machine global dedup:
   • A second machine with an empty cache uploading a 1 MB-modified copy transfers ≈ 1 MB.
   • Dedup responses are shards with a footer and HMAC-keyed hashes.
   • They cover every xorb of the files containing the chunk.
7. LFS batch responses use exactly X-Xet-Cas-Url, X-Xet-Access-Token, X-Xet-Token-Expiration (a real epoch), and absolute hrefs. ref is optional.
8. The git-xet push and git-lfs basic download round trip, and plain git-lfs push and pull, all work against an openxet-hosted repo.
9. Incompressible data: xorbs up to the protocol maximum (64 MiB plus framing) are accepted. Body limits derive from protocol limits, never below them.
10. A crash (SIGKILL) at any point in an upload never blocks re-uploading the same content.
11. verify (our fsck) has zero false positives on healthy data and a non-zero exit on real corruption.
12. Repo creation is authorized per namespace; scoped tokens can't create elsewhere. Unauthorized requests get a clean 403 that hf renders well.
13. No per-chunk object duplication: storage overhead over unique compressed data stays ≤ 5%, measured.
14. From a cold machine to the first successful hf upload of a ≥ 1 GiB file, following only the README, in under 2 minutes, measured per release (design §1a).

Added from the 2026-10-23 hands-on runs of ggoggam, kebyn, guilt and Taytay. Lesson 5 is reconfirmed: in two projects a 1 KB range fetched ~67 MB.

15. Global dedup answers at GET /v1/chunks/default/<hash>, the path stock hf_xet 1.5.2 and 1.7.0 send, and also at the spec's default-merkledb. Lesson 6's second-machine test runs against both. (ggoggam routes only default-merkledb, so cross-machine dedup never worked with a real client.)
16. Shard upload is served at POST /v2/shards and POST /v1/shards; uploads pass with hf_xet 1.5.2 and 1.7.0. (ggoggam answers /v2/shards with 405 and every upload from hf_xet ≥ 1.6.0 fails.)
17. Xorbs are accepted without a footer, exactly as stock hf_xet sends them, covered by a reference-file round trip. (kebyn rejects them.)
18. POST /api/validate-yaml answers, and every 4xx carries a JSON body. Tested by an hf upload of a folder containing README.md. (kebyn and guilt fail it.)
19. Commits from stock huggingface_hub work beyond the first. One test each:
    • Two consecutive hf uploads to the same repo both succeed. (kebyn allows one commit per repo.)
    • A commit with parentCommit set to the current head is accepted, and commit IDs are 40-hex. (kebyn's 64-hex IDs are rejected by the client.)
    • create_commit with a delete removes the file from the next tree. (guilt answers 200 and keeps the file.)
    • create_branch starts the branch from its base revision, so tree on the new branch equals tree on the base. (guilt starts it empty.)
20. The LFS batch never answers transfer: xet to a request that didn't offer xet, and every object in a response has actions or an error. A test pushes with plain git-lfs and checks the object arrives. (In guilt, git-lfs reads an action-less object as already stored, so the push exits 0 having uploaded nothing.)
21. An acknowledged commit survives a SIGKILL immediately after the acknowledgement. (guilt loses commits within its 1-minute snapshot window.)
22. A shard that references xorbs the server doesn't hold is rejected at upload with a clear error, so the client re-sends the xorbs and no file is registered that can't be reconstructed.
23. Compatibility tests drive the real, pinned client binaries, never a replay of their requests, across a version matrix (oldest supported, pinned, latest). A 404 or 405 seen by a newer client fails the suite. (ggoggam's replayed tests likely hid both of its fatal path bugs.)
24. Large transfers use idle timeouts, not whole-transfer timeouts, and signed URLs and tokens outlive client retries: a 1 GiB upload at 10 MB/s with one retry succeeds. (kebyn's 1 GiB upload hit a 504, then a 401 on retry.)