Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

xet:// resource addresses

A xet:// address names one resource on one openxet deployment: a repository, a typed view of it, a bucket, a revision, or a file or folder inside one. This page is the contract that every openxet parser, formatter and client follows (ADR 0009, M2-URI). The shared implementation is the openxet-uri crate (crates/uri); its golden vectors (crates/uri/testdata/vectors.json) are part of this specification, and a test fails when the crate and the vectors disagree.

Status, as of openxet M2:

  • The grammar, the parser and the canonical formatter are implemented in openxet-uri. No released command takes a xet:// address yet; the CLI, SDK and UI adopt it in M6 (M6-URI). Until then, use the released commands and HTTPS links.
  • HTTPS stays the canonical link to share in a browser, and every stock client (hf, huggingface_hub, git) keeps using HTTPS. Adding xet:// changes nothing they recognise.
  • xet is not registered with IANA. The URI scheme registry had no xet entry when checked on 2026-10-25, and openxet has not yet filed a registration request under RFC 7595. The name was used before by XetHub's deprecated tools; see Legacy forms and the legacy audit.
  • A xet:// address is an openxet resource address, not a name for the Xet protocol. Chunking, hashes, xorbs, shards and the CAS API are the Xet specification's, unchanged.

Shape

xet://{host}[:{port}]/[{kind}/]{namespace}/{name}[@{revision}][/{path}]
xet://hub.example.org/team/corpus
xet://hub.example.org/datasets/team/corpus@v2/train/part-0001.parquet
xet://hub.example.org/team/model@refs/pr/3/config.json
xet://hub.example.org/team/model@feature%2Fsplit/README.md
xet://hub.example.org/team/model@0f9a6e1c8d2b4a7f5e3c1b9d8a7f6e5d4c3b2a19/weights.bin
xet://127.0.0.1:8080/dev/scratch/notes.txt
xet://[2001:db8::7]:8443/team/corpus

The host is always written: an address means the same thing for everyone who reads it, and never falls back to a reader's default server.

Grammar

ABNF (RFC 5234), using ALPHA, DIGIT, HEXDIG, IPv4address, IPv6address, pct-encoded, unreserved and sub-delims from RFC 3986. A xet:// address is an RFC 3986 URI with an authority, no userinfo, no query and no fragment.

xet-uri      = scheme "://" authority "/" resource
scheme       = "xet"                            ; case-insensitive, written lowercase
authority    = host [ ":" port ]                ; no userinfo: an "@" here is refused
host         = "[" IPv6address "]" / IPv4address / reg-name
reg-name     = "localhost" / label 1*( "." label )
label        = lchar [ *( lchar / "-" ) lchar ] ; at most 63 octets
lchar        = ALPHA / DIGIT / "_"
port         = %x31-39 *4DIGIT                  ; 1-65535, no leading zero

resource     = [ kind "/" ] namespace "/" name [ "@" revision ] [ "/" path ]
kind         = "datasets" / "spaces" / "kernels" / "buckets"
namespace    = id
name         = id
id           = 1*( unreserved / pct-encoded )      ; decoded: M2-R2's rule, below

revision     = special-ref / rev-segment
special-ref  = "refs/pr/" 1*DIGIT
             / "refs/convert/" 1*( ALPHA / DIGIT / "_" / "." / "-" )
rev-segment  = 1*( rchar / pct-encoded / ucschar )
rchar        = unreserved / sub-delims / ":" / "@"

path         = segment *( "/" segment )
segment      = 1*( pchar / ucschar )
pchar        = unreserved / pct-encoded / sub-delims / ":" / "@"
ucschar      = <any Unicode scalar value above U+007F, encoded as UTF-8>

The grammar is read left to right: the authority ends at the first /; the first path segment is a kind when it is one of the four kind names; the next two are the namespace and the name; the name ends at its first @, which starts the revision; the revision ends at the next / (after a special-ref, at the end of the ref); everything after that / is the path.

Each rule also carries a constraint the ABNF can't express:

PartConstraintError code
schemexet, in any casenot_xet_uri
authorityno @ (userinfo); the refusal never repeats what was before the @credentials_in_uri
hostpresent; IPv4 in strict dotted-decimal form (no leading zeros, four parts); a name whose last label is all digits is not a namemissing_host, invalid_host
hosta name has two labels or more, or is localhosthost_not_qualified, legacy_no_host
hostASCII only: an internationalised name is written as its A-label (xn--…); the error names the A-labelnon_ascii_host
hostIPv6 without a zone identifier (%25…) and no IPvFutureinvalid_host
hostnot xethub.com or a subdomain of itlegacy_xethub
port1-65535, decimal, no leading zero. A non-numeric "port" is pyxet's explicit-endpoint form only when it is a valid namespace and a repository follows (see Legacy forms); anything else is a malformed portinvalid_port, legacy_pyxet_address
kindexactly one of the four names, written literally in lowercase. The first segment is matched after percent-decoding: models, model, dataset, space, kernel and bucket there are refused, never read as a namespace, and so is a percent-encoded spelling of any kind word (dataset%73)unknown_kind
namespace, nameafter percent-decoding, [A-Za-z0-9][A-Za-z0-9._-]{0,95} with no .. (ruling M2-R2); a name does not end in .git. An escaped unreserved character is accepted and written plainly in the canonical form (%63orpus is corpus)invalid_namespace, invalid_name, missing_repository
namespacenot reserved (ruling M2-R2), ignoring ASCII case: models, datasets, spaces, kernels, buckets, api, v1, v2, openapi.json, healthz, readyzreserved_namespace
revisionnot on a bucket; not empty; a valid Git ref name or object id after decoding (below)revision_on_bucket, empty_revision, invalid_revision
revisiona bare refs followed by a path is refused: a ref with / is written with %2Fambiguous_revision
pathno empty segment (so no trailing / and no //), no . or .. segment, no encoded / (%2F) and no NUL (%00)invalid_path
everywhere% is followed by two hex digits, and decoded octets are UTF-8invalid_percent_encoding
everywhereno space, control character, ", <, >, \, ^, `, {, `, }, [or]` outside the host
everywhereno ? (query) and no # (fragment); the refusal never repeats the queryquery_not_supported, fragment_not_supported

A revision is valid when it is a Git ref name under the rules of git check-ref-format (one-level names such as main included): no control character, space, ~, ^, :, ?, *, [ or \; no .. and no @{; not @ alone; no leading, trailing or doubled /; no component starting with . or ending in .lock; not ending in .. A 40- or 64-character hexadecimal object id passes the same rules.

Every error has a stable code, a message that says what is wrong and a hint that says how to write the address instead.

Resource kinds

The kind says which view of the deployment's storage the address goes through (ADR 0010, design §6):

WrittenKindMeansResolves from
(none)Repositorythe repository {namespace}/{name}, whatever its labelsM2
datasets/Datasetthe repository through its dataset view; a repository without the dataset label answers as not foundM2
spaces/Spacethe space viewM2
kernels/Kernelthe kernel viewM2
buckets/Bucketthe bucket primitive {namespace}/{name}; buckets have no revisionsM5

There is no models/ kind. A repository is addressed by {namespace}/{name} alone, so the address stays valid when its labels change. On huggingface.co, hf://models/… is the model view; in a xet:// address the first segment models is refused with a hint rather than read as a namespace, as are the singular forms (dataset/ and the rest), which hf:// refuses too. The plural kind words, and the first segments of openxet's own routes, are reserved namespaces (ruling M2-R2, ignoring case), so no repository can take a name that an address would read as something else.

Bucket addresses parse and format in M2 so that one parser serves every milestone; resolving them needs the bucket primitive, which ships in M5.

Mutable and immutable references

  • No revision names the repository's default branch, main (ruling M2-R5), as it is when the address is used. It is mutable.
  • A branch or tag (@main, @v2, @refs/pr/3) is mutable: the same address can name different content tomorrow.
  • A full object id, 40 or 64 lowercase hexadecimal characters, is immutable: the address names one commit's content for as long as the commit exists. XetUri::is_immutable reports this. An abbreviated id is read as a ref name, never expanded.
  • A bucket address is always mutable: buckets have no history.

Revisions are opaque to the parser apart from the rules above: it never looks a ref up, expands it, or changes its case.

Revisions containing / are written with %2F (@feature%2Fsplit), because the first / after the revision starts the path. The two special ref families of huggingface.co, refs/pr/{n} and refs/convert/{name}, are written as they are (@refs/pr/3/config.json), matching hf:// (huggingface_hub 2.2.0, _hf_uris.py). Any other refs/… ref is written with %2F; @refs/heads/main/x is refused as ambiguous rather than read as the branch refs and the path heads/main/x.

Namespaces and names can't contain @ (ruling M2-R2), so the first @ in the name segment always starts the revision. A %40 there decodes to @ and is refused as an invalid name. In a revision or a path, @ needs no escaping.

Escaping and Unicode

  • Namespaces, names, revisions and paths are compared and stored decoded. The parser decodes %XX (either case), and the formatter writes every octet outside unreserved (A-Z a-z 0-9 - . _ ~) as %XX with uppercase hex. / between path segments, and inside a special ref, is written as is.
  • Paths and revisions are UTF-8. Raw non-ASCII characters are accepted on input, as in an IRI (RFC 3987 §3.1), and the canonical form percent-encodes their UTF-8 octets.
  • No Unicode normalization is applied. é written as U+00E9 and as e + U+0301 are different paths, because Git compares path bytes. Percent-decoded octets that are not UTF-8 are refused.
  • The canonical form also lowercases the scheme and the host, writes an IPv6 address as the WHATWG URL standard serializes it (the RFC 5952 compression, with an IPv4-mapped address in hex: [::ffff:1.2.3.4] is [::ffff:102:304]), so the host equals the HTTPS URL's, and drops port 443, the default. Nothing else is rewritten: the namespace, name, revision and path keep their case.

Parsing is a function of the text alone. format(parse(s)) is the canonical form of s, parse(format(u)) == u for every valid u, and formatting is idempotent; property tests and a fuzz target check all three (crates/uri/tests/props.rs, fuzz/fuzz_targets/xet_uri.rs).

Hosts and ports

  • The host is the deployment's DNS name or IP address, exactly as clients reach it. Two addresses name the same deployment only if their hosts are equal after lowercasing and their ports are equal; no DNS lookup or alias is consulted.
  • A single-label name such as storage is refused: it resolves through the reader's search domains, which makes it a local alias that names a different server on another network. Write the fully qualified name. localhost is the one single-label name allowed.
  • IPv4 must be four decimal parts without leading zeros (127.0.0.1, not 127.1 or 0x7f.0.0.1), because other parsers read those forms differently.
  • IPv6 is written in brackets: xet://[::1]:8080/…. Zone identifiers are refused: they name an interface of the reader's machine.
  • The port is optional. Without one, the address names port 443.

Local development

openxet serve listens on http://127.0.0.1:8080 by default, and a local address names it directly:

xet://127.0.0.1:8080/dev/scratch
xet://localhost:8080/dev/scratch/notes.txt

Loopback hosts (localhost, 127.0.0.0/8, [::1]) are reached over plain HTTP unless the client is configured otherwise; every other host is reached over HTTPS. An address without a port names port 443 over plain HTTP too: xet://localhost/… is http://localhost:443/…, so a local server's address carries its port. Such an address is meaningful only on the machine that runs the server: share an HTTPS link or a public host's address instead.

The HTTPS mapping

Each address maps to an HTTPS URL on the same host and port (XetUri::https), the routes huggingface_hub uses (REPO_TYPES_URL_PREFIXES: datasets/, spaces/, kernels/, none for the unprefixed form):

AddressHTTPS URL
xet://h/{ns}/{name}https://h/{ns}/{name}
xet://h/datasets/{ns}/{name}https://h/datasets/{ns}/{name}
xet://h/{ns}/{name}@{rev}https://h/{ns}/{name}/tree/{rev}
xet://h/{ns}/{name}/{path}https://h/{ns}/{name}/resolve/main/{path}
xet://h/{ns}/{name}@{rev}/{path}https://h/{ns}/{name}/resolve/{rev}/{path}
xet://h/buckets/{ns}/{name}https://h/buckets/{ns}/{name}
xet://h/buckets/{ns}/{name}/{path}https://h/buckets/{ns}/{name}/resolve/{path}

The revision is percent-encoded as one segment (/ as %2F), except a special ref, which is written as is, as huggingface_hub's to_url does. A file path maps to resolve, the download route openxet serves from M2 (Task 13), not to a web viewer.

The unprefixed resolve route is the model view (design §6), so in M2 a file address without a kind resolves on repositories that carry the model label, and a file of a repository with only another label is addressed through that label's kind (xet://h/datasets/…). The unprefixed Git URL serves every repository whatever its labels. M6 revisits the mapping of a kind-less file address, so that it resolves whatever the repository's labels.

Reverse-proxy prefixes

A deployment served under a base path, such as https://example.org/hub/, can't be addressed in M2: the first segment after the authority is always a kind or a namespace, so there is no room for a prefix, and guessing one would let the same text name different resources. A client configured with an endpoint that has a path refuses it (base_path_unsupported) and says to serve openxet at the root of its own host name. M6 revisits base paths together with endpoint discovery.

Credentials and endpoints

A xet:// address never carries a credential:

  • xet://user:token@host/… is refused (credentials_in_uri), and the error does not repeat the userinfo. There is no query string either, so no ?token=.
  • A client selects the credential for an address by the address's exact host and port, from the endpoints it is configured with (openxet_uri::Endpoints). A default endpoint's credential is never sent to another host: an address naming a host the client has no endpoint for is fetched without a credential.
  • A redirect keeps the credential only when it stays on the same origin (scheme, host and port). A redirect to another host, or from HTTPS to HTTP, drops it, even when the client holds a credential for the new host: a server can't choose which of the client's credentials to receive. A Location with userinfo is refused.
  • A credential is configured only for an HTTPS endpoint, or for a plain HTTP endpoint on a loopback host (credential_over_http otherwise), so it never crosses a network unencrypted.
  • Opening an address identifies or reads a resource. It never uploads, deletes or runs anything; a write needs its command's own authorization.

These rules are tested at the library level in crates/uri/tests/endpoints.rs (endpoints_isolate_credentials): two endpoint fixtures hold the same names with different content and different credentials, and each address reaches only its own host with only its own credential.

Legacy forms

XetHub's deprecated tools used xet:// with other grammars: Git-Xet (git-xet 0.14.5) read the host as a user name on the server chosen by XET_ENDPOINT (default xethub.com), and pyxet (0.2.0-rc1) wrote the endpoint before a colon (xet://{endpoint}:{user}/{repo}/{branch}/{path}) and defaulted to xethub.com without one. Both projects are deprecated since XetHub joined Hugging Face, and the legacy audit records the sources, versions and registrations this policy is based on.

The policy, tested row by row in crates/uri/tests/legacy.rs (legacy_matrix), is:

  • A form that names no host is refused: it pointed at whichever server the reader had configured, and openxet never sends an address to a default server.
  • A form that names xethub.com is refused: no openxet deployment answers there, and the data's new home can't be inferred.
  • pyxet's explicit-endpoint form names a host, so it is translated to the equivalent address on the same host and reported. The parser never applies the translation silently: classify_legacy returns it, and parsing fails with the translated address in its hint.
  • An address that is valid under the current grammar is read with the current meaning, even where a legacy tool read it differently; only the host it names is ever contacted.
#InputOriginOutcome
L1xet://hub.example.org/team/corpus@main/a.txtcurrent grammarAccepted
L2xet://xethub.com/user/repo/branch/hello/worldGit-Xet (tests), pyxet (.com rule)Rejected, legacy_xethub
L3xet://xethub.com:user/repo/main/data.csvpyxet explicit endpointRejected, legacy_xethub
L4xet://www.xethub.com/user/repoXetHub subdomainRejected, legacy_xethub
L5xet://user/repo/branch/hello/worldGit-Xet and pyxet without a hostRejected, legacy_no_host; hint xet://<host>/user/repo@branch/hello/world
L6xet://user/repo/branch/Git-Xet and pyxet without a hostRejected, legacy_no_host; hint xet://<host>/user/repo@branch
L7xet://user/repoGit-Xet and pyxet without a hostRejected, legacy_no_host; hint xet://<host>/user/repo
L8xet://userGit-Xet and pyxet without a hostRejected, legacy_no_host
L9xet://:user/repo/mainpyxet with an empty endpointRejected, legacy_no_host
L10xet://hub.example.org:user/repo/main/a/b.csvpyxet explicit endpointTranslated to xet://hub.example.org/user/repo@main/a/b.csv
L11xet://localhost:8080:user/repo/main/xpyxet explicit endpoint with a portTranslated to xet://localhost:8080/user/repo@main/x
L12xet://hub.example.org:user/repo/main/dir/pyxet, folder with trailing /Translated to xet://hub.example.org/user/repo@main/dir (the / is dropped and the note says so)
L13xet://hub.example.org:user/repopyxet explicit endpoint, no branchTranslated to xet://hub.example.org/user/repo
L14xet://hub.example.org:userpyxet, user onlyRejected, invalid_port (no repository follows, so it isn't read as pyxet)
L15xet://storage:user/repo/mainpyxet with a single-label endpointRejected, host_not_qualified
L16/user/repo/branch/pathpyxet without a schemeRejected, not_xet_uri
L17https://xethub.com/user/repo/branch/pathpyxet HTTPS formRejected, not_xet_uri
L18xet://example.com/user/repo/branch/pathGit-Xet read example.com as a user; pyxet as an endpoint with branch branchAccepted, current meaning: host example.com, repository user/repo, path branch/path
L19xet://hub.example.org:8080/user/repo/mainpyxet would read 8080 as a userAccepted, current meaning: port 8080, repository user/repo, path main
L20xet://localhost/user/repoGit-Xet would read localhost as a userAccepted, current meaning: host localhost

fsspec

pyxet registers its fsspec filesystem for the protocol xet when it is imported, with clobber=True, fsspec's option to take over a name that is already registered. fsspec 2026.9.0 has no built-in xet entry. openxet's own fsspec adapter arrives in M6 and will not silently replace another installed xet handler (ADR 0009 item 5); its coexistence policy is part of M6-URI.

Registration and coordination

Neither an IANA registration nor any agreement with Hugging Face or the former XetHub maintainers has happened. The work that remains, tracked under M6-URI and M7 (design §12):

  • request provisional registration of xet under RFC 7595 §7, with this page as the specification, once a released client implements it;
  • tell the deprecated projects' maintainers (xetdata/xet-core, xetdata/pyxet) about the legacy policy before the first release that parses xet://.

Read this page as Markdown · Report a docs issue