HfBucketsVFS mounts one
Hugging Face Storage Bucket:
every file in it becomes a file and every / in its path a folder. Set up the bucket and token as in HF Buckets setup.
Install
opendal is a native binding, so the mount runs in Node only, not in the
browser.
Configure
bucket is the bucket’s id, <owner>/<name>. endpoint points the mount at
a mirror or a self-hosted Hub. A mount takes MountMode.READ, WRITE or
EXEC.
Fields
Hub origin requests go to, for a mirror or a self-hosted Hub.
Seconds a request may wait to connect, or between two reads of its answer, before it fails; a download that keeps flowing is never cut off.
Authentication
token is a Hugging Face access token: Read for a read mount, Write to
upload. A fine-grained token needs read access, or write access to upload, on
the bucket’s namespace. A bucket the token cannot reach, or one that does not
exist, answers Permission denied.
Layout
/hf/logs/train.log is the file logs/train.log in the bucket. Folders come
from the / in file paths.
Behavior
- Key prefix:
keyPrefix: 'users/42/'mounts only that part of the bucket, so/hf/notes.mdis the fileusers/42/notes.md. A leading/is dropped and a trailing/added. - Folders: a folder exists only while it holds a file, so
mkdircreates nothing. - No move:
mvandrmdirare not supported. - No history: a bucket keeps no history, so a removed file is gone for good.
Integration
HF Buckets answers these commands through its own API, so fewer files are read. See Integration.find: one recursive listing.du: one recursive listing.ls -Randtree: one recursive listing.
VFS.md
The VFS.md an agent reads with HF Buckets mounted at/<path>.
<mode> is the mount’s mode: read, write or exec. The commands shown are
those of a write mount.
- Rendered
- Raw
Virtual Filesystem
Supported Unix Commands for VFSUnix commands work on each path as on a local filesystem, as far as the path supports them:
Limitations of Unix Commands for VFSWhere a path differs from a local filesystem:
Run
man <cmd> or <cmd> --help for one command’s usage.