HF Buckets

HfBucketsVFS mounts a Hugging Face Bucket at a prefix such as /hf/. The TypeScript backend mirrors the Python one and returns identical results. For credential setup, see HF Buckets Setup.

Node only. The backend uses the native opendal binding, which does not run in the browser.

Node

pnpm add @struktoai/mirage-node opendal

import { HfBucketsVFS, MountMode, Workspace } from '@struktoai/mirage-node'

const bucket = new HfBucketsVFS({
  bucket: process.env.HF_BUCKET_NAME!, // "namespace/bucket-name"
  token: process.env.HF_TOKEN,
  // Optional:
  // endpoint: 'https://huggingface.co',
  // keyPrefix: 'data/',
})

const ws = new Workspace({ '/hf/': bucket }, { mode: MountMode.READ })

console.log((await ws.shell('ls /hf/')).stdoutText)
console.log((await ws.shell('tree /hf/')).stdoutText)
console.log((await ws.shell("find /hf/ -name '*.json'")).stdoutText)
console.log((await ws.shell('stat /hf/data/file.txt')).stdoutText)

HfBucketsVFS takes the bucket in namespace/bucket-name form plus an optional access token. Both READ and WRITE modes are supported out of the box.

Config

Field	Description
`bucket` required	Bucket id in `namespace/name` form.
`token` secret	Hugging Face access token.
`endpoint` default `'https://huggingface.co'`	Hub origin requests go to, for a mirror or a self-hosted Hub.
`timeout` default `DEFAULT_TIMEOUT`	Seconds a request may wait to connect, or between two reads of its answer, before it fails; a download that keeps flowing is never cut off.
`keyPrefix`	Path inside the bucket that becomes the mount root; files outside it are not listed.

Fields marked secret are redacted in snapshots.

VFS.md

The VFS.md an agent reads with HF Buckets mounted at /<path>.

Rendered
Raw

Virtual Filesystem

Path	Description
`/<path>`	`hf_buckets`, write

Supported Unix Commands for VFS

Unix commands work on each path as on a local filesystem, as far as the path supports them:

Path	Supported
`/<path>`	Almost all Unix commands.

Limitations of Unix Commands for VFS

Where a path differs from a local filesystem:

Path	Limitations
`/<path>`	Not supported: `mv`, `rmdir`. `grep -r`, `find` and `du` stop after 1000 files; a glob matching more, `tar -c`, `cp -r` and `rm -r` are refused.

Run man <cmd> or <cmd> --help for one command’s usage.

---
name: workspace-filesystem
description: The virtual filesystem layout of this workspace.
---

# Virtual Filesystem

| Path | Description |
| --- | --- |
| `/<path>` | `hf_buckets`, write |

## Supported Unix Commands for VFS

Unix commands work on each path as on a local filesystem, as far as the path supports them:

| Path | Supported |
| --- | --- |
| `/<path>` | Almost all Unix commands. |

## Limitations of Unix Commands for VFS

Where a path differs from a local filesystem:

| Path | Limitations |
| --- | --- |
| `/<path>` | Not supported: `mv`, `rmdir`. `grep -r`, `find` and `du` stop after 1000 files; a glob matching more, `tar -c`, `cp -r` and `rm -r` are refused. |

Run `man <cmd>` or `<cmd> --help` for one command's usage.

Bucket object keys map to virtual paths under the mount prefix: virtual /hf/data/file.txt maps to bucket key data/file.txt.

Cache

Directory listings are cached with indexTtl = 600 (10 minutes) and populate file-size/type entries that stat reads via a fast path, so a readdir followed by per-entry stat calls (what ls and most shell commands trigger) costs one HTTP request instead of N. A configured index: replaces this lifetime with its own ttl, and the mount’s ttl: caps it.

Integration

What Hugging Face buckets answers through its own API, so fewer files are read. Integration explains each command.

Command	On Hugging Face buckets
`find`, `du`, `ls -R`, `tree`	One recursive listing

Shell Commands

The same command set as the Python backend, operating on real file content. Large files benefit from range reads to avoid downloading entire objects.

Read Commands

Command	Notes
`cat`	Read file content
`head` / `tail`	First/last N lines
`grep` / `rg`	Pattern search (file or directory level)
`jq`	Query JSON fields
`wc`	Line/word/byte counts
`stat`	File metadata (name, size, type, modified)
`find`	Recursive search with `-name`, `-maxdepth`
`tree`	Directory tree view
`nl`	Number lines
`du`	Disk usage summary
`file`	Detect file type
`strings`	Extract printable strings from binary
`xxd`	Hex dump
`md5`	MD5 checksum
`sha256sum`	SHA-256 checksum

Text Processing

awk, sed, tr, sort, uniq, cut, join, paste, column, fold, expand, unexpand, fmt, rev, tac, look, shuf, tsort, comm, cmp, diff, iconv, base64.

File Operations

rm (-r removes a subtree), mkdir (a no-op: a directory exists only while it holds a file), touch, mktemp, split, csplit.

Path Utilities

ls, basename, dirname, realpath, readlink.

Compression

gzip, gunzip, zcat, zgrep, tar, zip, unzip.

Getting Started

Access

AI Agents

VFS

CLIs

Runtimes

Node

Config

VFS.md

Virtual Filesystem

Supported Unix Commands for VFS

Limitations of Unix Commands for VFS

Cache

Integration

Shell Commands

Read Commands

Text Processing

File Operations

Path Utilities

Compression

​Node

​Config

​VFS.md

​Virtual Filesystem

​Supported Unix Commands for VFS

​Limitations of Unix Commands for VFS

​Cache

​Integration

​Shell Commands

​Read Commands

​Text Processing

​File Operations

​Path Utilities

​Compression

Node

Config

VFS.md

Virtual Filesystem

Supported Unix Commands for VFS

Limitations of Unix Commands for VFS

Cache

Integration

Shell Commands

Read Commands

Text Processing

File Operations

Path Utilities

Compression