Skip to main content
The GridFS VFS mounts a MongoDB GridFS bucket at some prefix such as /gridfs/. Files are stored inside MongoDB itself: metadata in the <bucket>.files collection, content in 255KB chunks in <bucket>.chunks. Uses pymongo’s native async API. Install the extra:

Config

Both READ and WRITE modes are supported. Small files are fine: a file smaller than the chunk size is stored as a single chunk document, so tiny config files and large binaries go through the same path. Fields marked secret are redacted in snapshots.

VFS.md

The VFS.md an agent reads with GridFS mounted at /<path>.

Virtual Filesystem

Supported Unix Commands for VFS

Unix commands work on each path as on a local filesystem, as far as the path supports them:

Limitations of Unix Commands for VFS

Where a path differs from a local filesystem:Run man <cmd> or <cmd> --help for one command’s usage.
GridFS filenames are used as slash-separated keys, exactly like S3 object keys. Directories are prefix-based; mkdir creates a zero-byte key/ marker document so empty directories stay visible.
mounted at /gridfs/ exposes:

Revisions

GridFS filenames are not unique: every write uploads a new revision and keeps the old ones. The VFS always reads and lists the newest revision per filename, and snapshot-pinned reads resolve old revisions by their file _id, so time-travel reads work natively. mv retags every revision’s filename server-side (history moves with the file). rm destroys nothing: it retags every revision of the file into the trash, /trash/<filename>, the same way mv does. A mount key never starts with a slash, so no path on any mount, whatever its key_prefix, can list, stat or read a trashed revision, and the name reads as gone; a snapshot pinned to one of those revisions still stats and reads it by its _id. A file mv overwrites goes to the trash the same way. The trash keeps the whole original filename, so a revision is traceable to the key it was written under. keep_revisions bounds the history. Set it, and a write or an rm deletes the revisions past that count per filename, oldest first, under the file’s name and under its trash name alike; a snapshot pinned to a pruned revision then fails to read it. Unset, the default, keeps every revision.

Walks

GridFS metadata lives in an ordinary MongoDB collection, so a walk (find, du, ls -R, tree, grep -r, rg, cp -r) reads every file under the start directory with one fs.files query and answers each directory below it from that answer. See Walks.

Cache

The GridFS VFS uses IndexCacheStore with index_ttl = 600 (10 minutes). Directory listings are cached before being refreshed from MongoDB, reducing queries for repeated traversals. A configured index: replaces this lifetime with its own ttl, and the mount’s ttl: caps it.

Integration

What GridFS answers through its own API, so fewer files are read. Integration explains each command.

Example

Shell Commands

The GridFS VFS supports the same full command surface as the S3 VFS: read commands (cat, head, tail, grep, rg, jq, wc, stat, find, tree, du, checksums), text processing (awk, sed, sort, cut, diff, …), file operations (cp, mv, rm, mkdir, touch, tee, split, …), compression, and encoding. Range reads (head -c, tail -c) seek chunk-wise, so large files are never downloaded whole.

Scoping a VFS to a key prefix

Pass key_prefix to GridFSConfig to transparently scope every operation to a subpath of the bucket, mirroring the S3 key_prefix behavior:
Agents see clean paths like /data/notes.md; the underlying GridFS filename is users/{user_id}/data/notes.md. Leading slashes are stripped and a trailing slash is added automatically; None and an empty string mean “no prefix.”

Use Cases

  • Files next to your data: keep agent-visible files in the same MongoDB deployment as the rest of your application state, with one backup story
  • Revision-aware storage: writes never destroy old content; snapshots can pin reads to the exact revision they saw
  • Cheap find on big buckets: metadata queries replace full listings
  • Multi-tenant mounts: one bucket, per-tenant key_prefix scoping