/gridfs/. Files are stored inside MongoDB itself: metadata in the
<bucket>.files collection, content in 255KB chunks in <bucket>.chunks.
Uses pymongo’s native async API.
Install the extra:
Config
READ and WRITE modes are supported. Small files are fine: a file
smaller than the chunk size is stored as a single chunk document, so tiny
config files and large binaries go through the same path.
Fields marked secret are redacted in snapshots.
VFS.md
The VFS.md an agent reads with GridFS mounted at/<path>.
- Rendered
- Raw
mkdir creates a zero-byte key/
marker document so empty directories stay visible.
/gridfs/ exposes:
Revisions
GridFS filenames are not unique: every write uploads a new revision and keeps the old ones. The VFS always reads and lists the newest revision per filename, and snapshot-pinned reads resolve old revisions by their file_id, so time-travel reads work natively. mv retags every revision’s
filename server-side (history moves with the file).
rm destroys nothing: it retags every revision of the file into the trash,
/trash/<filename>, the same way mv does. A mount key never starts with a
slash, so no path on any mount, whatever its key_prefix, can list, stat or
read a trashed revision, and the name reads as gone; a snapshot pinned to one
of those revisions still stats and reads it by its _id. A file mv overwrites goes
to the trash the same way. The trash keeps the whole original filename, so a
revision is traceable to the key it was written under.
keep_revisions bounds the history. Set it, and a write or an rm deletes
the revisions past that count per filename, oldest first, under the file’s
name and under its trash name alike; a snapshot pinned to a pruned revision
then fails to read it. Unset, the default, keeps every revision.
Walks
GridFS metadata lives in an ordinary MongoDB collection, so a walk (find,
du, ls -R, tree, grep -r, rg, cp -r) reads every file under the
start directory with one fs.files query and answers each directory below it
from that answer. See Walks.
Cache
The GridFS VFS usesIndexCacheStore with index_ttl = 600
(10 minutes). Directory listings are cached before being refreshed from
MongoDB, reducing queries for repeated traversals. A configured index: replaces this lifetime with its own ttl, and the mount’s ttl: caps it.
Integration
What GridFS answers through its own API, so fewer files are read. Integration explains each command.Example
Shell Commands
The GridFS VFS supports the same full command surface as the S3 VFS: read commands (cat, head, tail,
grep, rg, jq, wc, stat, find, tree, du, checksums), text
processing (awk, sed, sort, cut, diff, …), file operations
(cp, mv, rm, mkdir, touch, tee, split, …), compression, and
encoding. Range reads (head -c, tail -c) seek chunk-wise, so large
files are never downloaded whole.
Scoping a VFS to a key prefix
Passkey_prefix to GridFSConfig to transparently scope every operation
to a subpath of the bucket, mirroring the S3 key_prefix behavior:
/data/notes.md; the underlying GridFS
filename is users/{user_id}/data/notes.md. Leading slashes are stripped
and a trailing slash is added automatically; None and an empty string
mean “no prefix.”
Use Cases
- Files next to your data: keep agent-visible files in the same MongoDB deployment as the rest of your application state, with one backup story
- Revision-aware storage: writes never destroy old content; snapshots can pin reads to the exact revision they saw
- Cheap find on big buckets: metadata queries replace full listings
- Multi-tenant mounts: one bucket, per-tenant
key_prefixscoping