S3VFS mounts one bucket: every object becomes a file and every / in a key a
folder. It works with Amazon S3 and any S3-compatible server, such as MinIO,
Ceph or SeaweedFS. Set up credentials as in S3 setup.
Install
Configure
endpoint and, if the server
needs it, forcePathStyle: true. A mount takes MountMode.READ, WRITE or
EXEC.
Fields
AWS region requests are signed for and sent to. Unset leaves it to the AWS SDK’s environment variables and profile.
Endpoint URL of an S3-compatible server. Unset, the node mount uses
AWS_ENDPOINT_URL_S3 or AWS_ENDPOINT_URL when set, else AWS’s endpoint for region; an endpoint set only in an AWS profile is not read.Address the bucket in the URL path (
<endpoint>/<bucket>/<key>) instead of as a subdomain of the endpoint.Seconds a request may wait to connect, or between two reads of its answer, before it fails; a download that keeps flowing is never cut off.
Function
(path, operation, options) that returns a signed URL for each S3 operation. When set, the mount fetches those URLs and needs no credentials.Function that returns the
httpAgent and httpsAgent for one S3 client, called once per client. The node mount builds it from proxy when that is set.Content-Type stored with every object the mount writes. Unset sends none, so the store applies its own default; a write through presignedUrlProvider sends application/octet-stream.Key prefix inside the bucket that the mount treats as its root; keys outside it are not visible. Leading slashes are dropped and a trailing
/ is added.Named profile in the shared AWS config and credentials files that supplies credentials when no access key pair is given.
Authentication
Credentials come fromaccessKeyId and secretAccessKey, with sessionToken
for temporary credentials. Without them, the AWS SDK reads its own chain:
environment variables, ~/.aws/credentials, then the instance or container
role.
The mount needs s3:ListBucket on the bucket, s3:GetObject to read, and
s3:PutObject and s3:DeleteObject to write. An object the credentials cannot
read answers Permission denied. Without s3:ListBucket, a missing file
answers Permission denied too, since S3 then hides which keys exist.
Layout
/s3/data/file.txt is the key data/file.txt. Folders come from the / in
keys; S3 stores no folder objects.
Behavior
- Key prefix:
keyPrefix: 'users/42/'mounts only that part of the bucket, so/s3/notes.mdis the keyusers/42/notes.md.
Browser
The browserS3VFS from @struktoai/mirage-browser keeps credentials off the
page: a server signs a URL for each S3 call, and the browser fetches it.
- The bucket’s CORS rules must allow the page’s origin, exactly as served.
- A browser mount cannot declare
write: conditional, and any HTTP 403 from a signed URL, an expired one included, answersPermission denied. keyPrefixset in the browser is no security boundary: enforce the prefix in the signing server and its IAM policy.
Signing server
Signing server
An Express route that signs URLs:Sign only the operations the app needs: a viewer needs
GET, HEAD and LIST.Bucket CORS rules
Bucket CORS rules
Browser fields
Browser fields
AWS region requests are signed for and sent to. Unset leaves it to the AWS SDK’s environment variables and profile.
Endpoint URL of an S3-compatible server. Unset, the node mount uses
AWS_ENDPOINT_URL_S3 or AWS_ENDPOINT_URL when set, else AWS’s endpoint for region; an endpoint set only in an AWS profile is not read.Seconds a request may wait to connect, or between two reads of its answer, before it fails; a download that keeps flowing is never cut off.
Function
(path, operation, options) that returns a signed URL for each S3 operation. When set, the mount fetches those URLs and needs no credentials.Content-Type stored with every object the mount writes. Unset sends none, so the store applies its own default; a write through presignedUrlProvider sends application/octet-stream.Key prefix inside the bucket that the mount treats as its root; keys outside it are not visible. Leading slashes are dropped and a trailing
/ is added.presignedUrlProvider, the browser S3VFS signs requests itself with
the Node fields’ credentials, which puts the keys in the page.Integration
S3 answers these commands through its own API, so fewer objects are read. See Integration.md5sum: the object’s ETag, when it is the MD5 (a single-part upload, unencrypted or under SSE-S3).find: one recursive listing.du: one recursive listing.ls -Randtree: one recursive listing.
VFS.md
The VFS.md an agent reads with S3 mounted at/<path>.
<mode> is the mount’s mode: read, write or exec. The commands shown are
those of a write mount.
- Rendered
- Raw
Virtual Filesystem
Supported Unix Commands for VFSUnix commands work on each path as on a local filesystem, as far as the path supports them:
Limitations of Unix Commands for VFSWhere a path differs from a local filesystem:
Run
man <cmd> or <cmd> --help for one command’s usage.