GCS

The GCS VFS mounts a Google Cloud Storage bucket at some prefix such as /gcs/. All operations involve network I/O to the remote object store. Uses aioboto3 against GCS’s S3-compatible XML API via HMAC keys, inheriting all S3 VFS capabilities. For credential setup, see GCS Setup. Install the extra:

pip install "mirage-ai[gcs]"

Config

import os

from mirage import MountMode, Workspace
from mirage.vfs.gcs import GCSConfig, GCSVFS

config = GCSConfig(
    bucket=os.environ["GCS_BUCKET"],
    access_key_id=os.environ["GCS_ACCESS_KEY_ID"],
    secret_access_key=os.environ["GCS_SECRET_ACCESS_KEY"],
    # Optional:
    # endpoint_url="https://storage.googleapis.com",
    # region="auto",
    # timeout=30,
    # proxy="http://proxy:8080",
)
vfs = GCSVFS(config)
ws = Workspace({"/gcs": vfs}, mode=MountMode.READ)

GCSVFS(config) takes a GCSConfig object with the bucket name and HMAC credentials. Both READ and WRITE modes are supported.

Field	Description
`timeout` default `30`	Seconds a request may wait to connect, or between two reads of its answer, before it fails; a download that keeps flowing is never cut off.
`bucket` required	Name of the bucket to mount.
`region` default `'auto'`	Region requests are signed for; GCS serves every region from one host.
`endpoint_url` default `'https://storage.googleapis.com'`	Endpoint URL of the S3 interoperability API.
`access_key_id` secret	Access key ID; set it with `secret_access_key`. Unset uses the AWS credential chain.
`secret_access_key` secret	Secret access key paired with `access_key_id`.
`session_token` secret	Session token for temporary credentials, sent with the access key pair.
`aws_profile`	Named profile in the shared AWS config and credentials files that supplies credentials when no access key pair is given.
`path_style` default `False`	Address the bucket in the URL path (`<endpoint>/<bucket>/<key>`) instead of as a subdomain of the endpoint.
`key_prefix`	Key prefix inside the bucket that the mount treats as its root; keys outside it are not visible. Leading slashes are dropped and a trailing `/` is added.
`proxy` secret	Proxy URL for every HTTP and HTTPS request to the endpoint.

Fields marked secret are redacted in snapshots. For write: conditional on this provider, see S3-compatible stores.

VFS.md

The VFS.md an agent reads with GCS mounted at /<path>.

Rendered
Raw

Virtual Filesystem

Path	Description
`/<path>`	`gcs`, write

Supported Unix Commands for VFS

Unix commands work on each path as on a local filesystem, as far as the path supports them:

Path	Supported
`/<path>`	Almost all Unix commands.

Limitations of Unix Commands for VFS

Where a path differs from a local filesystem:

Path	Limitations
`/<path>`	`grep -r`, `find` and `du` stop after 1000 files; a glob matching more, `tar -c`, `cp -r` and `rm -r` are refused.

Run man <cmd> or <cmd> --help for one command’s usage.

---
name: workspace-filesystem
description: The virtual filesystem layout of this workspace.
---

# Virtual Filesystem

| Path | Description |
| --- | --- |
| `/<path>` | `gcs`, write |

## Supported Unix Commands for VFS

Unix commands work on each path as on a local filesystem, as far as the path supports them:

| Path | Supported |
| --- | --- |
| `/<path>` | Almost all Unix commands. |

## Limitations of Unix Commands for VFS

Where a path differs from a local filesystem:

| Path | Limitations |
| --- | --- |
| `/<path>` | `grep -r`, `find` and `du` stop after 1000 files; a glob matching more, `tar -c`, `cp -r` and `rm -r` are refused. |

Run `man <cmd>` or `<cmd> --help` for one command's usage.

The GCS VFS maps object keys to virtual paths under the mount prefix, identical to the S3 VFS. GCS “directories” are prefix-based — there are no real directory objects. For example, if bucket mirage-ai contains:

data/example.json
data/example.parquet
data/example.jsonl

Then mounting at /gcs/ exposes:

/gcs/
  data/
    example.json
    example.parquet
    example.jsonl

Path mapping: virtual /gcs/data/example.json maps to GCS key data/example.json.

Cache

The GCS VFS uses IndexCacheStore with index_ttl = 600 (10 minutes), same as S3. Directory listings are cached for up to 600 seconds before being refreshed from GCS. This reduces API calls for repeated directory traversals. A configured index: replaces this lifetime with its own ttl, and the mount’s ttl: caps it.

Integration

What GCS answers through its own API, so fewer files are read. Integration explains each command.

Command	On GCS
`md5sum`	The object’s ETag, when it is the MD5: a single-part upload, unencrypted or under SSE-S3
`find`, `du`, `ls -R`, `tree`	One recursive listing

Example

import asyncio
import os

from dotenv import load_dotenv

from mirage import MountMode, Workspace
from mirage.vfs.gcs import GCSConfig, GCSVFS

load_dotenv(".env.development")

config = GCSConfig(
    bucket=os.environ["GCS_BUCKET"],
    access_key_id=os.environ["GCS_ACCESS_KEY_ID"],
    secret_access_key=os.environ["GCS_SECRET_ACCESS_KEY"],
)

vfs = GCSVFS(config)


async def main() -> None:
    ws = Workspace({"/gcs/": vfs}, mode=MountMode.READ)

    r = await ws.shell("ls /gcs/")
    print(await r.stdout_str())

    r = await ws.shell("cat /gcs/data/example.json | head -n 10")
    print(await r.stdout_str())

    r = await ws.shell("tree /gcs/")
    print(await r.stdout_str())

    r = await ws.shell("find /gcs/ -name '*.parquet'")
    print(await r.stdout_str())

    r = await ws.shell("stat /gcs/data/example.json")
    print(await r.stdout_str())


if __name__ == "__main__":
    asyncio.run(main())

Shell Commands

The GCS VFS supports the full set of shell commands since it operates on real file content (text, binary, JSON, CSV, etc.). Large files benefit from range reads to avoid downloading entire objects.

Read Commands

Command	Notes
`cat`	Read file content
`head` / `tail`	First/last N lines
`grep` / `rg`	Pattern search (file or directory level)
`jq`	Query JSON fields
`wc`	Line/word/byte counts
`stat`	File metadata (name, size, type, modified)
`find`	Recursive search with `-name`, `-maxdepth`
`tree`	Directory tree view
`nl`	Number lines
`du`	Disk usage summary
`file`	Detect file type
`strings`	Extract printable strings from binary
`xxd`	Hex dump
`md5`	MD5 checksum
`sha256sum`	SHA-256 checksum

Text Processing

Command	Notes
`awk`	Pattern scanning and processing
`sed`	Stream editor
`tr`	Translate or delete characters
`sort`	Sort lines
`uniq`	Remove duplicate lines
`cut`	Extract fields/columns
`join`	Join lines on a common field
`paste`	Merge lines side by side
`column`	Columnate output
`fold`	Wrap lines to a specified width
`expand`	Convert tabs to spaces
`unexpand`	Convert spaces to tabs
`fmt`	Simple text formatter
`rev`	Reverse lines
`tac`	Concatenate and print in reverse
`look`	Display lines beginning with a given string
`shuf`	Shuffle lines
`tsort`	Topological sort
`comm`	Compare two sorted files
`cmp`	Compare two files byte by byte
`diff`	Compare files line by line
`patch`	Apply a diff patch
`iconv`	Character encoding conversion

File Operations

Command	Notes
`cp`	Copy files
`mv`	Move/rename files
`rm`	Remove files
`mkdir`	Create directories
`touch`	Create empty file or update timestamp
`ln`	Create symbolic links
`tee`	Write stdin to file and stdout
`mktemp`	Create temporary file
`split`	Split file into pieces
`csplit`	Split file by context

Path Utilities

Command	Notes
`basename`	Strip directory from path
`dirname`	Strip filename from path
`realpath`	Resolve path
`readlink`	Print symbolic link target
`ls`	List directory contents

Compression

Command	Notes
`gzip`	Compress files
`gunzip`	Decompress gzip files
`zip`	Create zip archives
`unzip`	Extract zip archives
`tar`	Archive files
`zcat`	Cat compressed files
`zgrep`	Grep compressed files

Encoding

Command	Notes
`base64`	Base64 encode/decode

Use Cases

AI agents accessing GCS data: Mount GCS buckets for agents to read and process datasets
Data pipelines: Read and write GCS objects with shell-like commands
FUSE mounting: Expose GCS buckets through a virtual FUSE mount for external tools

Getting Started

Access

AI Agents

VFS

CLIs

Runtimes

Config

VFS.md

Virtual Filesystem

Supported Unix Commands for VFS

Limitations of Unix Commands for VFS

Cache

Integration

Example

Shell Commands

Read Commands

Text Processing

File Operations

Path Utilities

Compression

Encoding

Use Cases

​Config

​VFS.md

​Virtual Filesystem

​Supported Unix Commands for VFS

​Limitations of Unix Commands for VFS

​Cache

​Integration

​Example

​Shell Commands

​Read Commands

​Text Processing

​File Operations

​Path Utilities

​Compression

​Encoding

​Use Cases

Config

VFS.md

Virtual Filesystem

Supported Unix Commands for VFS

Limitations of Unix Commands for VFS

Cache

Integration

Example

Shell Commands

Read Commands

Text Processing

File Operations

Path Utilities

Compression

Encoding

Use Cases