> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mirage.strukto.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Google Cloud Storage

> Virtualize a Google Cloud Storage bucket from Node or the browser.

`GCSVFS` mounts one Google Cloud Storage bucket: every object becomes a file and
every `/` in a key a folder. It reaches GCS through the S3-compatible XML API at
`https://storage.googleapis.com`, signed with an HMAC key. Set up credentials as in [GCS setup](https://docs.mirage.strukto.ai/home/setup/gcs.md).

## Install

```bash theme={null}
npm install @struktoai/mirage-node
```

## Configure

```ts theme={null}
import { GCSVFS, MountMode, Workspace } from '@struktoai/mirage-node'

const gcs = new GCSVFS({
  bucket: process.env.GCS_BUCKET!,
  accessKeyId: process.env.GCS_ACCESS_KEY_ID!,
  secretAccessKey: process.env.GCS_SECRET_ACCESS_KEY!,
})
const ws = new Workspace({ '/gcs': gcs }, { mode: MountMode.WRITE })

const result = await ws.shell("ls /gcs && find /gcs -name '*.csv'")
console.log(result.stdoutText)
await ws.close()
```

`endpoint` defaults to `https://storage.googleapis.com` and `region` to `auto`:
one host serves every region. A mount takes `MountMode.READ`, `WRITE` or `EXEC`.

### Fields

**bucket** `string` *required*

Name of the bucket to mount.

**accessKeyId** `string`

Access key ID; set it with `secretAccessKey`. Unset uses the AWS SDK credential chain.

**secretAccessKey** `string`

Secret access key paired with `accessKeyId`.

**sessionToken** `string`

Session token for temporary credentials, sent with the access key pair.

**profile** `string`

Named profile in the shared AWS config and credentials files that supplies credentials when no access key pair is given.

**forcePathStyle** `boolean`

Address the bucket in the URL path (`<endpoint>/<bucket>/<key>`) instead of as a subdomain of the endpoint.

**keyPrefix** `string`

Key prefix inside the bucket that the mount treats as its root; keys outside it are not visible. Leading slashes are dropped and a trailing `/` is added.

**timeout** `number` (default: DEFAULT\_TIMEOUT)

Seconds a request may wait to connect, or between two reads of its answer, before it fails; a download that keeps flowing is never cut off.

**proxy** `string`

Proxy URL for every HTTP and HTTPS request to the endpoint.

**region** `string` (default: 'auto')

Region requests are signed for; GCS serves every region from one host.

**endpoint** `string` (default: '[https://storage.googleapis.com](https://storage.googleapis.com)')

Endpoint URL of the S3 interoperability API.

Fields marked secret are redacted in snapshots.

## Authentication

Credentials come from `accessKeyId` and `secretAccessKey`, a service account's
HMAC key, with `sessionToken` for temporary credentials. Without them, the AWS
SDK reads its own chain: environment variables, `~/.aws/credentials`, then the
instance or container role.

The service account needs **Storage Object Viewer** on the bucket, or **Storage
Object User** (or **Storage Object Admin**) to write. A new key can take up to a
minute to work. A request the credentials may not make answers
`Permission denied`.

## Layout

```text theme={null}
/gcs/
  data/
    config.json
    file.txt
  reports/
    q1.csv
    q2.csv
```

`/gcs/data/file.txt` is the key `data/file.txt`. Folders come from the `/` in
keys; GCS stores no folder objects.

## Behavior

* **Key prefix:** `keyPrefix: 'users/42/'` mounts only that part of the bucket,
  so `/gcs/notes.md` is the key `users/42/notes.md`.

## Browser

The browser `GCSVFS` from `@struktoai/mirage-browser` keeps credentials off the
page: a server signs a URL for each S3 call with the GCS keys, and the browser
fetches it.

```bash theme={null}
npm install @struktoai/mirage-browser
```

```ts theme={null}
import { GCSVFS, MountMode, Workspace } from '@struktoai/mirage-browser'

const gcs = new GCSVFS({
  bucket: 'my-bucket',
  presignedUrlProvider: async (path, op, opts) => {
    const r = await fetch('/presign', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ path, op, opts }),
    })
    return ((await r.json()) as { url: string }).url
  },
})
const ws = new Workspace({ '/gcs': gcs }, { mode: MountMode.READ })
```

* The bucket's CORS rules must allow the page's origin, exactly as served.
* A browser mount cannot declare `write: conditional`, and any HTTP 403 from a
  signed URL, an expired one included, answers `Permission denied`.
* `keyPrefix` set in the browser is no security boundary: enforce the prefix in
  the signing server.
* `endpoint` only names the mount's storage location: requests go to the host
  each signed URL names.

> [accordion] Signing server
>
> An Express route that signs URLs:
>
> ```ts theme={null}
> import {
>   CopyObjectCommand,
>   DeleteObjectCommand,
>   GetObjectCommand,
>   HeadObjectCommand,
>   ListObjectsV2Command,
>   PutObjectCommand,
>   S3Client,
> } from '@aws-sdk/client-s3'
> import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
>
> const client = new S3Client({
>   region: 'auto',
>   endpoint: 'https://storage.googleapis.com',
>   credentials: {
>     accessKeyId: process.env.GCS_ACCESS_KEY_ID!,
>     secretAccessKey: process.env.GCS_SECRET_ACCESS_KEY!,
>   },
> })
> const BUCKET = process.env.GCS_BUCKET!
>
> app.post('/presign', async (req, res) => {
>   const { path, op, opts } = req.body as {
>     path: string
>     op: 'GET' | 'PUT' | 'HEAD' | 'DELETE' | 'LIST' | 'COPY'
>     opts?: Record<string, unknown>
>   }
>   const key = path.replace(/^\/+/, '')
>   const ttl = typeof opts?.ttlSec === 'number' ? opts.ttlSec : 300
>   let cmd
>   switch (op) {
>     case 'GET':    cmd = new GetObjectCommand({ Bucket: BUCKET, Key: key }); break
>     case 'PUT':    cmd = new PutObjectCommand({ Bucket: BUCKET, Key: key, ContentType: opts?.contentType as string | undefined }); break
>     case 'HEAD':   cmd = new HeadObjectCommand({ Bucket: BUCKET, Key: key }); break
>     case 'DELETE': cmd = new DeleteObjectCommand({ Bucket: BUCKET, Key: key }); break
>     case 'LIST':   cmd = new ListObjectsV2Command({
>       Bucket: BUCKET,
>       Prefix: opts?.listPrefix as string | undefined,
>       Delimiter: opts?.listDelimiter as string | undefined,
>       ContinuationToken: opts?.listContinuationToken as string | undefined,
>     }); break
>     case 'COPY':   cmd = new CopyObjectCommand({
>       Bucket: BUCKET,
>       Key: key,
>       CopySource: `${BUCKET}/${opts?.copySource as string}`,
>     }); break
>   }
>   const url = await getSignedUrl(client, cmd, { expiresIn: ttl })
>   res.json({ url })
> })
> ```
>
> Google's own signer (`getSignedUrl` in `@google-cloud/storage`) works too: the
> mount only needs a URL that `fetch()` accepts, and native signing needs no HMAC
> key on the server.
>
> Sign only the operations the app needs: a viewer needs `GET`, `HEAD` and `LIST`.

> [accordion] Bucket CORS rules
>
> GCS refuses the S3 `PutBucketCors` call, since its CORS schema differs. Set the
> rules with `gsutil`:
>
> ```json theme={null}
> [
>   {
>     "origin": ["http://localhost:5173", "https://app.example.com"],
>     "method": ["GET", "PUT", "HEAD", "DELETE", "POST"],
>     "responseHeader": ["Content-Type", "ETag", "Last-Modified"],
>     "maxAgeSeconds": 3000
>   }
> ]
> ```
>
> ```bash theme={null}
> gsutil cors set cors.json gs://$GCS_BUCKET
> ```
>
> GCS lets every request header through, so the rules name none.

> [accordion] Browser fields
>
> **bucket** `string` *required*
>
> Name of the bucket to mount.
>
> **presignedUrlProvider** `function` *required*
>
> Function `(path, operation, options)` that returns a signed URL for each S3 operation; your server signs it, so the browser holds no keys.
>
> **region** `string` (default: 'auto')
>
> Region the presigned URLs are signed for; GCS accepts `auto`.
>
> **endpoint** `string` (default: '[https://storage.googleapis.com](https://storage.googleapis.com)')
>
> Endpoint URL of the GCS XML API. Requests go to the host each presigned URL names; the mount uses it to name its storage location.
>
> **defaultContentType** `string` (default: 'application/octet-stream')
>
> `Content-Type` stored with every object the mount writes.
>
> **keyPrefix** `string`
>
> Key prefix the mount treats as its root. The browser can change it before a URL is signed, so enforce the prefix on the signing server.
>
> **timeout** `number` (default: DEFAULT\_TIMEOUT)
>
> Seconds a request may wait to connect, or between two reads of its answer, before it fails; a download that keeps flowing is never cut off.
>
> Fields marked secret are redacted in snapshots.

## Integration

GCS answers these commands through its own API, so fewer objects are read. See
[Integration](https://docs.mirage.strukto.ai/home/integration/overview.md).

* `md5sum`: the object's ETag, when it is the MD5 (a single-part upload,
  unencrypted or under SSE-S3).
* `find`: one recursive listing.
* `du`: one recursive listing.
* `ls -R` and `tree`: one recursive listing.

## VFS.md

The [VFS.md](https://docs.mirage.strukto.ai/home/setup/vfs-md.md) an agent reads with GCS mounted at `/<path>`.
`<mode>` is the mount's mode: `read`, `write` or `exec`. The commands shown are
those of a `write` mount.

> [tab] Rendered
>
> **Virtual Filesystem**
>
> | Path | Description |
> | - | - |
> | `/<path>` | `<mode>` |
>
> **Supported Unix Commands for VFS**
>
> Unix commands work on each path as on a local filesystem, as far as the path supports them:
>
> | Path | Supported |
> | - | - |
> | `/<path>` | Almost all Unix commands. |
>
> **Limitations of Unix Commands for VFS**
>
> Where a path differs from a local filesystem:
>
> | Path | Limitations |
> | - | - |
> | `/<path>` | `grep -r`, `find` and `du` stop after 1000 files; a glob matching more, `tar -c`, `cp -r` and `rm -r` are refused. |
>
> Run `man <cmd>` or `<cmd> --help` for one command's usage.

> [tab] Raw
>
> ```markdown theme={null}
> ---
> name: workspace-filesystem
> description: The virtual filesystem layout of this workspace.
> ---
>
> # Virtual Filesystem
>
> | Path | Description |
> | --- | --- |
> | `/<path>` | `<mode>` |
>
> ## Supported Unix Commands for VFS
>
> Unix commands work on each path as on a local filesystem, as far as the path supports them:
>
> | Path | Supported |
> | --- | --- |
> | `/<path>` | Almost all Unix commands. |
>
> ## Limitations of Unix Commands for VFS
>
> Where a path differs from a local filesystem:
>
> | Path | Limitations |
> | --- | --- |
> | `/<path>` | `grep -r`, `find` and `du` stop after 1000 files; a glob matching more, `tar -c`, `cp -r` and `rm -r` are refused. |
>
> Run `man <cmd>` or `<cmd> --help` for one command's usage.
> ```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.