Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.mirage.strukto.ai/llms.txt

Use this file to discover all available pages before exploring further.

This guide covers what you need to implement when adding a new resource to MIRAGE. Use the Telegram, Discord, or Slack resources as reference.

​
File Structure

mirage/
  resource/<name>/
    __init__.py          # lazy-loading exports
    config.py            # Pydantic config (credentials)
    <name>.py            # BaseResource subclass
  accessor/<name>.py     # Accessor wrapping config
  core/<name>/
    __init__.py
    _client.py           # HTTP client (get/post with rate limiting)
    readdir.py           # directory listing
    read.py              # file reading
    stat.py              # file metadata
    scope.py             # scope detection
    glob.py              # glob pattern resolution
    ...                  # data fetching modules (history, search, post, etc.)
  ops/<name>/
    __init__.py           # exports OPS list
    read.py               # @op wrapper
    readdir.py            # @op wrapper
    stat.py               # @op wrapper
  commands/builtin/<name>/
    __init__.py           # exports COMMANDS list
    _plan.py              # cost estimation helpers
    cat.py, ls.py, ...    # standard commands
    <name>_send.py, ...   # resource-specific commands

​
Implementation Steps

​
1. Config, Accessor, ResourceName

Create a Pydantic config model to hold credentials and an accessor class that wraps it. 
# mirage/resource/<name>/config.py
from pydantic import BaseModel


class MyConfig(BaseModel):
    token: str

# mirage/accessor/<name>.py
from mirage.accessor.base import Accessor
from mirage.resource.<name>.config import MyConfig


class MyAccessor(Accessor):

    def __init__(self, config: MyConfig) -> None:
        self.config = config
Add MY_RESOURCE = "<name>" to the ResourceName enum in mirage/types.py.

​
2. HTTP Client

Wrap the resource’s API with rate-limit handling. All resources follow the same pattern: async get/post functions with retry on 429. 
# mirage/core/<name>/_client.py
async def my_get(config, endpoint, params=None) -> dict: ...
async def my_post(config, endpoint, body=None) -> dict: ...

​
3. Core VFS

Implement the three VFS operations that map API data to a filesystem:
  • readdir.py — Returns list[str] of child paths for a directory.
  • read.py — Returns bytes content of a file.
  • stat.py — Returns FileStat with name, type, and extras (e.g., IDs).
All three accept (accessor, path, index, prefix) and use IndexCacheStore to cache name-to-ID mappings.

​
4. Scope Detection and GlobScope Optimization

GlobScope carries the raw path and pattern before expansion. This lets commands decide how to resolve paths efficiently — skipping expensive glob expansion when the resource has a native API for the operation. scope.py parses the unexpanded path to determine the level: 
@dataclass
class MyScope:
    level: str  # "root", "category", "item", "file"
    item_id: str | None = None
How commands use scope for optimization: 
# Example: grep at different scopes

@command("grep", resource="my_resource", spec=SPECS["grep"])
async def grep(accessor, paths, *texts, **_extra):
    pattern = texts[0]
    scope = detect_scope(paths[0], index)

    if scope.level in ("category", "item"):
        # CHEAP: use native search API (1 API call)
        results = await search_api(accessor.config, scope.item_id, pattern)
        return format_results(results), IOResult()

    # EXPENSIVE fallback: expand glob, download each file, grep locally
    paths = await resolve_glob(accessor, paths, index=index)
    for p in paths:
        data = await read(accessor, p.original, index, prefix=p.prefix)
        # ... grep the bytes ...
When to use this pattern:
ScenarioApproach
Resource has a search API (Discord, Slack)Use scope to route to native API at directory level
Resource has no search API (Telegram)Always fall through to file-level reads; scope is a noop
head/tail with direct message fetchUse scope to detect file level, fetch N messages directly
When the resource has no search API, keep scope.py as a noop for structural consistency. The file still parses path parts but does not trigger any API calls: 
# Noop scope -- no search API, no resource-relative paths
def detect_scope(path: str | GlobScope) -> MyScope:
    key = path.strip("/")
    parts = key.split("/")
    # Just parse path structure, no index lookups
    ...

​
5. Glob Resolution

# mirage/core/<name>/glob.py
async def resolve_glob(accessor, paths, index=None) -> list[GlobScope]:
    # Expand patterns via readdir + fnmatch

​
6. Ops Layer

Thin wrappers that bridge core functions to the command framework: 
@op("read", resource="<name>")
async def read(accessor, scope, **kwargs) -> bytes:
    return await core_read(accessor, scope.original,
                           kwargs.get("index"), prefix=scope.prefix)

​
7. Commands

Copy from an existing resource (Discord/Slack), then:
  1. Replace accessor and core imports.
  2. Set resource="<name>" in decorators.
  3. Add or remove scope-based optimizations depending on API capabilities.
  4. Add resource-specific commands (send message, etc.).

​
8. Resource Class

class MyResource(BaseResource):
    name: str = ResourceName.MY_RESOURCE
    is_remote: bool = True

    def __init__(self, config: MyConfig) -> None:
        super().__init__()
        self.config = config
        self.accessor = MyAccessor(self.config)
        from mirage.commands.builtin.<name> import COMMANDS
        from mirage.ops.<name> import OPS

        for fn in COMMANDS:
            self.register(fn)
        for fn in OPS:
            self.register_op(fn)