Install v0.49.0

The fastest path: paste this into your coding agent (Claude Code, Codex, Cursor, …) and let it do the setup.

Set up mirroir-mcp so you can drive my iPhone: follow https://github.com/jfarcand/mirroir-mcp/blob/main/docs/onboarding.md step by step, and finish by reading back what my iPhone screen shows.

Or install it yourself:

$ brew tap jfarcand/tap && brew install mirroir-mcp

or via npx:

$ npx -y mirroir-mcp install

or via shell script:

$ /bin/bash -c "$(curl -fsSL https://mirroir.dev/get-mirroir.sh)"

or as a Claude Code plugin:

$ claude plugin marketplace add jfarcand/mirroir-mcp && claude plugin install mirroir@mirroir

or as a Hermes Agent plugin (Hermes 0.21.6 or later), which asks before every call that can change the phone:

$ hermes plugins install jfarcand/mirroir-mcp/plugins/hermes --enable

or, for Claude Desktop, download mirroir-mcp.mcpb from the latest release (v0.43.0 and later) and double-click it.

What mirroir-mcp can do

42 tools exposed as an MCP server.

Touch

  • tap Tap visible text, or a point from describe_screen
  • double_tap Double-tap for zoom or text selection
  • long_press Hold for context menus
  • swipe Swipe between two points: a fast one flicks, a slower one moves the content as far as the finger
  • drag Slow drag for sliders and icons
  • touch Hold a finger across calls for joysticks and press-and-hold
  • pinch Two-finger pinch to zoom
  • rotate Two-finger rotation

Input

  • type_text Type text via virtual keyboard
  • press_key Send special keys with modifiers
  • shake Shake gesture for undo or dev menus
  • hold_keys Hold keys during a mouse drag for keyboard-and-mouse games

Observe

  • screenshot Capture screen as PNG
  • describe_screen OCR or LLM vision with tap coordinates
  • listen Hear what the phone plays, transcribed on the Mac
  • classify_screen Group screen elements into UI components with an AI model
  • start_recording Begin video recording
  • stop_recording End recording, get file path
  • get_orientation Portrait or landscape
  • status Connection and device readiness
  • check_health Full setup diagnostic
  • calibrate_component Test UI component definitions against live screen
  • list_targets List configured automation targets

Navigate

  • launch_app Open app by name via Spotlight
  • open_url Open URL in Safari
  • press_home Return to home screen
  • press_app_switcher Show recent apps
  • press_back Go back via the OCR-detected back chevron
  • spotlight Open Spotlight search
  • scroll_to Scroll toward the content you name until its text shows, or the end is reached
  • reset_app Force-quit app via App Switcher
  • set_network Toggle airplane, Wi-Fi, cellular
  • measure Time screen transitions
  • run_steps Run several steps in one call, stopping at the first failure
  • generate_skill Autonomous BFS exploration → navigation graph → SKILL.md bundle
  • list_skills List available skills
  • get_skill Read skill with env substitution + compilation status
  • get_notes Read what an agent learned about an app in earlier sessions
  • save_note Keep one line about an app for the next session
  • record_step Record a compiled step during execution
  • save_compiled Save compiled .json for zero-OCR replay
  • switch_target Switch active automation target

Listening

listen records only the sound iPhone Mirroring plays, through ScreenCaptureKit and the Screen & System Audio Recording grant mirroir already needs, and macOS's own SpeechAnalyzer transcribes it on the Mac. The audio never leaves the Mac and its temporary file is deleted; the reply holds the words only. Measured on an iPhone XR: 15 seconds of a spoken French recipe reel came back nearly word for word, also with the Mac muted, and silence is reported as silence. It does not say who spoke, sung lyrics come out approximate, and it needs macOS 26. It ships since 0.47.0.

In 0.46.0, scroll_to's direction names where the content is: "down" shows what is below the screen, so the finger moves up. Skills that passed the finger's direction must swap it, and compiled skills must be compiled again. Every parameter: docs/tools.md.

Drive the phone from an agent on another machine

By default mirroir-mcp talks to one MCP client over standard input and output and opens no network port. mirroir-mcp serve speaks MCP over HTTP instead, on 127.0.0.1 only, behind a bearer token, with browsers refused and the same permissions.json. install-agent keeps it running as a LaunchAgent from its own app bundle, "mirroir serve", signed with a certificate kept in your login keychain, so its Accessibility and Screen Recording grants survive upgrades. It needs the Homebrew install.

mirroir-mcp install-agent
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/dev.mirroir.serve.plist

From the other machine, forward the port and point the client at the forward:

ssh -N -L 8765:127.0.0.1:8765 you@your-mac
claude mcp add --transport http mirroir http://127.0.0.1:8765/mcp \
  --header "Authorization: Bearer <contents of ~/.mirroir-mcp/serve-token on the Mac>"

Whoever holds the token controls the phone, within what permissions.json allows. Read Security first; options, grants and removal are in docs/cli.md.

Repeatable flows: Skills

When you find yourself repeating the same agent workflow, capture it as a skill. Skills are SKILL.md files — numbered steps the AI follows, adapting to layout changes and unexpected dialogs. Steps like Tap "Email" use OCR, not coordinates. Share them on the community repository.

Branching + looping

---
version: 1
name: Expo Go Signup + Onboarding
app: Expo Go
tags: ["expo", "signup", "onboarding"]
---

Sign up, handle error or success, then swipe through onboarding.

## Steps

1. Launch **Expo Go**
2. Wait for "LoginDemo" to appear
3. Tap "LoginDemo"
4. Tap "Sign Up"
5. Tap "Email"
6. Type "${TEST_EMAIL}"
7. Tap "Password"
8. Type "${TEST_PASSWORD}"
9. Tap "Create Account"
10. If "already exists" is visible:
    - Tap "Sign In Instead"
    - Tap "Email"
    - Type "${TEST_EMAIL}"
    - Tap "Password"
    - Type "${TEST_PASSWORD}"
    - Tap "Sign In"
11. Otherwise:
    - Wait for "Welcome" to appear
12. Verify "Welcome" is visible
13. Screenshot: "authenticated"
14. Repeat until "Get Started" is visible (max 5 iterations):
    - Swipe left
    - Screenshot: "onboarding"
15. Tap "Get Started"
16. Verify "Dashboard" is visible
17. Screenshot: "complete"

Cross-app workflow

---
version: 1
name: Commute ETA Notification
app: Waze, Messages
tags: ["waze", "messages", "cross-app", "commute"]
---

Get ETA from Waze, text it via iMessage.

## Steps

1. Launch **Waze**
2. Wait for "Where to?" to appear
3. Tap "Where to?"
4. Tap "${DESTINATION:-Work}"
5. Tap "Go"
6. Wait for "min" to appear
7. Remember: Read the commute ETA.
8. Press Home
9. Launch **Messages**
10. Tap "New Message"
11. Tap "To:"
12. Type "${RECIPIENT}"
13. Tap "${RECIPIENT}"
14. Tap "iMessage"
15. Type "On my way! ETA {eta}"
16. Press **Return**
17. Screenshot: "message_sent"

macOS window (experimental)

---
version: 1
name: Play a Song
app: Spotify (macOS)
tags: ["spotify", "macos", "music", "multi-target"]
---

Search for a song on the Spotify macOS app and play it.

## Steps

1. Switch to target "spotify"
2. Tap "Search"
3. Wait for "What do you want to listen to?" to appear
4. Type "${SONG:-Bohemian Rhapsody}"
5. Wait for "${SONG:-Bohemian Rhapsody}" to appear
6. Tap "${SONG:-Bohemian Rhapsody}"
7. Verify "Playing" is visible
8. Screenshot: "now_playing"

Community skill library

The mirroir-skills repository is an open collection of ready-made SKILL.md files — login flows, cross-app workflows, settings automation, and more.${VAR} placeholders resolve from environment variables, so the same skill works across accounts and devices.

Install the full library as a Claude Code plugin*:

$ claude plugin marketplace add jfarcand/mirroir-mcp && claude plugin install skills@mirroir

* Also supported by GitHub Copilot.

    Testing and CI

    The same tools that run your agent also test your app. When you need repeatable tests, mirroir takes you from a first exploration to CI. Point it at an app: it explores the screens it can reach, breadth-first, then writes a bundle of SKILL.md files. Pair it with an APP.md map for guided exploration. Edit the skills, test them from the CLI, diagnose failures with --agent.Skills, exploration and CI.

    1

    Generate

    A single generate_skill(action: "explore") call explores each screen breadth-first, replays paths to reach child screens, and builds a navigation graph of what it reached. Pass explorer: "dfs" for depth-first instead.

    generate_skill(action: "explore", app_name: "Weather")
    2

    Write

    Edit the generated skill or author one from scratch — numbered steps, no coordinates.

    vim login.md
    3

    Run

    The AI reads the skill via get_skill, executes each step with MCP tools, records each one with record_step, and saves the coordinates with save_compiled.

    get_skill → record_step → save_compiled
    4

    Test

    Replay without AI from recorded coordinates — and every step is still checked: a button that moved is found, one that is gone fails the run.

    mirroir test login
    5

    Diagnose

    When a step fails, --agent compares expected vs. actual OCR and tells you the root cause and fix.

    mirroir test --agent claude-sonnet-5-5 login

    Agent diagnosis runs in two tiers: deterministic OCR analysis first (free, no API key), then optionally an AI model for richer analysis. Supports Anthropic, OpenAI, local Ollama, embacle, and CLI agents.

    Describe your app with APP.md

    mirroir can explore any iOS app blindly — but it works best when you tell it what to expect. Drop an APP.md file in ~/.mirroir-mcp/skills/ (or the community patterns/apps folder) and mirroir gets a map before it starts.

    ---
    app: Santé
    archetype: dashboard
    obstacle_mode: auto
    ---
    
    ## Structure
    Dashboard with 4 tabs: Résumé, Partage, Parcourir, Profil.
    
    ## Résumé Tab
    - Summary cards for health metrics that drill down to charts
    - Cards often show "Aucune donnée" on test devices
    
    ## Obstacles
    - Health Access permission → tap "Autoriser"
    - Notification permission → tap "Ne pas autoriser"
    
    ## Skip
    - Supprimer les données de Santé
    - Réinitialiser

    Archetype

    dashboard, social-feed, settings-list, and more — tells mirroir how the app navigates so exploration uses the right strategy.

    Structure & tabs

    Tab names become priority targets when component detection misses non-standard UIs. Section descriptions flow into AI context for generated skills.

    Obstacles

    Permission prompts, onboarding sheets, rating dialogs — declared once, dismissed automatically every time they appear.

    Skip list

    Destructive actions like “Delete account” or “Reset” are never tapped during autonomous exploration.

    Three levels of patterns work together — elements (what rows look like), screens (what the page layout means), and apps (what the developer knows). 30 seconds of writing saves minutes of blind exploration.

    How it compares

    mirroirphone-harnessmobile-mcpAppium
    Physical iPhone, no Developer Mode or signingYes — stock phone via iPhone MirroringYes — stock phone via iPhone MirroringUSB-connected, trusted deviceNo — WebDriverAgent via Xcode, Developer Mode
    App Store apps you can't buildYesYesYesYes, on a provisioned device
    Your own iPhone from any MCP clientYesNo — its MCP server is cloud-only; local phones use its CLIYesThrough the separate appium-mcp
    Replay without an LLMYes — compiled skillsPython scripts you write; no recorder or compiled replayNo — the agent drives every runYes — test code
    Distinct drift verdictYes — DRIFT, exit 65NoNoNo
    One scenario across iPhone + web + HTTPYes — mirroir-runNoNoVia your own test code
    AndroidNoYesYesYes
    iOS SimulatorNoNoYesYes
    Hosted cloud phonesNoYes — Phone Harness CloudYes — Mobile Next CloudVia third-party device clouds

    phone-harness is the closest tool here: it also drives a stock iPhone through iPhone Mirroring, and it adds Android and hosted phones, which mirroir doesn't have. mobile-mcp and Appium cover Android and simulators too. mirroir goes further after the agent's first run: that run becomes a replay that needs no LLM, and a test that tells you when the output moves.

    Replay anywhere with mirroir-run

    mirroir-run replays the same scenarios beyond the iPhone: web pages through Playwright (Chromium, Firefox, WebKit), processes and HTTP natively, and iOS blocks through mirroir-mcp. One YAML file can drive a browser and a phone and check that they show the same thing — and its web, process, and HTTP steps run on Linux CI.

    1

    Validate

    Check a scenario against the grammar before anything runs.

    mirroir-run --validate scenario.yaml
    2

    Compile

    Write the Playwright spec without running it, for review.

    mirroir-run --emit playwright scenario.yaml
    3

    Run

    Run one scenario: web in Playwright, iOS through mirroir-mcp.

    mirroir-run --run-scenario scenario.yaml
    4

    Sample

    Boot once, run several scenarios, aggregate the verdicts.

    mirroir-run --sample samples/foo
    5

    Diff

    Compare two text outputs for drift, exit-coded for CI.

    mirroir-run --diff-text a.txt b.txt

    Every primitive in one run: runner/samples/mega-sample/. Scenario grammar, judge profiles, and CI setup: runner/docs/ and Web and iOS in one scenario.

    Install the runner:

    $ cargo install mirroir-run

    or via Homebrew:

    $ brew install jfarcand/tap/mirroir-run

    or grab a prebuilt binary for macOS / Linux / Windows from the runner-v* releases.

    In GitHub Actions

    setup-mirroir-run installs the released binary on Linux, macOS or Windows, verifies its SHA-256 and adds it to PATH.

    - uses: jfarcand/mirroir-mcp/actions/setup-mirroir-run@main
      with:
        version: latest
    - run: mirroir-run

    Bring your Maestro flows

    Maestro flows run on a real, stock iPhone. Each flow becomes an .ios.yaml scenario; commands mirroir cannot run faithfully fail the import with their file and line. Full mapping: maestro-import.md.

    $ mirroir-run import maestro flows/*.yaml -o .mirroir/apps/shop/scenarios --app com.example.shop=Shop

    More documentation

    Setup, permissions, targets and troubleshooting are in the README; guides are in docs/.