Look Mum No Hands
PandelisZ/look-mum-no-hands
MCP para IA operar apps macOS com cursor virtual e Accessibility
Ver no GitHubServidor MCP local para macOS que permite agentes IA inspecionar e operar aplicações desktop via Accessibility API, sem interferir no cursor do mouse real. Inclui overlay visual, auditoria de chamadas e controle por SwiftUI.
Instalação
npx convez-ai PandelisZ/look-mum-no-hands1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Automação de aplicações desktop com agentes IA
- Teste e inspeção de UIs via Accessibility state
- Integração com Cursor Agent para tarefas GUI
- Auditoria e logging de interações de IA com desktop
Veredito

Ferramenta especializada e bem-pensada para agentes IA operarem macOS. Implementação robusta em Swift com permissões claras, logging integrado e interface de controle. Essencial para fluxos que exigem automação GUI cross-app em ambiente Apple.
README
A tradução para o português deste README ainda está na fila.
Look Mum No Hands
Look Mum No Hands is a local macOS MCP server that lets AI agents inspect and operate desktop apps without taking over your real mouse pointer.
It gives agents a focused macOS tool surface:
- read Accessibility state, running apps, windows, and UI elements
- click, type, scroll, and perform semantic Accessibility actions
- show a visual-only virtual cursor so you can see what the agent is doing
- keep a local audit log of MCP calls
- install local MCP configs for Cursor, Codex, and Claude from a small SwiftUI control app
The real mouse does not move. The virtual cursor is an overlay only.
Setup in Cursor
Use this section if you want LMNH in Cursor Agent (or Chat with MCP tools enabled).
1. Open this repo in Cursor
Clone the repo and open the checkout as your Cursor workspace. LMNH is configured per-workspace via .cursor/mcp.json.
2. Build the MCP server
From the repo root:
swift build
The wrapper script auto-builds lmnh-mcp on first use if you skip this step, but building once upfront is faster.
3. Grant macOS permissions
swift run lmnh-control
In the control app Permissions card:
- Click Request Accessibility and enable Look Mum No Hands in System Settings.
- Click Screen Recording if you want screenshot tools (
macos_get_screenshot). - Quit and reopen Cursor after changing permissions.
macOS ties permissions to the executable that requests them. Cursor launches lmnh-mcp as a child process, so you may need to enable Accessibility for both Cursor and lmnh-mcp (or the debug binary under .build/debug/).
4. Install the MCP server config
Option A — control app (recommended)
In the control app, click Install Cursor. This writes .cursor/mcp.json in the repo.
Option B — use the bundled config
If you cloned this repo, .cursor/mcp.json is already checked in. It should look like:
{
"mcpServers": {
"look-mum-no-hands": {
"type": "stdio",
"command": "bash",
"args": [
"${workspaceFolder}/plugins/look-mum-no-hands/scripts/run-lmnh-mcp.sh"
],
"env": {
"LMNH_REPO_ROOT": "${workspaceFolder}",
"LMNH_LOG_LEVEL": "debug",
"LMNH_OVERLAY_RENDERER": "headless"
}
}
}
}
The launch script builds lmnh-mcp if needed and runs .build/debug/lmnh-mcp.
5. Enable MCP in Cursor
- Open Cursor Settings → MCP (or Features → MCP).
- Confirm
look-mum-no-handsappears under MCP Servers with a green/connected status. - If it is offline, click refresh or fully quit and reopen Cursor.
- Make sure MCP tools are enabled for the mode you use (Agent, Chat, etc.).
6. Verify with an example prompt
Start a new Agent chat in this workspace and paste:
Use the Look Mum No Hands MCP tools (not osascript or shell keyboard events).
1. Call macos_permission_status and tell me if anything is missing.
2. Call macos_list_apps and macos_snapshot for the frontmost window.
3. Find a clickable button or menu item in the snapshot and press it with macos_perform_action (AXPress) or macos_click.
4. Tell me what changed and whether the real mouse moved (it should not).
A shorter smoke-test prompt:
Use Look Mum No Hands: check macos_permission_status, snapshot the frontmost window, and describe the three most prominent interactive elements you see.
If tools are missing from the agent's tool list, the MCP server is not connected — recheck step 5 and the troubleshooting section.
Optional: teach the agent the LMNH workflow
The repo bundles an agent skill at plugins/look-mum-no-hands/skills/look-mum-no-hands/SKILL.md. To make Cursor prefer LMNH over osascript / AppleScript on every session, add a project rule (.cursor/rules/look-mum-no-hands.mdc) that points agents at that skill, or paste its guidance into a User Rule.
Quick Start
1. Build the local server
From the repo root:
swift build
This creates the development binaries under .build/debug/:
.build/debug/lmnh-control.build/debug/lmnh-mcp.build/debug/lmnh-mcp-http
2. Open the control app
swift run lmnh-control
The control app is the easiest setup path. It shows permission status, MCP logs, cursor options, HTTP server controls, and install buttons for supported clients.
3. Grant macOS permissions
In the control app, open the Permissions card.
- Click
Request Accessibility. - Enable Look Mum No Hands in System Settings.
- Click
Screen Recordingand enable Screen & System Audio Recording if you want screenshot-related tools. - Quit and reopen the app or restart your MCP client after changing permissions.
macOS permissions are tied to the exact app or executable identity. If permissions look enabled but LMNH still reports denied, see Troubleshooting Permissions.
4. Install MCP client configs
In the control app, click Install All Plugins, or install one client at a time.
For Cursor, see Setup in Cursor for the full walkthrough and example prompts.
Codex and Claude register a stdio server that runs .build/debug/lmnh-mcp directly. Run swift build again whenever you change the server code, then restart the client.
5. Restart your AI client
Restart Cursor, Codex, Claude, or the specific session using LMNH so it starts the current lmnh-mcp binary.
Requirements
- macOS 26.0 or newer, matching
Package.swift - Swift 6.3 toolchain
- Accessibility permission for UI inspection and actions
- Screen & System Audio Recording permission for screenshot capture
- One or more MCP clients, such as Cursor, Codex, or Claude
What Gets Installed
The control app installs development configs. It does not move the repo or copy a packaged server somewhere else.
Cursor
Install Cursor writes a repo-local config at .cursor/mcp.json. It launches the server through plugins/look-mum-no-hands/scripts/run-lmnh-mcp.sh, which builds and runs .build/debug/lmnh-mcp from the repo root.
See Setup in Cursor for step-by-step instructions and example prompts.
Codex
Install Codex runs the Codex CLI and registers:
look-mum-no-hands-dev
The registered command points at:
/absolute/path/to/look-mum-no-hands/.build/debug/lmnh-mcp
The app searches common CLI locations for codex, including PATH, ~/.local/bin, Homebrew, fnm, and nvm installs.
Claude
Install Claude registers:
look-mum-no-hands-dev
It also links the bundled Claude skill into:
~/.claude/skills/look-mum-no-hands
The app searches common CLI locations for claude, including PATH, ~/.local/bin, Homebrew, fnm, and nvm installs.
Manual Client Setup
Use the control app if possible. Manual setup is useful if you want to configure a different MCP client.
First build the server:
swift build --product lmnh-mcp
Then configure your client to run:
/absolute/path/to/look-mum-no-hands/.build/debug/lmnh-mcp
Example MCP server entry:
{
"mcpServers": {
"look-mum-no-hands-dev": {
"command": "/absolute/path/to/look-mum-no-hands/.build/debug/lmnh-mcp",
"args": [],
"env": {
"LMNH_LOG_LEVEL": "debug"
}
}
}
}
For Codex CLI, the equivalent command is:
codex mcp add look-mum-no-hands-dev -- /absolute/path/to/look-mum-no-hands/.build/debug/lmnh-mcp
For Claude CLI:
claude mcp add --scope user look-mum-no-hands-dev -- /absolute/path/to/look-mum-no-hands/.build/debug/lmnh-mcp
Using The Control App
Permissions
The Permissions card shows:
- Accessibility status
- Screen Recording status
- current bundle identifier
- current process identifier
- LMNH state directory
Buttons:
Request Accessibility: asks macOS to prompt for Accessibility permission.Accessibility Settings: opens the Accessibility privacy pane.Screen Recording: opens the Screen & System Audio Recording privacy pane.
MCP Server
The MCP Server card includes HTTP controls and install buttons.
Buttons:
Start HTTP: starts the HTTP MCP server in the control app.Stop HTTP: stops the HTTP MCP server.Copy URL: copies the HTTP endpoint.Install Cursor: writes.cursor/mcp.json.Install Codex: registers the Codex MCP server.Install Claude: registers the Claude MCP server and links the Claude skill.Install All Plugins: runs all three install steps.
The default HTTP endpoint is:
http://127.0.0.1:8765/mcp
Most desktop clients should use the stdio server (lmnh-mcp). Use HTTP only when your client supports HTTP JSON-RPC MCP connections or for local testing.
Cursor Options
The Cursor Options pane controls the virtual cursor overlay:
- choose the cursor artwork
- preview the selected cursor
- adjust size and animation timing
The bundled cursor styles include the default LMNH cursor and several RealWorld Designer cursor assets. The source and license notes for those assets live at:
Sources/LMNHCore/Resources/Cursors/RealWorldPointers/RealWorldPointers-SOURCE.md
The virtual cursor:
- tilts in the direction of travel
- returns upright at the destination
- shrinks briefly on click and restores to full size
- hides after 10 seconds of inactivity
- never moves the real macOS pointer
Logs
The Logs pane tails the local MCP audit log.
If no calls have happened yet, the app shows the log path. Once a client starts using LMNH, recent MCP calls appear in this pane.
MCP Tools
LMNH currently exposes these tools:
macos_permission_statusmacos_open_appmacos_list_appsmacos_list_windowsmacos_snapshotmacos_get_elementmacos_find_elementsmacos_get_screenshot— ScreenCaptureKit capture of the frontmost window, a specific window, a display, or a snapshot element region; supportsmax_widthdownscaling andpng/jpegoutputmacos_set_virtual_cursormacos_hide_virtual_cursormacos_perform_actionmacos_clickmacos_scroll— focusless scrolling that drives the target's Accessibility scroll bars; works on background windows and never moves the real mouse or changes focusmacos_type_text— focusless AXValue text mutation with optionalsubmit: trueto fire the element'sAXConfirmaction afterwards
Typical agent flow:
- Call
macos_permission_status. - Call
macos_list_appsormacos_open_app. - Call
macos_snapshotfor the target app or frontmost window. - Pick a stable
snapshot_idandelement_id. - Use
macos_perform_action,macos_click, ormacos_type_text. - Watch the virtual cursor and review logs if needed.
How It Works
LMNH is intentionally not a raw remote-control tunnel.
The MCP server gives agents structured macOS state and focused actions:
- app and window enumeration
- Accessibility snapshots
- stable element lookup
- semantic AX actions before fallback click behavior
- local command audit logging
- visual cursor overlay for user awareness
Background by design
LMNH never activates or raises a window. Inspection (macos_snapshot, macos_get_screenshot), interaction (macos_perform_action, macos_click, macos_scroll), and text entry (macos_type_text) all operate on background windows through the Accessibility API and per-window screen capture, so the app you are looking at never changes. There is deliberately no global key-press tool: keyboard shortcuts on a background app are expressed as Accessibility actions (AXPress on a menu item, AXConfirm, AXCancel, AXPick) instead of synthesized global keystrokes.
Chromium- and Electron-based apps (Chrome, Slack, VS Code, Discord, ...) keep their Accessibility tree collapsed until a client opts in. macos_snapshot sets the enhanced-accessibility attributes (AXManualAccessibility / AXEnhancedUserInterface) automatically, so those apps become inspectable without being focused.
The project is split into:
LMNHCore: shared permissions, Accessibility, actions, snapshots, cursor state, cursor overlay, logging, and MCP routinglmnh-mcp: stdio MCP server for Cursor, Codex, Claude, and similar clientslmnh-mcp-http: local HTTP JSON-RPC MCP serverlmnh-control: SwiftUI setup and monitoring app
Development Commands
Build everything:
swift build
Build specific products:
swift build --product lmnh-control
swift build --product lmnh-mcp
swift build --product lmnh-mcp-http
Run the control app:
swift run lmnh-control
Run the stdio MCP server directly:
swift run lmnh-mcp
Run the HTTP MCP server:
swift run lmnh-mcp-http --port 8765
Run tests:
swift test
Run the local verification script:
scripts/verify.sh
Run MCP smoke tests after building:
python3 scripts/mcp_smoke.py
python3 scripts/screenshot_smoke.py
python3 scripts/http_mcp_smoke.py
The screenshot smoke test needs Screen Recording permission and saves sample captures under /tmp/lmnh-screenshot-smoke/.
Troubleshooting
Accessibility says denied even though it is enabled
macOS tracks Accessibility permission by executable identity. Rebuilding, moving, or re-signing an app can create a new identity.
Try this:
- Quit Look Mum No Hands.
- Quit the MCP client that launches
lmnh-mcp. - Open System Settings > Privacy & Security > Accessibility.
- Remove stale entries such as old
lmnh-control, oldlmnh-mcp, or duplicate Look Mum No Hands entries. - Reopen the control app.
- Click
Request Accessibility. - Enable the current Look Mum No Hands entry.
- Restart your MCP client.
- Ask the client to call
macos_permission_status.
If the MCP client launches .build/debug/lmnh-mcp directly, macOS may also ask for permission for that server binary or for the host client process. Grant the permission shown by macOS for the process actually requesting access.
The install buttons say codex or claude was not found
Make sure the CLI works from a normal terminal:
codex --version
claude --version
The control app checks common local install paths, but if your CLI lives somewhere custom, configure the MCP client manually using the absolute .build/debug/lmnh-mcp path.
Client still uses an old version
Rebuild and restart:
swift build --product lmnh-mcp
Then quit and reopen the MCP client. Existing MCP server processes keep running old code until the client restarts them.
You can check for old local server processes with:
pgrep -fl lmnh-mcp
Cursor overlay does not appear
Check:
macos_permission_statusreports Accessibility granted.- The selected cursor appearance is saved in the control app.
- The MCP client restarted after install.
- The cursor has not timed out after 10 seconds of inactivity.
Call macos_set_virtual_cursor again or perform another action to show it.
HTTP server port is already in use
The default HTTP port is 8765. Stop any existing server from the control app, or run the HTTP product manually with another port:
swift run lmnh-mcp-http --port 8766
Security Notes
LMNH runs locally on your Mac. It does not send desktop state anywhere by itself. The MCP client you connect decides what to do with tool results.
Use normal care when giving an AI client desktop-control tools:
- grant permissions only when you intend to use LMNH
- keep an eye on the virtual cursor
- review the Logs pane for recent calls
- disconnect or remove the MCP server when you are done testing
App Icon
The app icon source is bundled at:
Sources/LMNHControlApp/Resources/AppIcon.png
The macOS asset catalog lives at:
Sources/LMNHControlApp/Resources/Assets.xcassets/AppIcon.appiconset