Computer Use MCP Server
MrSongP/computer-use
Automação UI completa no Windows via MCP com screenshots e controle total de desktop
Ver no GitHubMCP server local para Windows que expõe UI Automation, screenshots, controle de entrada (mouse/teclado), lançamento de apps e gerenciamento de ciclo de vida. Funciona com Codex e Claude Code através de um runtime TypeScript e host nativo em C#.
Instalação
npx convez-ai MrSongP/computer-use1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Automação de tarefas repetitivas em aplicações Windows
- Teste automatizado de interfaces desktop
- Captura de screenshots e análise de estado de janelas
- Controle remoto de aplicativos via UI Automation
Veredito

Ferramenta sólida e completa para quem precisa automatizar fluxos desktop complexos no Windows. Requer setup técnico robusto (Node 20+, .NET 8+) mas oferece controle granular sobre UI Automation, screenshots estruturados e gerenciamento de ciclo de vida via MCP.
README
A tradução para o português deste README ainda está na fila.
computer_use
computer_use is a local Windows computer-use plugin for Codex and Claude Code. It exposes window discovery, screenshots, UI Automation text trees, mouse and keyboard input, app launch, turn lifecycle, interrupt handling, and trace evidence through one TypeScript runtime and an MCP server. The plugin root is computer_use, with entrypoints in computer_use/.codex-plugin/plugin.json, computer_use/.mcp.json, and computer_use/skills/computer-use/SKILL.md.
Quick Start
Requirements:
- Windows 10 or Windows 11
- Node.js 20+
- Codex CLI for Codex installs, or Claude Code CLI for Claude installs
- .NET SDK 8+ for the C# native host build
Check the required command-line tools:
node --version
npm --version
dotnet --info
claude --version
codex --version
Install .NET SDK 8 with Windows Package Manager when dotnet --info does not list an SDK:
winget install --id Microsoft.DotNet.SDK.8 --exact --accept-source-agreements --accept-package-agreements
If dotnet is not available in the current terminal after installation, the installer also checks C:\Program Files\dotnet\dotnet.exe. If that file is missing, open a new terminal after installing the SDK or add C:\Program Files\dotnet to PATH.
Install into Codex from the repository root:
cd <path-to-cloned-repo>
npm run install:codex
Install into Claude Code:
cd <path-to-cloned-repo>
npm run install:claude
Install both:
npm run install:all
The npm installer installs TypeScript dependencies, builds the runtime, builds the C# native host, runs an MCP smoke test, registers the local marketplace, installs the plugin, and runs a Node-based doctor check.
Native Host Build
Build only the C# Windows native host:
npm run build:native
Build TypeScript plus C#:
npm run build:all
If you already built everything and only want to reinstall the current compiled artifacts:
npm run install:codex:compiled
npm run install:claude:compiled
The native-host builder uses dotnet build through .NET SDK 8+. It checks PATH, COMPUTER_USE_DOTNET_PATH, and the standard Windows install locations for dotnet.exe.
The .NET build targets net8.0-windows10.0.19041.0 so the native host can consume the Windows SDK C#/WinRT projections used by Windows.Graphics.Capture. Keep the TypeScript native-host launch path in sync with that target framework when changing the project file.
Capabilities
list_apps,list_windows,get_window,launch_appget_window_statewith direct MCP image content for screenshots and structured UIA nodesclick,click_element,press_key,type_text,scroll,set_value,drag,perform_secondary_action,activate_windowend_turn, turn lifecycle, physical Escape interrupt handling, and trace evidence- Codex adapter, Claude Code MCP adapter, and Windows native-host bridge
launch_app blocks duplicate cold launches by default. If an existing session is detected, the hook returns guidance to restore the app from windows.shell.taskbar; only an explicit force_new bypasses this behavior.
The Windows native host is turn-scoped. Normal completion, adapter close/shutdown, host stdio disconnect, process cleanup hooks, and a short native-host idle timeout release Computer Use resources; the host is restarted on demand because post-build startup is lightweight compared with desktop operations.
Development
cd <path-to-cloned-repo>
npm run typecheck
npm test
Focused checks:
npm --prefix computer_use run test -- tests/integration/codex-adapter.test.ts
npm --prefix computer_use run test -- tests/integration/claude-code-adapter.test.ts
npm --prefix computer_use run test -- tests/integration/stdio-runtime.test.ts
npm --prefix computer_use run test -- tests/integration/native-host-p5-smoke.test.ts
npm run doctor:codex
npm run doctor:claude
npm run codex:helper inside computer_use/ is only a local JSON-RPC helper harness for development. It is not the normal entrypoint after installing the Codex plugin.
Documentation
- Documentation index: doc/README.md
- Project overview: doc/computer-use.md
- Capability matrix: doc/acceptance/capability-matrix.md
- Windows native interface: doc/windows_native_interface/windows-native-interface-design.md
- Agent harness: .claude/computer-use-harness.md and .agents/computer-use-harness.md
Windows Notes
- Windows 10 and Windows 11 are both supported targets.
- The native-host builder runs through .NET SDK 8+ and automatically checks the standard Windows
dotnet.exeinstall path. - The taskbar target supports the primary Win10/Win11 taskbar and falls back to
Shell_SecondaryTrayWndfor secondary-taskbar layouts. - Windows UI automation can affect the real desktop state, so keep target windows clear before running action tools.