convez.ai
MCP ServerAutomaçãoAvançado
RandyNorthrup

Win32 MCP Server

RandyNorthrup/win32-mcp-server

Automação Windows completa para agentes IA via MCP

Ver no GitHub
BomPluginScore · de 0 a 100

Servidor MCP enterprise-grade para automação completa de aplicações Windows via agentes de IA. Oferece 53 ferramentas especializadas incluindo busca inteligente de texto, OCR estruturado, captura de telas, controle de mouse/teclado, gerenciamento de janelas e processos. Perfecto para automatizar tarefas desktop complexas sem necessidade de coordenadas manuais.

Instalação

$npx convez-ai RandyNorthrup/win32-mcp-server

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
8
Forks
2
Acompanhando
0
Saúde
71%
Último commit
há 1 mês
Versão
v2.6.1
Issues abertas
0
Contribuidores
1

Casos de uso

  • Automação de fluxos RPA em aplicações legadas Windows
  • Testes de UI automatizados com busca inteligente de elementos
  • Controle remoto de desktops por agentes de IA para tarefas administrativas
  • Captura e processamento de dados de múltiplas janelas simultaneamente
  • Orquestração de processos multi-etapas com confirmação de segurança

Veredito

Ferramenta madura e robusta para automação Windows em escala enterprise. Bem documentada, com controles de segurança modernos (perfis read-only, tokens de confirmação) e integração nativa com o ecossistema Claude. Recomendada para cenários complexos de RPA e testes automatizados.

README

A tradução para o português deste README ainda está na fila.

Win32 MCP Server

Enterprise-grade Windows automation for AI agents — 53 tools over MCP

The most comprehensive Windows desktop automation server for the Model Context Protocol. Give any MCP-compatible AI agent full control over Windows applications: intelligent text finding and clicking, structured OCR, screenshot capture, mouse/keyboard input, window management, process control, and multi-step batch operations — all through a single MCP server.

Version PyPI VS Code Marketplace Python License MCP


What's New in v2.6

  • Security profiles — read_only, interactive, allow/block lists, dry-run, and confirmation tokens
  • Central execution controls — serialized OS mutation, hard timeouts, bounded subprocess output
  • Typed argument validation — consistent bounds for mouse, keyboard, clipboard, window, capture, OCR, process, UIA, and smart tools
  • UIA-first smart actions — click_text and fill_field try control-based automation before OCR fallback
  • Stable response option — WIN32_MCP_RESULT_ENVELOPE for {success, tool, data} JSON dict responses
  • Advertised safety args — mutating tool schemas expose dry_run; high-risk tool schemas expose confirmation_token
  • CI and release hygiene — Windows GitHub Actions, contract tests, SECURITY.md, CONTRIBUTING.md, wheel package-data checks
  • VS Code extension hardening — shell-free commands, pinned package version checks, custom Tesseract path support

Features

Smart Automation (the most powerful tools)

ToolDescription
click_textFind text on screen and click it — no coordinates needed
find_text_on_screenLocate all occurrences of text with screen coordinates
wait_for_textPoll until text appears on screen (with timeout)
assert_text_visibleVerify text is or is not visible (for UI testing)
fill_fieldClick a labeled input field and type a value
get_window_snapshotScreenshot + structured OCR in one call
right_click_menuRight-click and OCR the context menu items
execute_sequenceRun up to 50 tools in sequence without round-trips

Screen Capture (6 tools)

  • Full screen, per-window, and per-monitor capture
  • PNG, JPEG, and WebP output with quality/scale controls
  • Pixel color sampling at any coordinate
  • Screenshot comparison with similarity metrics

OCR — Optical Character Recognition (5 tools)

  • Full screen and region-based text extraction
  • Per-window OCR with automatic focus and capture
  • Structured mode — every word with bounding box, confidence, line/block/word numbers
  • Intelligent preprocessing: auto-detects light/dark backgrounds
  • Coordinates map back to original screen space for accurate clicking

Mouse Control (8 tools)

  • Click, double-click, triple-click (left/right/middle buttons)
  • Drag-and-drop with configurable duration and button
  • Mouse move with smooth animation
  • Vertical and horizontal scrolling at any position
  • Current position reporting

Keyboard Control (3 tools)

  • Type text with Unicode support (auto-fallback to clipboard paste)
  • Press individual keys or key combinations (ctrl+c, alt+f4)
  • Execute hotkey combos from arrays (["ctrl", "shift", "s"])

Clipboard (2 tools)

  • Copy text to system clipboard
  • Read current clipboard contents

Window Management (10 tools)

  • List all windows with fuzzy title filtering
  • Detailed window info (PID, position, size, state, process name, memory)
  • Focus, close, minimize, maximize, restore
  • Resize and move to exact coordinates
  • Wait for a window to appear (polling with timeout)
  • Fuzzy matching with intelligent suggestions on miss

Process Management (4 tools)

  • List processes with filtering, sorting, and pagination
  • Graceful termination with force-kill fallback
  • Launch applications with optional wait-for-completion
  • Wait for a process to become idle (CPU threshold monitoring)

System (1 tool)

  • health_check — verify all dependencies, DPI, monitors, Tesseract, and tool count

UI Automation (6 tools)

  • uia_inspect_window — get the control tree of a window
  • uia_find_control — find controls by type, name, or automation ID
  • uia_click_control — click a control by name (more reliable than coordinates)
  • uia_get_control_value — read a control's value or text
  • uia_set_control_value — set a control's value (edit boxes, etc.)
  • uia_get_focused — get info about the currently focused control

Installation

Prerequisites

  1. Python 3.10+
  2. Tesseract OCR (optional — required only for OCR tools):

Install Package

From PyPI (recommended):

pip install win32-mcp-server

From GitHub (latest unreleased):

pip install git+https://github.com/RandyNorthrup/win32-mcp-server.git

From source:

git clone https://github.com/RandyNorthrup/win32-mcp-server.git
cd win32-mcp-server
pip install -e .

Configuration

VS Code with GitHub Copilot

Add to your MCP configuration (%APPDATA%\Code\User\mcp.json):

{
  "servers": {
    "win32-inspector": {
      "type": "stdio",
      "command": "win32-mcp-server"
    }
  }
}

Or install from the VS Code Marketplace — search "Windows Automation Inspector".

Claude Desktop

Add to %APPDATA%\Claude\claude_desktop_config.json:

{
  "mcpServers": {
    "win32-inspector": {
      "command": "win32-mcp-server"
    }
  }
}

Any MCP Client

The server uses STDIO transport and works with any MCP-compatible client.

CLI Smoke Checks

python -m win32_mcp_server --version
python -m win32_mcp_server --list-tools
python -m win32_mcp_server --health-check

Runtime Environment Settings

Set these in your MCP client's server environment when you need production controls:

VariablePurpose
WIN32_MCP_SECURITY_PROFILEinteractive (default), read_only, or unrestricted
WIN32_MCP_ALLOWED_TOOLS / WIN32_MCP_BLOCKED_TOOLSComma-separated tool allow/block lists
WIN32_MCP_ALLOWED_COMMANDS / WIN32_MCP_BLOCKED_COMMANDSComma-separated process command allow/block lists
WIN32_MCP_DRY_RUNSimulate mutating tools without touching the OS
WIN32_MCP_CONFIRMATION_TOKENRequire a token argument for high-risk tools (start_process, kill_process, close_window)
WIN32_MCP_REDACT_SENSITIVE_OUTPUTRedact typed, pasted, and clipboard values in responses/logs
WIN32_MCP_RESULT_ENVELOPEOpt into stable JSON dict responses: {success, tool, data}
WIN32_MCP_TESSERACT_PATHExplicit Tesseract executable path. Usually not needed on Windows; common install paths are auto-detected.
WIN32_MCP_OCR_LANGUAGE / WIN32_MCP_OCR_PREPROCESSOCR language and preprocessing defaults
WIN32_MCP_CAPTURE_FORMAT / WIN32_MCP_CAPTURE_QUALITY / WIN32_MCP_CAPTURE_SCALEScreenshot defaults
WIN32_MCP_MAX_TIMEOUT_SECONDS / WIN32_MCP_MAX_TOOL_RUNTIME_SECONDS / WIN32_MCP_MAX_SUBPROCESS_OUTPUT_BYTESRuntime resource bounds
WIN32_MCP_COORDINATE_VALIDATIONValidate coordinates against actual monitor rectangles
WIN32_MCP_PYAUTOGUI_FAILSAFEKeep PyAutoGUI fail-safe enabled by default; set false only for controlled automation rigs

Usage Examples

Smart Automation (Natural Language)

"Click the 'Submit' button"
"Wait for 'Loading complete' to appear, then click 'Continue'"
"Fill in the 'Username' field with 'admin@example.com'"
"Take a snapshot of the Chrome window and tell me what you see"
"Right-click the desktop and show me the menu options"

Screen Capture

"Capture a screenshot of the entire screen"
"Capture the Notepad window as a compressed JPEG at 50% scale"
"Compare the current screen to this reference image"

OCR

"Extract all text from the screen"
"Get structured OCR data from the region at (100, 200) size 800x600"
"Read all text in the Chrome window"

Mouse & Keyboard

"Click at (500, 300) with the right mouse button"
"Drag from (100, 100) to (500, 500)"
"Type 'Hello World' — use clipboard paste for Unicode characters"
"Press Ctrl+Shift+S"

Window & Process Management

"List all open windows containing 'Visual Studio'"
"Maximize the Chrome window"
"Resize Notepad to 800x600 and move it to (0, 0)"
"Wait for a window titled 'Installation Complete' to appear"
"List the top 20 processes by memory usage"
"Kill process with PID 1234"

Batch Operations

"Execute this sequence: click (100,100), wait 500ms, type 'hello', press Enter"

All 53 Tools

Smart Automation

ToolDescription
click_textFind text on screen and click it
find_text_on_screenFind all text occurrences with coordinates
wait_for_textWait until text appears (polling)
assert_text_visibleAssert text is/isn't visible
fill_fieldClick labeled field and type value
get_window_snapshotScreenshot + OCR in one call
right_click_menuRight-click and OCR the menu
execute_sequenceBatch up to 50 tool calls

Screen Capture

ToolDescription
capture_screenFull screen screenshot (PNG/JPEG/WebP)
capture_windowWindow screenshot with fuzzy title match
capture_monitorCapture specific monitor by index
list_monitorsList monitors with resolution and DPI
get_pixel_colorGet RGB/hex color at coordinates
compare_screenshotsPixel-level comparison with similarity score

OCR

ToolDescription
ocr_screenFull screen text extraction
ocr_regionRegion text extraction
ocr_windowWindow text extraction
ocr_screen_structuredFull screen OCR with bounding boxes
ocr_region_structuredRegion OCR with bounding boxes

Mouse

ToolDescription
clickClick at coordinates (left/right/middle, N clicks)
double_clickDouble-click at coordinates
triple_clickTriple-click to select line/paragraph
dragDrag from start to end with duration
mouse_positionGet current cursor position
mouse_moveMove cursor with smooth animation
scrollVertical scroll at position
scroll_horizontalHorizontal scroll at position

Keyboard

ToolDescription
type_textType text (auto Unicode detection, clipboard fallback)
press_keyPress key or combo (ctrl+c, alt+f4)
hotkeyHotkey from key array (["ctrl","shift","s"])

Clipboard

ToolDescription
clipboard_copyCopy text to clipboard
clipboard_pasteRead clipboard contents

Window Management

ToolDescription
list_windowsList windows with optional title filter
get_window_infoDetailed window info (PID, process, memory)
focus_windowBring window to foreground
close_windowClose window by title
minimize_windowMinimize window
maximize_windowMaximize window
restore_windowRestore from minimized/maximized
resize_windowResize to exact dimensions
move_windowMove to exact position
wait_for_windowWait for window to appear (polling)

Process Management

ToolDescription
list_processesList processes (filter, sort, paginate)
kill_processTerminate process (graceful + force fallback)
start_processLaunch application with optional wait
wait_for_idleWait for process CPU to drop below threshold

System

ToolDescription
health_checkFull dependency and system status report

UI Automation

ToolDescription
uia_inspect_windowInspect control tree (buttons, edits, etc.)
uia_find_controlFind controls by name, automation ID, or type
uia_click_controlClick a control by name (no coordinates needed)
uia_get_control_valueRead a control's value/text
uia_set_control_valueSet a control's value
uia_get_focusedGet info about the focused control

Architecture

win32-mcp-server/
├── win32_mcp_server/
│   ├── __init__.py              # Package entry, version
│   ├── __main__.py              # python -m support
│   ├── config.py                # Dataclass config, PreprocessMode
│   ├── registry.py              # Decorator-based tool registry + dispatch
│   ├── server.py                # MCP server, stdio transport, health_check
│   ├── utils/
│   │   ├── args.py              # Typed tool argument readers and bounds
│   │   ├── coordinates.py       # DPI awareness, screen geometry, validation
│   │   ├── errors.py            # ToolError with suggestions
│   │   ├── execution.py         # Timeouts, dry-run, serialized mutations
│   │   ├── imaging.py           # Image preprocessing, encoding, diffing
│   │   ├── security.py          # Safety policy, redaction, command controls
│   │   └── window_match.py      # Fuzzy title matching, deduplication, PID
│   └── tools/
│       ├── capture.py           # Screenshot tools (6)
│       ├── ocr.py               # OCR tools (5)
│       ├── mouse.py             # Mouse tools (8)
│       ├── keyboard.py          # Keyboard tools (3)
│       ├── clipboard.py         # Clipboard tools (2)
│       ├── window.py            # Window management tools (10)
│       ├── process.py           # Process management tools (4)
│       ├── smart.py             # Smart automation tools (8)
│       └── uia.py               # UI Automation API tools (6)
├── extension.js                 # VS Code extension bootstrap
├── tests/                       # Safety, registry, and contract tests
├── .github/workflows/ci.yml     # Windows CI quality gate
├── package.json                 # VS Code extension manifest
├── pyproject.toml               # Python package config
└── LICENSE                      # MIT License

Security Considerations

This server has powerful system control capabilities. Only use in trusted environments where you control the MCP client.

The server can:

  • Capture screenshots of any window or the entire desktop
  • Read and write the system clipboard
  • Control mouse and keyboard input
  • Terminate processes
  • Launch applications

Recommended Practices

  1. Start restrictive — default interactive blocks high-risk process/window actions; use read_only for observation-only sessions
  2. Use allow/block lists — set WIN32_MCP_ALLOWED_TOOLS, WIN32_MCP_BLOCKED_TOOLS, and command lists for production
  3. Dry-run first — use WIN32_MCP_DRY_RUN=true or per-call dry_run: true before granting mutating control
  4. Require confirmation tokens — set WIN32_MCP_CONFIRMATION_TOKEN for high-risk process/window actions
  5. Review automation logs — tool calls are logged with sensitive fields redacted
  6. Restrict MCP client access — only trusted clients should invoke this server

Quality Gates

Run the same checks as CI before shipping changes:

python -m pip install -e .[dev]
python -m ruff check win32_mcp_server tests
python -m mypy win32_mcp_server
python -m pytest -q
node --check extension.js

Release Publishing

GitHub release assets are published from local build output. PyPI publishing runs through the Publish PyPI GitHub Actions workflow using PyPI Trusted Publishing. Configure the PyPI project for repository RandyNorthrup/win32-mcp-server, workflow publish-pypi.yml, environment pypi, then run the workflow with the target release tag.


Troubleshooting

ProblemSolution
TesseractNotFoundErrorInstall with winget install --id tesseract-ocr.tesseract --exact; PATH is optional for common Windows installs
PermissionError: Access is deniedRun VS Code / MCP client as Administrator
ModuleNotFoundError: No module named 'mcp'pip install -e . or pip install win32-mcp-server
Window not found: [title]Use partial title. Run list_windows to see exact titles. Fuzzy matching is automatic.
OCR returns empty/garbled textTry preprocess: "dark_bg" or "high_contrast" for better results
Coordinates are wrong on HiDPIThe server auto-enables DPI awareness. Run health_check to verify DPI settings.

Dependencies

PackagePurpose
mcpModel Context Protocol SDK
mssFast cross-platform screen capture
PillowImage processing and encoding
NumPyImage preprocessing for OCR
PyAutoGUIMouse and keyboard automation
PyGetWindowWindow enumeration and control
pyperclipClipboard operations
pytesseractTesseract OCR wrapper
psutilProcess management
RapidFuzzFast fuzzy string matching
uiautomationWindows UI Automation API

Contributing

Contributions welcome!

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/my-feature)
  3. Commit your changes (git commit -am 'Add my feature')
  4. Push to the branch (git push origin feature/my-feature)
  5. Open a Pull Request

License

MIT License — see LICENSE file.

Links


Author: Randy Northrup Built for Windows automation and AI agents

Ver o original no GitHub

Bom para

Relacionados