Let AI agents drive your Mac apps. Never your mouse.

why not computer-use

Backstage, not on stage

Screen-driving agents borrow your seat: they move the real cursor and read pixels. Kuroko stays out of the audience's view entirely.

the surface an agent gets

Thirteen tools, one protocol

Plain MCP over stdio — point Claude Code, Claude Desktop or any MCP host at the binary and the whole stage crew is available.

• list_appsrunning apps and their bundle ids
• check_permissionsthe server's own AX and recording status
• get_ui_treethe accessibility tree; outline mode for huge windows
• screenshotbackground PNG of one window, size-capped
• tappress an element by label, id or role
• double_tapdouble-click an element
• input_texttype into an element; refuses password fields
• press_key⌘S, ⌘F, arrows — named keys with modifiers
• scrollbring off-screen elements into view
• read_elementone element's fields, as JSON
• assert_visiblewait for an element to appear
• launch_appstart an app in the background
• stop_appquit or force-kill a running app

the trust model

Assume the agent goes rogue

Handing an agent your open apps is a real grant of power — a prompt-injected agent is the realistic threat. Kuroko is built so that the grant is constrained, auditable and revocable.

• App policy

  Terminal, Keychain Access, 1Password and System Settings are denied by default. Flip to a strict allowlist when only one app should be drivable.

• Read-only switch

  "armWriteTools": false disarms tapping, typing, keys and app control in one move. The agent can look, not touch.

• Secure fields refused

  input_text will not type into a password field. Not configurable — on purpose.

• Everything on the record

  Every call lands in a local JSONL action log the app tails live. Typed text is logged by length only; content never hits disk.

• Bounded by design

  Timeouts, tree depth and node budgets are clamped, so a hung target app can never freeze the server.

• No network surface

  stdio only — nothing listens on a port.

setup

Connect in a minute

1. Grant permissions

   Accessibility goes to the process that launches the server — your terminal or agent host. Screen Recording is needed only for the screenshot tool.

2. Add the server

   Claude Code:

   claude mcp add kuroko -- /path/to/kuroko mcp

   Claude Desktop — or any other MCP host:

   {
     "mcpServers": {
       "kuroko": { "command": "/path/to/kuroko", "args": ["mcp"] }
     }
   }

3. Ask for something real

   “Open the git client, tap the commit ‘Improve error messages’, and confirm the detail panel shows its author and date.”