{ "schemaVersion": "1.0", "item": { "slug": "mac-use", "name": "mac-use", "source": "tencent", "type": "skill", "category": "开发工具", "sourceUrl": "https://clawhub.ai/kekejun/mac-use", "canonicalUrl": "https://clawhub.ai/kekejun/mac-use", "targetPlatform": "OpenClaw" }, "install": { "downloadMode": "redirect", "downloadUrl": "/downloads/mac-use", "sourceDownloadUrl": "https://wry-manatee-359.convex.site/api/v1/download?slug=mac-use", "sourcePlatform": "tencent", "targetPlatform": "OpenClaw", "installMethod": "Manual import", "extraction": "Extract archive", "prerequisites": [ "OpenClaw" ], "packageFormat": "ZIP package", "includedAssets": [ "requirements.txt", "README.md", "SKILL.md", "scripts/mac_use.py" ], "primaryDoc": "SKILL.md", "quickSetup": [ "Download the package from Yavira.", "Extract the archive and review SKILL.md first.", "Import or place the package into your OpenClaw setup." ], "agentAssist": { "summary": "Hand the extracted package to your coding agent with a concrete install brief instead of figuring it out manually.", "steps": [ "Download the package from Yavira.", "Extract it into a folder your agent can access.", "Paste one of the prompts below and point your agent at the extracted folder." ], "prompts": [ { "label": "New install", "body": "I downloaded a skill package from Yavira. Read SKILL.md from the extracted folder and install it by following the included instructions. Then review README.md for any prerequisites, environment setup, or post-install checks. Tell me what you changed and call out any manual steps you could not complete." }, { "label": "Upgrade existing", "body": "I downloaded an updated skill package from Yavira. Read SKILL.md from the extracted folder, compare it with my current installation, and upgrade it while preserving any custom configuration unless the package docs explicitly say otherwise. Then review README.md for any prerequisites, environment setup, or post-install checks. Summarize what changed and any follow-up checks I should run." } ] }, "sourceHealth": { "source": "tencent", "status": "healthy", "reason": "direct_download_ok", "recommendedAction": "download", "checkedAt": "2026-05-07T17:22:31.273Z", "expiresAt": "2026-05-14T17:22:31.273Z", "httpStatus": 200, "finalUrl": "https://wry-manatee-359.convex.site/api/v1/download?slug=afrexai-annual-report", "contentType": "application/zip", "probeMethod": "head", "details": { "probeUrl": "https://wry-manatee-359.convex.site/api/v1/download?slug=afrexai-annual-report", "contentDisposition": "attachment; filename=\"afrexai-annual-report-1.0.0.zip\"", "redirectLocation": null, "bodySnippet": null }, "scope": "source", "summary": "Source download looks usable.", "detail": "Yavira can redirect you to the upstream package for this source.", "primaryActionLabel": "Download for OpenClaw", "primaryActionHref": "/downloads/mac-use" }, "validation": { "installChecklist": [ "Use the Yavira download entry.", "Review SKILL.md after the package is downloaded.", "Confirm the extracted package contains the expected setup assets." ], "postInstallChecks": [ "Confirm the extracted package includes the expected docs or setup files.", "Validate the skill or prompts are available in your target agent workspace.", "Capture any manual follow-up steps the agent could not complete." ] }, "downloadPageUrl": "https://openagent3.xyz/downloads/mac-use", "agentPageUrl": "https://openagent3.xyz/skills/mac-use/agent", "manifestUrl": "https://openagent3.xyz/skills/mac-use/agent.json", "briefUrl": "https://openagent3.xyz/skills/mac-use/agent.md" }, "agentAssist": { "summary": "Hand the extracted package to your coding agent with a concrete install brief instead of figuring it out manually.", "steps": [ "Download the package from Yavira.", "Extract it into a folder your agent can access.", "Paste one of the prompts below and point your agent at the extracted folder." ], "prompts": [ { "label": "New install", "body": "I downloaded a skill package from Yavira. Read SKILL.md from the extracted folder and install it by following the included instructions. Then review README.md for any prerequisites, environment setup, or post-install checks. Tell me what you changed and call out any manual steps you could not complete." }, { "label": "Upgrade existing", "body": "I downloaded an updated skill package from Yavira. Read SKILL.md from the extracted folder, compare it with my current installation, and upgrade it while preserving any custom configuration unless the package docs explicitly say otherwise. Then review README.md for any prerequisites, environment setup, or post-install checks. Summarize what changed and any follow-up checks I should run." } ] }, "documentation": { "source": "clawhub", "primaryDoc": "SKILL.md", "sections": [ { "title": "Mac Use", "body": "Control any macOS GUI application through a screenshot → pick element → click → verify loop." }, { "title": "Setup", "body": "Platform: macOS only (requires Apple Vision framework for OCR)\n\nSystem binaries (pre-installed on macOS):\n\npython3 — via Homebrew (brew install python)\nscreencapture — built-in macOS utility\n\nPython packages — install from the skill directory:\n\npip3 install --break-system-packages -r {baseDir}/requirements.txt" }, { "title": "How It Works", "body": "The screenshot command captures a window, uses Apple Vision OCR to detect all text elements, draws numbered annotations on the image, and returns both:\n\nAnnotated image at /tmp/mac_use.png — numbered green boxes around each detected text\nElement list in JSON — [{num: 1, text: \"Submit\", at: [500, 200]}, {num: 2, text: \"Cancel\", at: [600, 200]}, ...] where at is the center point [x, y] on the 1000x1000 canvas (origin at top-left)\n\nYou receive both by calling Bash (gets JSON with element list) and then Read on /tmp/mac_use.png (gets the visual). Always do both so you can cross-reference the numbers with what you see." }, { "title": "Quick Reference", "body": "# List all visible windows\npython3 {baseDir}/scripts/mac_use.py list\n\n# Screenshot + annotate (returns image + numbered element list)\npython3 {baseDir}/scripts/mac_use.py screenshot [--id N]\n\n# Click element by number (primary click method)\npython3 {baseDir}/scripts/mac_use.py clicknum \n\n# Click at canvas coordinates (fallback for unlabeled icons)\npython3 {baseDir}/scripts/mac_use.py click --app [--id N] \n\n# Scroll inside a window\npython3 {baseDir}/scripts/mac_use.py scroll --app [--id N] \n\n# Type text (uses clipboard paste — supports all languages)\npython3 {baseDir}/scripts/mac_use.py type [--app ] \"text here\"\n\n# Press key or combo\npython3 {baseDir}/scripts/mac_use.py key [--app ] " }, { "title": "Workflow", "body": "Open the target app with open -a \"App Name\" (optionally with a URL or file path)\nWait for it to load: sleep 2\nScreenshot the app:\npython3 {baseDir}/scripts/mac_use.py screenshot [--id N]\n\nThis returns JSON with file (image path) and elements (numbered text list).\nRead the annotated image at /tmp/mac_use.png to see the numbered elements visually\nDecide which element to interact with:\n\nPrefer clicknum N — pick the number of a detected text element\nFallback click --app x y — only for unlabeled icons (arrows, close buttons, cart icons) that have no text and therefore no number\n\n\nAct using clicknum, type, key, or scroll\nScreenshot again to verify the result\nRepeat from step 3" }, { "title": "list", "body": "Show all visible app windows.\n\npython3 {baseDir}/scripts/mac_use.py list\n\nReturns JSON array: [{\"app\":\"Google Chrome\",\"title\":\"Wikipedia\",\"id\":4527,\"x\":120,\"y\":80,\"w\":1200,\"h\":800}, ...]" }, { "title": "screenshot", "body": "Capture a window, detect text elements via OCR, annotate with numbered markers, and return the element list. The target window is automatically raised to the top before capture, so overlapping windows are handled.\n\npython3 {baseDir}/scripts/mac_use.py screenshot chrome\npython3 {baseDir}/scripts/mac_use.py screenshot chrome --id 4527\n\n: fuzzy, case-insensitive match (e.g. \"chrome\" matches \"Google Chrome\")\n--id N: target a specific window ID (required when multiple windows of the same app exist)\nReturns JSON with:\n\nfile: path to annotated screenshot (/tmp/mac_use.png)\nid, app, title, scale: window metadata\nelements: array of {num, text, at} — the numbered clickable text elements, where at is [x, y] center coordinates on the 1000x1000 canvas (origin at top-left)\n\n\nIf multiple windows match, returns a list of windows instead — pick one and retry with --id\nThe image is 1000x1000 pixels with green bounding boxes and blue number badges\nElement map is saved to /tmp/mac_use_elements.json for clicknum" }, { "title": "clicknum", "body": "Click on a numbered element from the last screenshot. This is the primary click method.\n\npython3 {baseDir}/scripts/mac_use.py clicknum 5\npython3 {baseDir}/scripts/mac_use.py clicknum 12\n\nN: the element number from the last screenshot output\nReads the saved element map, activates the window, and clicks at the element's center\nReturns JSON with clicked_num, text, canvas coords, and absolute screen coords" }, { "title": "click", "body": "Click at a position using canvas coordinates. Fallback only — use for unlabeled icons.\n\npython3 {baseDir}/scripts/mac_use.py click --app chrome 500 300\npython3 {baseDir}/scripts/mac_use.py click --app chrome --id 4527 500 300\n\nCoordinates are canvas positions (0-1000) from the screenshot image\nx=0 is left, x=1000 is right; y=0 is top, y=1000 is bottom\nUse this only when Vision OCR didn't detect the element (icon-only buttons, images, etc.)" }, { "title": "scroll", "body": "Scroll inside an app window.\n\npython3 {baseDir}/scripts/mac_use.py scroll --app chrome down 5\npython3 {baseDir}/scripts/mac_use.py scroll --app notes up 10\n\nDirections: up, down, left, right\nAmount: number of scroll clicks (3-5 for moderate, 10+ for fast scrolling)\nMouse is moved to the center of the window before scrolling" }, { "title": "type", "body": "Type text into the currently focused input field.\n\npython3 {baseDir}/scripts/mac_use.py type --app chrome \"hello world\"\npython3 {baseDir}/scripts/mac_use.py type --app chrome \"你好世界\"\n\n--app: activates the app first to ensure keystrokes go to the right window\nUses clipboard paste (Cmd+V) for reliable Unicode/CJK support\nAlways click on the target input field first before typing" }, { "title": "key", "body": "Press a single key or key combination.\n\npython3 {baseDir}/scripts/mac_use.py key --app chrome return\npython3 {baseDir}/scripts/mac_use.py key --app chrome cmd+a\npython3 {baseDir}/scripts/mac_use.py key --app chrome cmd+shift+s\n\n--app: activates the app first\nCommon keys: return, tab, escape, space, delete, backspace, up, down, left, right\nModifiers: cmd, ctrl, alt/opt, shift" }, { "title": "Important Rules", "body": "Always screenshot before your first interaction with an app\nAlways screenshot after an action to verify the result\nAlways Read the screenshot image after running the screenshot command — you need both the element list AND the visual\nPrefer clicknum over click — only use direct coordinates for unlabeled icons\nClick before typing — ensure the correct input field has focus first\nMultiple windows: if you get multiple_windows error, use list to see all windows, then pass --id\nPopup windows (like WeChat mini-program panels) are separate windows with their own IDs — use list to find them and --id to target them\nWait after opening apps: use sleep 2-3 after open -a before taking a screenshot\nActivate the app before screenshot/click: prepend osascript -e 'tell application \"AppName\" to activate' && sleep 1 when the target app may be behind other windows\nDo not type passwords or secrets via this tool" }, { "title": "Coordinate System (for fallback click only)", "body": "Screenshots are rendered onto a 1000x1000 canvas:\n\nOrigin (0, 0) is at the top-left corner\nx increases left to right (0 = left edge, 1000 = right edge)\ny increases top to bottom (0 = top edge, 1000 = bottom edge)\nThe app window is scaled to fit (aspect ratio preserved), centered, with dark gray padding" }, { "title": "Example: Order food on Meituan in WeChat", "body": "# 1. Open WeChat\nopen -a \"WeChat\"\nsleep 3\n\n# 2. Screenshot WeChat — find the mini program window\npython3 {baseDir}/scripts/mac_use.py list\n# → find the mini program window ID\n\n# 3. Screenshot the mini program (annotated + element list)\npython3 {baseDir}/scripts/mac_use.py screenshot 微信 --id 41266\n# → returns: {\"file\": \"/tmp/mac_use.png\", \"elements\": [{num: 1, text: \"搜索\", at: [500, 200]}, ...]}\n# → Read /tmp/mac_use.png to see annotated image\n\n# 4. Click \"搜索\" (element #1)\npython3 {baseDir}/scripts/mac_use.py clicknum 1\n\n# 5. Type search query\npython3 {baseDir}/scripts/mac_use.py type --app 微信 \"炸鸡\"\n\n# 6. Press Enter\npython3 {baseDir}/scripts/mac_use.py key --app 微信 return\nsleep 2\n\n# 7. Screenshot to see results\npython3 {baseDir}/scripts/mac_use.py screenshot 微信 --id 41266\n# → Read /tmp/mac_use.png, pick a restaurant by number\n\n# 8. Click on a restaurant (e.g. element #5)\npython3 {baseDir}/scripts/mac_use.py clicknum 5" } ], "body": "Mac Use\n\nControl any macOS GUI application through a screenshot → pick element → click → verify loop.\n\nSetup\n\nPlatform: macOS only (requires Apple Vision framework for OCR)\n\nSystem binaries (pre-installed on macOS):\n\npython3 — via Homebrew (brew install python)\nscreencapture — built-in macOS utility\n\nPython packages — install from the skill directory:\n\npip3 install --break-system-packages -r {baseDir}/requirements.txt\n\nHow It Works\n\nThe screenshot command captures a window, uses Apple Vision OCR to detect all text elements, draws numbered annotations on the image, and returns both:\n\nAnnotated image at /tmp/mac_use.png — numbered green boxes around each detected text\nElement list in JSON — [{num: 1, text: \"Submit\", at: [500, 200]}, {num: 2, text: \"Cancel\", at: [600, 200]}, ...] where at is the center point [x, y] on the 1000x1000 canvas (origin at top-left)\n\nYou receive both by calling Bash (gets JSON with element list) and then Read on /tmp/mac_use.png (gets the visual). Always do both so you can cross-reference the numbers with what you see.\n\nQuick Reference\n# List all visible windows\npython3 {baseDir}/scripts/mac_use.py list\n\n# Screenshot + annotate (returns image + numbered element list)\npython3 {baseDir}/scripts/mac_use.py screenshot [--id N]\n\n# Click element by number (primary click method)\npython3 {baseDir}/scripts/mac_use.py clicknum \n\n# Click at canvas coordinates (fallback for unlabeled icons)\npython3 {baseDir}/scripts/mac_use.py click --app [--id N] \n\n# Scroll inside a window\npython3 {baseDir}/scripts/mac_use.py scroll --app [--id N] \n\n# Type text (uses clipboard paste — supports all languages)\npython3 {baseDir}/scripts/mac_use.py type [--app ] \"text here\"\n\n# Press key or combo\npython3 {baseDir}/scripts/mac_use.py key [--app ] \n\nWorkflow\nOpen the target app with open -a \"App Name\" (optionally with a URL or file path)\nWait for it to load: sleep 2\nScreenshot the app:\npython3 {baseDir}/scripts/mac_use.py screenshot [--id N]\n\nThis returns JSON with file (image path) and elements (numbered text list).\nRead the annotated image at /tmp/mac_use.png to see the numbered elements visually\nDecide which element to interact with:\nPrefer clicknum N — pick the number of a detected text element\nFallback click --app x y — only for unlabeled icons (arrows, close buttons, cart icons) that have no text and therefore no number\nAct using clicknum, type, key, or scroll\nScreenshot again to verify the result\nRepeat from step 3\nCommands\nlist\n\nShow all visible app windows.\n\npython3 {baseDir}/scripts/mac_use.py list\n\n\nReturns JSON array: [{\"app\":\"Google Chrome\",\"title\":\"Wikipedia\",\"id\":4527,\"x\":120,\"y\":80,\"w\":1200,\"h\":800}, ...]\n\nscreenshot\n\nCapture a window, detect text elements via OCR, annotate with numbered markers, and return the element list. The target window is automatically raised to the top before capture, so overlapping windows are handled.\n\npython3 {baseDir}/scripts/mac_use.py screenshot chrome\npython3 {baseDir}/scripts/mac_use.py screenshot chrome --id 4527\n\n: fuzzy, case-insensitive match (e.g. \"chrome\" matches \"Google Chrome\")\n--id N: target a specific window ID (required when multiple windows of the same app exist)\nReturns JSON with:\nfile: path to annotated screenshot (/tmp/mac_use.png)\nid, app, title, scale: window metadata\nelements: array of {num, text, at} — the numbered clickable text elements, where at is [x, y] center coordinates on the 1000x1000 canvas (origin at top-left)\nIf multiple windows match, returns a list of windows instead — pick one and retry with --id\nThe image is 1000x1000 pixels with green bounding boxes and blue number badges\nElement map is saved to /tmp/mac_use_elements.json for clicknum\nclicknum\n\nClick on a numbered element from the last screenshot. This is the primary click method.\n\npython3 {baseDir}/scripts/mac_use.py clicknum 5\npython3 {baseDir}/scripts/mac_use.py clicknum 12\n\nN: the element number from the last screenshot output\nReads the saved element map, activates the window, and clicks at the element's center\nReturns JSON with clicked_num, text, canvas coords, and absolute screen coords\nclick\n\nClick at a position using canvas coordinates. Fallback only — use for unlabeled icons.\n\npython3 {baseDir}/scripts/mac_use.py click --app chrome 500 300\npython3 {baseDir}/scripts/mac_use.py click --app chrome --id 4527 500 300\n\nCoordinates are canvas positions (0-1000) from the screenshot image\nx=0 is left, x=1000 is right; y=0 is top, y=1000 is bottom\nUse this only when Vision OCR didn't detect the element (icon-only buttons, images, etc.)\nscroll\n\nScroll inside an app window.\n\npython3 {baseDir}/scripts/mac_use.py scroll --app chrome down 5\npython3 {baseDir}/scripts/mac_use.py scroll --app notes up 10\n\nDirections: up, down, left, right\nAmount: number of scroll clicks (3-5 for moderate, 10+ for fast scrolling)\nMouse is moved to the center of the window before scrolling\ntype\n\nType text into the currently focused input field.\n\npython3 {baseDir}/scripts/mac_use.py type --app chrome \"hello world\"\npython3 {baseDir}/scripts/mac_use.py type --app chrome \"你好世界\"\n\n--app: activates the app first to ensure keystrokes go to the right window\nUses clipboard paste (Cmd+V) for reliable Unicode/CJK support\nAlways click on the target input field first before typing\nkey\n\nPress a single key or key combination.\n\npython3 {baseDir}/scripts/mac_use.py key --app chrome return\npython3 {baseDir}/scripts/mac_use.py key --app chrome cmd+a\npython3 {baseDir}/scripts/mac_use.py key --app chrome cmd+shift+s\n\n--app: activates the app first\nCommon keys: return, tab, escape, space, delete, backspace, up, down, left, right\nModifiers: cmd, ctrl, alt/opt, shift\nImportant Rules\nAlways screenshot before your first interaction with an app\nAlways screenshot after an action to verify the result\nAlways Read the screenshot image after running the screenshot command — you need both the element list AND the visual\nPrefer clicknum over click — only use direct coordinates for unlabeled icons\nClick before typing — ensure the correct input field has focus first\nMultiple windows: if you get multiple_windows error, use list to see all windows, then pass --id\nPopup windows (like WeChat mini-program panels) are separate windows with their own IDs — use list to find them and --id to target them\nWait after opening apps: use sleep 2-3 after open -a before taking a screenshot\nActivate the app before screenshot/click: prepend osascript -e 'tell application \"AppName\" to activate' && sleep 1 when the target app may be behind other windows\nDo not type passwords or secrets via this tool\nCoordinate System (for fallback click only)\n\nScreenshots are rendered onto a 1000x1000 canvas:\n\nOrigin (0, 0) is at the top-left corner\nx increases left to right (0 = left edge, 1000 = right edge)\ny increases top to bottom (0 = top edge, 1000 = bottom edge)\nThe app window is scaled to fit (aspect ratio preserved), centered, with dark gray padding\nExample: Order food on Meituan in WeChat\n# 1. Open WeChat\nopen -a \"WeChat\"\nsleep 3\n\n# 2. Screenshot WeChat — find the mini program window\npython3 {baseDir}/scripts/mac_use.py list\n# → find the mini program window ID\n\n# 3. Screenshot the mini program (annotated + element list)\npython3 {baseDir}/scripts/mac_use.py screenshot 微信 --id 41266\n# → returns: {\"file\": \"/tmp/mac_use.png\", \"elements\": [{num: 1, text: \"搜索\", at: [500, 200]}, ...]}\n# → Read /tmp/mac_use.png to see annotated image\n\n# 4. Click \"搜索\" (element #1)\npython3 {baseDir}/scripts/mac_use.py clicknum 1\n\n# 5. Type search query\npython3 {baseDir}/scripts/mac_use.py type --app 微信 \"炸鸡\"\n\n# 6. Press Enter\npython3 {baseDir}/scripts/mac_use.py key --app 微信 return\nsleep 2\n\n# 7. Screenshot to see results\npython3 {baseDir}/scripts/mac_use.py screenshot 微信 --id 41266\n# → Read /tmp/mac_use.png, pick a restaurant by number\n\n# 8. Click on a restaurant (e.g. element #5)\npython3 {baseDir}/scripts/mac_use.py clicknum 5" }, "trust": { "sourceLabel": "tencent", "provenanceUrl": "https://clawhub.ai/kekejun/mac-use", "publisherUrl": "https://clawhub.ai/kekejun/mac-use", "owner": "kekejun", "version": "1.0.0", "license": null, "verificationStatus": "Indexed source record" }, "links": { "detailUrl": "https://openagent3.xyz/skills/mac-use", "downloadUrl": "https://openagent3.xyz/downloads/mac-use", "agentUrl": "https://openagent3.xyz/skills/mac-use/agent", "manifestUrl": "https://openagent3.xyz/skills/mac-use/agent.json", "briefUrl": "https://openagent3.xyz/skills/mac-use/agent.md" } }