Open source · MIT · macOS

Let your agent operate your Mac

An agent in a terminal can edit code, but it cannot see the Safari tab you have open or press a button in another app. Cueward is a CLI that observes and operates macOS apps, Safari, Apple's built-in apps and your files, and returns JSON the agent can act on.

View on GitHubcrates.io

What it can do

Operate apps in the background

List windows across Spaces, read a window's accessibility tree, press buttons, set text fields, type, send keys, scroll, click and drag. Cueward does not move your pointer or bring the app to the front, so you keep working while the agent operates another window.

Wait and verify

Wait for an element to appear, become enabled or reach a value, compare window snapshots before and after an action, and launch apps or move windows between Spaces.

Safari

Read tabs, page text and source, run JavaScript, click, fill, wait, inspect elements, read console and network logs, and manage bookmarks, in the Safari you are already signed into.

ChatGPT and Gemini through Safari

Send prompts, start Gemini Deep Research, read conversations and save generated images using your own signed-in Safari session.

Notes, Reminders, Calendar

Create, update and delete notes, reminders and events, plus Quick Notes and Stickies. Read Voice Memos metadata. Reminders and Calendar reads use EventKit.

Files

Scoped browsing, bounded reads, search, Spotlight, previews, Finder tags and iCloud status. Copy, move, rename, batch rename and trash run with revision checks and verified backups; nothing is permanently deleted.

Screen, OCR and clipboard

Capture the screen or one window, extract text from images and PDFs with Vision OCR (Traditional and Simplified Chinese, English, Japanese), and read or write the clipboard.

Local memory and Shortcuts

Capture recent Safari, Notes and Messages content, tag it and search it from a local index. Build, edit and run Apple Shortcuts from the CLI or spec files. Read Reddit through its public JSON.

How it works

  • Native macOS APIs: Accessibility, SQLite reads, AppleScript, EventKit, Vision OCR and Shortcuts.
  • Local-first: local data is read on your Mac with no scraping proxy and no third-party data backend. Commands that talk to ChatGPT, Gemini or Reddit send requests to those services.
  • Cueward does not call any LLM. It returns JSON, and your agent decides what to do with it.
  • Actions return their result status. When Cueward cannot confirm an effect, it says so, and the agent observes again before retrying.

Current limits

  • macOS only. The Windows adapter is an unpublished stub.
  • Background window operation needs Accessibility access and the Swift toolchain. Canvas-only apps, a locked screen and some first-click or inactive-Space cases are not verified.
  • Command availability depends on the installed version. Check cueward --help.

Install

Requires macOS and Rust 1.88 or later.

cargo install cueward-cli --locked
cueward doctor

cueward doctor lists missing permissions. The terminal app running Cueward needs Full Disk Access; Notes, Reminders, Calendar and Finder also need Automation, and window operation needs Accessibility.

Agent skill

The repository ships a cueward-agent skill for Claude Code and Codex that tells the agent which command fits the task and how to verify the result. The skill is instructions only; install the CLI separately.

Works with Termdock, does not need it

Cueward is a separate project from the Termdock team and is not bundled with the Termdock app. Agents in Termdock terminals can call it, and so can agents in Terminal.app, iTerm2 or WezTerm.