macOS agent Round 2: ScreenCaptureKit remote desktop + install/uninstall + CI #13

Closed

hex wants to merge 0 commits from feature/macos-agent-round2 into main

pull from: feature/macos-agent-round2

merge into: hex:main

hex:main

hex:feature/macos-loginwindow-remote

hex:fix/macos-pointer-calibration

hex:fix/macos-remote-helper-parity

Conversation 0 Commits - Files changed -

hex commented 2026-07-10 16:53:09 -04:00

Owner

Copy link

Summary

macOS agent Round 2: real remote desktop (ScreenCaptureKit capture + CGEvent
input), LaunchDaemon install/uninstall, .pkg packaging, and a
Forgejo-self-hosted CI job. Complements 0cc2c2d (server-side macOS
download/deploy plumbing, already on main) — that commit's author called
this branch out by name as the agent-side counterpart.

• Capture: MacCapture — ScreenCaptureKit main-display stream (BGRA→RGBA,
  ~30fps), bridged to the existing pull-based Capture::next_frame contract
  via a bounded tokio::mpsc channel. Empty displays() + actionable
  unavailable_reason() on missing Screen Recording permission or macOS
  <12.3, matching LinuxCapture's contract.
• Input: MacInput — CGEvent posted via kCGHIDEventTap, gated on
  AXIsProcessTrusted() (Accessibility). Deviated from the brief's suggested
  "probe with a no-op CGEventCreate" gate: that probe can't actually detect a
  missing grant (CGEventCreate* always succeeds regardless of
  authorization; CGEventPost returns void). Fixed US-layout CGKeyCode
  table for non-printable keys; Printable routes through
  CGEventKeyboardSetUnicodeString for correct Unicode text on any layout.
• Enrollment gate: refuses to enroll macOS <12.3 (ScreenCaptureKit's
  floor), client-side, before the redeem call.
• Install/uninstall: zeno-agent install/uninstall CLI subcommands
  (LaunchDaemon, root-required). Also fixed the pre-existing
  server-triggered self-uninstall path, which only knew about systemd and
  silently no-op'd on macOS instead of unloading the LaunchDaemon.
• Packaging: packaging/macos/build-pkg.sh → dist/zeno-agent-<version>-aarch64.pkg.
• CI: new macos-agent job in .github/workflows/ci.yml, adapted for
  this repo's actual Forgejo self-hosted setup (not the GitHub-hosted
  macos-latest + marketplace actions the original brief assumed — this repo
  has neither). Deliberately excluded from publish's needs: so it can't
  block the Windows/Linux/server pipeline while no runner carries the
  macos label yet.
• Docs: new docs/macos-agent.md (shipped/skipped scope, TCC
  permissions, verification checklist); docs/roadmap.md updated.

⚠️ Verification status — read before merging

This code has not been compiled. Authored on a Linux host with no Apple
toolchain — confirmed the hard way: even cargo check --target aarch64-apple-darwin fails before reaching this crate's code, because
ring's (transitive, unrelated) assembly build script needs a real Apple
cc. There is no way to get compiler feedback on objc2/ScreenCaptureKit/
CGEvent bindings from this box.

Verified:

• objc2 dependency graph resolves for aarch64-apple-darwin
  (cargo metadata --filter-platform).
• Linux/Windows builds unaffected — cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace --all-targets all pass on this host (all new macOS code is
  #[cfg(target_os = "macos")]-gated).
• Rebased cleanly onto 0cc2c2d with no conflicts.

Not verified (exact objc2-screen-capture-kit/objc2-core-graphics 0.3.2
binding signatures, .pkg build on real pkgbuild, launchctl
idempotency-string matching, whether ScreenCaptureKit's completion handlers
fire correctly without a Cocoa run loop) — see docs/macos-agent.md's
Verification status section for the full list. Registering a Forgejo
runner with the macos label and pushing is the fastest way to get real
compiler feedback — expect a fixup round.

Test plan

• Register a Forgejo self-hosted runner with the macos label
• Push to this branch, watch the macos-agent CI job for compile errors, fix
• sudo installer -pkg dist/zeno-agent-*.pkg -target / on a real M-series Mac
• sudo zeno-agent install, grant Screen Recording + Accessibility + Input Monitoring, sudo launchctl kickstart -k system/com.zeno.agent
• SPA → Devices → Connect: desktop appears, right-click, type into Spotlight, Cmd+Tab
• sudo zeno-agent uninstall, confirm LaunchDaemon/plist/binary gone

🤖 Generated with Claude Code

## Summary macOS agent Round 2: real remote desktop (ScreenCaptureKit capture + CGEvent input), LaunchDaemon install/uninstall, `.pkg` packaging, and a Forgejo-self-hosted CI job. Complements `0cc2c2d` (server-side macOS download/deploy plumbing, already on `main`) — that commit's author called this branch out by name as the agent-side counterpart. - **Capture**: `MacCapture` — ScreenCaptureKit main-display stream (BGRA→RGBA, ~30fps), bridged to the existing pull-based `Capture::next_frame` contract via a bounded `tokio::mpsc` channel. Empty `displays()` + actionable `unavailable_reason()` on missing Screen Recording permission or macOS <12.3, matching `LinuxCapture`'s contract. - **Input**: `MacInput` — CGEvent posted via `kCGHIDEventTap`, gated on `AXIsProcessTrusted()` (Accessibility). Deviated from the brief's suggested "probe with a no-op CGEventCreate" gate: that probe can't actually detect a missing grant (`CGEventCreate*` always succeeds regardless of authorization; `CGEventPost` returns void). Fixed US-layout `CGKeyCode` table for non-printable keys; `Printable` routes through `CGEventKeyboardSetUnicodeString` for correct Unicode text on any layout. - **Enrollment gate**: refuses to enroll macOS <12.3 (ScreenCaptureKit's floor), client-side, before the redeem call. - **Install/uninstall**: `zeno-agent install`/`uninstall` CLI subcommands (LaunchDaemon, root-required). Also fixed the pre-existing server-triggered self-uninstall path, which only knew about systemd and silently no-op'd on macOS instead of unloading the LaunchDaemon. - **Packaging**: `packaging/macos/build-pkg.sh` → `dist/zeno-agent-<version>-aarch64.pkg`. - **CI**: new `macos-agent` job in `.github/workflows/ci.yml`, adapted for this repo's actual Forgejo self-hosted setup (not the GitHub-hosted `macos-latest` + marketplace actions the original brief assumed — this repo has neither). Deliberately excluded from `publish`'s `needs:` so it can't block the Windows/Linux/server pipeline while no runner carries the `macos` label yet. - **Docs**: new `docs/macos-agent.md` (shipped/skipped scope, TCC permissions, verification checklist); `docs/roadmap.md` updated. ## ⚠️ Verification status — read before merging **This code has not been compiled.** Authored on a Linux host with no Apple toolchain — confirmed the hard way: even `cargo check --target aarch64-apple-darwin` fails before reaching this crate's code, because `ring`'s (transitive, unrelated) assembly build script needs a real Apple `cc`. There is no way to get compiler feedback on `objc2`/ScreenCaptureKit/ CGEvent bindings from this box. Verified: - `objc2` dependency graph resolves for `aarch64-apple-darwin` (`cargo metadata --filter-platform`). - Linux/Windows builds unaffected — `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace --all-targets` all pass on this host (all new macOS code is `#[cfg(target_os = "macos")]`-gated). - Rebased cleanly onto `0cc2c2d` with no conflicts. Not verified (exact `objc2-screen-capture-kit`/`objc2-core-graphics` 0.3.2 binding signatures, `.pkg` build on real `pkgbuild`, `launchctl` idempotency-string matching, whether ScreenCaptureKit's completion handlers fire correctly without a Cocoa run loop) — see `docs/macos-agent.md`'s **Verification status** section for the full list. **Registering a Forgejo runner with the `macos` label and pushing is the fastest way to get real compiler feedback** — expect a fixup round. ## Test plan - [ ] Register a Forgejo self-hosted runner with the `macos` label - [ ] Push to this branch, watch the `macos-agent` CI job for compile errors, fix - [ ] `sudo installer -pkg dist/zeno-agent-*.pkg -target /` on a real M-series Mac - [ ] `sudo zeno-agent install`, grant Screen Recording + Accessibility + Input Monitoring, `sudo launchctl kickstart -k system/com.zeno.agent` - [ ] SPA → Devices → Connect: desktop appears, right-click, type into Spotlight, Cmd+Tab - [ ] `sudo zeno-agent uninstall`, confirm LaunchDaemon/plist/binary gone 🤖 Generated with [Claude Code](https://claude.com/claude-code)

hex added 9 commits 2026-07-10 16:53:09 -04:00

feat(macos-agent): add objc2 dependency family for Round 2 ScreenCaptureKit/CGEvent backend ... 953da6485e

Uses objc2/objc2-foundation/objc2-app-kit/objc2-screen-capture-kit/
objc2-core-graphics/objc2-core-video/objc2-core-foundation/block2 (all
gated cfg(target_os = "macos")) instead of the older standalone
core-graphics/core-video crates named in the brief: the objc2-core-graphics
and objc2-core-video crates are versioned in lockstep with objc2 itself
(0.3.x <-> objc2 0.6.x) and stay API-consistent with objc2-screen-capture-kit,
whereas the legacy crates are a separate, independently-versioned family.
objc2 0.6.4 pinned (0.5's SCStreamConfiguration builder silently set
width=0, upstream issue #543). Verified the dependency graph resolves for
aarch64-apple-darwin and that Linux/Windows builds are unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

feat(macos-agent): implement ScreenCaptureKit capture + CGEvent input (Round 2) ... 4eadaca918

MacCapture: SCShareableContent -> main-display SCStream (BGRA, ~30fps,
queue depth 3) bridged to the existing pull-based Capture::next_frame
contract via a bounded tokio mpsc channel (ScreenCaptureKit itself is
callback-driven on a GCD queue; no existing capture backend in this repo
has an mpsc-producer pattern to mirror, so this is a new bridge, not a
copy of an existing one). BGRA->RGBA conversion mirrors the Windows DXGI
staging-texture swizzle/stride handling. Falls back to an empty
displays() + actionable unavailable_reason() on pre-12.3 macOS or a
missing Screen Recording grant, matching LinuxCapture's contract.

MacInput: CGEventCreateMouseEvent/CGEventCreateKeyboardEvent/
CGEventCreateScrollWheelEvent2 posted via kCGHIDEventTap, gated on
AXIsProcessTrusted() (Accessibility). Deviates from the brief's suggested
"probe with a no-op CGEventCreate" gate: CGEventCreate/CGEventCreateMouseEvent/
CGEventCreateKeyboardEvent always succeed regardless of Accessibility
authorization, and CGEventPost returns void, so that probe can't actually
detect a missing grant. AXIsProcessTrusted is Apple's documented
synchronous check instead, requiring a new `-framework ApplicationServices`
link in build.rs (AXIsProcessTrusted is a legacy Carbon/HIServices C API,
not covered by any objc2-* framework crate already in the dependency
graph). Fixed US-layout CGKeyCode table for non-Printable keys; Printable
keys go through CGEventKeyboardSetUnicodeString for correct Unicode text
regardless of layout.

VERIFICATION LIMIT: authored on a Linux host with no Apple toolchain
available (confirmed: cargo check --target aarch64-apple-darwin fails
before reaching this crate's code, because ring's asm build script needs
a real Apple cc). This has not been compiled. Exact objc2/objc2-screen-
capture-kit/objc2-core-graphics binding signatures are written from API
knowledge, not verified against generated docs — expect compiler-error
fixups on the first real build on the upcoming [self-hosted, macos] CI
runner. Linux/Windows builds confirmed unaffected (cargo check -p
zeno-agent passes; all new code is cfg(target_os = "macos")-gated).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

feat(macos-agent): refuse enrollment on macOS pre-12.3 ... 0f9fea2d7b

ScreenCaptureKit (Round 2's capture backend) requires macOS 12.3+; a
device that enrolls on an older macOS would enroll successfully but
remote desktop would never work, discovered later as a silent black
screen instead of an actionable message. Client-side pre-flight check
before the redeem call, using the version probe already added in
platform::macos for the capture-open gate — a rejected Mac never
contacts the server, so the error surfaces directly on the terminal
that ran `zeno-agent enroll` via anyhow's default error printing (main's
`fn main() -> anyhow::Result<()>`).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

feat(macos-agent): add LaunchDaemon install/uninstall CLI subcommands ... f84d497b75

`zeno-agent install [--binary PATH] [--config PATH]` (macOS-only, root-
required): copies the binary to /usr/local/bin/zeno-agent, writes
platform::macos::launchdaemon_plist() to
/Library/LaunchDaemons/com.zeno.agent.plist (chmod 644, chown root:wheel),
and runs `launchctl bootstrap system` — idempotent, falling back to
`kickstart -k` if already bootstrapped. `zeno-agent uninstall` reverses
all three steps via `launchctl bootout` + file removal, idempotent
against a partially-installed or already-removed daemon.

Mirrors Windows' Cmd::Install shape (both variants coexist in the Cmd
enum under mutually-exclusive #[cfg], same pattern already used for
Service/Launcher). launchdaemon_plist() and run_checked() promoted from
private to pub(crate)/pub since install.rs now calls them.

Also fixes a real gap found while wiring this: the existing server-
triggered self-uninstall path (uninstall.rs, used when an operator
removes a device from the portal) only knew about systemd — its unix
branch already guarded the systemd cleanup behind `command -v systemctl`
so it degraded to a silent no-op on macOS rather than erroring, but never
actually unloaded the LaunchDaemon or removed its plist. Added the
equivalent `launchctl bootout` + plist removal, guarded behind
`command -v launchctl` the same way.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

feat(macos-agent): add packaging/macos/build-pkg.sh for .pkg builds ... 4a6a17facf

cargo build --release --target aarch64-apple-darwin -p zeno-agent, stage
the binary + LaunchDaemon plist into a flat root, pkgbuild to
dist/zeno-agent-<version>-aarch64.pkg. Must run natively on a Mac (no
Apple cc/SDK on the Linux dev host, confirmed in the previous commits).

The staged plist is rendered by the freshly-built binary itself via a
new hidden `print-launchdaemon-plist` subcommand rather than a second
copy of the template in bash, so the .pkg's plist can never drift from
platform::macos::launchdaemon_plist(). Also hardened `zeno-agent install`
against being run against an already-in-place binary (the .pkg installs
straight to /usr/local/bin/zeno-agent; running `install` afterwards to
bootstrap the LaunchDaemon would otherwise try to copy that file onto
itself) — it now skips the copy when source and destination canonicalize
to the same path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

ci(macos-agent): add self-hosted macos job for aarch64-apple-darwin ... da8426d5da

Adapted from the Round 2 brief's assumption of GitHub-hosted macos-latest
+ actions/checkout/dtolnay/rust-toolchain/Swatinem/rust-cache — this repo
actually runs Forgejo Actions on self-hosted [windows]/[linux] runners
only, using code.forgejo.org action mirrors, with no macOS runner
registered and none of those marketplace actions used anywhere in this
repo. User decision: write the job for Forgejo self-hosted with a
`macos` runner label (matching the existing windows/linux jobs'
conventions) rather than a GitHub-hosted job that can't run here at all.

Every push/PR: cargo check -p zeno-agent --target aarch64-apple-darwin
--all-targets (rustup toolchain ensured the same way the linux-agent job
does it; Homebrew ffmpeg installed first since ffmpeg-next's build
script needs pkg-config resolution even for `cargo check`). On push to
main: also runs packaging/macos/build-pkg.sh and uploads the binary +
.pkg as workflow artifacts.

Deliberately NOT added to `publish`'s `needs:` — the Windows/Linux/
server pipeline must keep shipping regardless of macOS runner
availability, since no runner carries the `macos` label yet (this job
sits queued/unscheduled on Forgejo until one is registered) and there's
no macOS download artifact wired into the portal to publish anyway.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

test(macos-agent): add unit tests for Round 2 helpers ... 17d4cb02b1

parse_macos_version (full/partial/garbage version strings), a new pure
version_meets_minimum helper split out of supports_screen_capture_kit
so the 12.3 gate is testable without shelling out to sw_vers, cg_keycode
spot-checks (including that Printable panics rather than silently
mapping — MacInput::key must route it through
CGEventKeyboardSetUnicodeString instead), unavailable-reason string
content, and MacCapture::unavailable()'s displays()/unavailable_reason()
contract. dscl parser and launchdaemon_plist tests (Round 1) untouched
and still passing.

These tests live inside the same #[cfg(target_os = "macos")]-gated file
as everything else in this module, so — like the rest of Round 2 — they
cannot run until the macos-agent CI job has a real runner; verified only
that they're self-consistent and that the crate's Linux/Windows test
suites are unaffected (cargo test -p zeno-agent passes on this host).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

docs(macos-agent): add docs/macos-agent.md, update roadmap for Round 2 ... 8d9de7e031

New docs/macos-agent.md: what's shipped (Round 1 + Round 2), what's
explicitly skipped (GUI installer, multi-monitor, audio, code signing),
the three TCC permissions an operator must grant (Screen Recording,
Accessibility, Input Monitoring) and why, and a prominent Verification
status section stating plainly that none of the Round 2 objc2/
ScreenCaptureKit/CGEvent code has been compiled — authored on a Linux
host with no Apple toolchain available (proven in earlier commits: even
cargo check --target aarch64-apple-darwin fails before reaching this
crate's code). Includes the manual hardware verification checklist from
the original brief.

roadmap.md: subsystem map now lists platform/macos.rs and install.rs
(was silently absent, describing the agent as "Windows + Linux" only,
stale since Round 1 already shipped); Slice 1 section gets a macOS
bullet pointing at the new doc plus a "still open" item for hardware
verification once a runner exists.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

style(macos-agent): cargo fmt ... 79df30cad3

CI's quality gate runs cargo fmt --all -- --check; run it now so the
formatting-only diff doesn't ride along with a future functional commit.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

hex added 1 commit 2026-07-10 19:15:47 -04:00

ci: rerun macOS agent on native host runner 72bfd5a55a

hex added 1 commit 2026-07-10 19:18:32 -04:00

ci: retry macOS native runner with launchd PATH a57efeb865

hex added 1 commit 2026-07-10 19:36:59 -04:00

fix(macos-agent): address native CI compile errors c7c22a2548

hex added 1 commit 2026-07-10 19:37:50 -04:00

chore(macos-agent): update lockfile for CoreMedia bindings 5cf7f5cee1

hex added 1 commit 2026-07-10 19:42:58 -04:00

fix(macos-agent): keep capture open future Send 1c8d661eb9

hex added 1 commit 2026-07-10 20:02:37 -04:00

ci(macos-agent): shallow checkout on native runner 81fdd5ac61

hex added 1 commit 2026-07-10 20:16:25 -04:00

ci(macos-agent): use GitHub-hosted artifact build 8c04856b75

hex added 1 commit 2026-07-10 20:22:36 -04:00

ci(macos-agent): link Homebrew lame for mac artifact 493e0cab1c

hex added 1 commit 2026-07-10 20:38:49 -04:00

ci(macos-agent): link libc++ for ffmpeg-sys-next static C++ libs 8a91ad1537

hex added 1 commit 2026-07-10 20:42:11 -04:00

ci(macos-agent): use self-hosted macos runner, remove github-only workflow ... 45943b9f64

- Route to Forgejo [self-hosted, macos] runner (user's hardware)
- Add RUSTFLAGS=-lc++ for ffmpeg-sys-next C++ static libs
- Add LIBRARY_PATH for Homebrew lame
- Always build+package+upload (no main-only gate)
- Delete macos-agent-artifact.yml (GitHub-hosted workflow)
- Remove if: github.server_url guard

hex added 1 commit 2026-07-10 20:46:36 -04:00

ci(macos-agent): reinstall ffmpeg to fix stale Homebrew Cellar paths 76ce8ff743

hex added 1 commit 2026-07-10 21:03:41 -04:00

ci(macos-agent): noninteractive brew reinstall ffmpeg for current dep paths 780f727c99

hex added 1 commit 2026-07-10 21:25:09 -04:00

ci(macos-agent): pipe yes to brew reinstall to bypass interactive prompt 4dd29c4a9f

hex added 1 commit 2026-07-10 21:28:18 -04:00

ci(macos-agent): use brew opt symlinks to avoid stale Cellar paths ... a67a540535

- Drop brew reinstall (caused SIGPIPE exit 141 with yes pipe)
- Set LIBRARY_PATH to stable /opt/homebrew/opt symlinks for libvmaf,
  openssl@3, lame, and the main lib dir
- Set PKG_CONFIG_PATH to stable pkgconfig dirs
- This avoids stale versioned Cellar paths (e.g. libvmaf/3.1.0/lib)
  that break when Homebrew upgrades dependencies

hex referenced this pull request from a commit 2026-07-11 15:51:54 -04:00

feat(macos): merge Round 2 ScreenCaptureKit + CGEvent + install/uninstall

hex referenced this pull request from a commit 2026-07-11 15:51:54 -04:00

feat(macos): merge PR #13 — ScreenCaptureKit remote desktop + install/uninstall + Round 2 parity

hex closed this pull request 2026-07-11 19:47:27 -04:00

All checks were successful

Hide all checks

CI / Linux — musl agent (pull_request) Has been skipped

Details

CI / macOS — agent build + package (pull_request) Successful in 2m12s

Details

CI / Windows — fmt, clippy, test, agent (pull_request) Successful in 10m10s

Details

CI / Publish server image (pull_request) Has been skipped

Details

CI / Redeploy dev server (SSH to hermes-vm) (pull_request) Has been skipped

Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

bug

Something isn't working

  

documentation

Improvements or additions to documentation

  

duplicate

This issue or pull request already exists

  

enhancement

New feature or request

  

good first issue

Good for newcomers

  

help wanted

Extra attention is needed

  

invalid

This doesn't seem right

  

question

Further information is requested

  

wontfix

This will not be worked on

No labels

bug

documentation

duplicate

enhancement

good first issue

help wanted

invalid

question

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

hex/zeno-rmm!13

No description provided.