
想让 AI 智能体对一台 iPhone 做点什么?你可以尝试越狱设备并做大量软件 hack,或者……直接从货架上拿一块 microcontroller,把苦活交给它。[Henry Yuan] 演示了如何用一块 ESP32 实现这一目标。
第一步是让 AI 智能体获取手机当前屏幕状态——[Henry] 通过 AirPlay 屏幕镜像和 UxPlay 实现。随后用 Gstreamer 流水线把屏幕以 JPEG 流的形式喂给任意 AI 智能体。控制环节则由一块 ESP32-S3 完成,它伪装成 USB-HID 设备,同时模拟 USB 鼠标和键盘;配合开启 iOS 的辅助触控(AssistiveTouch),智能体就获得了对设备输入的控制权。接下来,就由 AI 智能体自己琢磨该执行哪些操作来达成目标了——这显然高度依赖具体用例。
这套方案听起来有些曲折,但好处明显:不需要 Apple 开发者账号,不需要 Xcode,甚至完全不需要 Mac。你可以让智能体直接在手机上折腾,省去所有这些麻烦。此前 Hackaday 也报道过其他有趣的 iPhone hack。
本文来源:https://hackaday.com/2026/10/14/ai-controls-iphone-via-esp32/
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。
Do you want to get an AI agent to do something to an iPhone? You could try and jailbreak the device and do some heavy software hacking, or you could grab a microcontroller off the shelf and have it do the hard work. Henry Yuan explains how to achieve this with an ESP32.

The first step is getting an AI agent access to the current state of the phone screen - which Henry achieves with AirPlay screen mirroring and UxPlay. A Gstreamer pipeline is then used to feed the screen as a JPEG stream to the AI agent of your choice. Then comes control, which is achieved by an ESP32-S3 acting as a USB-HID device, emulating both USB mouse and keyboard. With iOS AssistiveTouch enabled, this gives the agent control over inputs to the device. From there, it's up to the AI agent to figure out the actions to make to achieve the desired behavior, which is obviously very use-case specific.
It might sound a little convoluted, but this approach has benefits. There's no requirement to have an Apple Developer Account, or XCode, or a Macintosh at all. You can just get your agent tinkering away on the phone without any of that fussy overhead.