
本文来源:https://www.worldprogramming.org/posts/building-a-touchscreen-macro-deck-on-an-esp32-p4-with-lvgl-96-ehs2c4
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。
作者想要一块类似 Stream Deck 的宏键盘,但不想付 Stream Deck 的价钱,于是基于一块 GUITION 的 ESP32-P4 开发板自己做了一块。这篇文章讲的是面板固件与上位机协议背后的设计取舍,供想在嵌入式显示屏上做类似项目的人参考。封面图是用桌面控制台 CAD 模型做的 Blender 渲染,实物外壳还没打印出来;下文那些面板截图也不是照片,而是真实 UI 代码在 PC 上原生编译后绘制的,agents 页面用的是示例数据。
UI 代码在 PC 上用与固件相同的 LVGL 源码原生编译,测试程序喂入触摸输入并渲染 1024×600 的帧。布局问题几秒钟就能暴露,不用烧录、不用接线,还能顺手给 README 生成不用斜着拍屏幕的截图。代价是原生构建不会覆盖显示驱动、触摸控制器和 PSRAM 限制,这些只能上板验证;作者的定位很明确:原生构建管 UI 逻辑,板子负责暴露硬件意外。
面板只负责绘制与上报触摸,动作执行和编辑器都放在 PC 上一个轻量 Python 上位机里,由它占用 USB Serial/JTAG 端口并在 127.0.0.1 提供服务,两边用 115200 波特率的行协议通信。每条消息是一行 UTF-8:上位机发文本命令或一个 JSON 对象,面板应答并主动以 JSON 行上报事件,比如按键事件里只有长按才会出现 hold 字段。按键到底做什么全部由 PC 决定,因此新增动作无需改固件,固件也能保持很小。
这套设计也有边界:面板最多读取 1,099 字节的一行,超长直接忽略,所以上位机把布局切成 380 字符的块、图像切成 160 字节的块发送;按键在收到上位机应答前保持高亮,没有上位机时会立刻闪红。PC 关机时面板保留最后一帧布局。触屏最怕重建:如果仪表数值到达时正好在按键上重建网格,手指就落在一个从未收到按下事件的新对象上,所以作者采用原地更新——标签、颜色和仪表数值都改在已有 LVGL 对象上,只有换页或布局变化才重建网格,控件同理,新增控件只是上位机的一个功能而不需要新固件。
图标是唯一不能当文本处理的东西:上位机以 RGBA 加 CRC32 校验逐块下发,面板只在结束时回复 OK、INVALID 或 FULL,FULL 表示缓存放不下这张图。图标缓存占 PSRAM 中的 8 MiB,需要空间时淘汰最久未使用且当前没有任何按键正在显示的图片;面板发现缺图就发一个 image 请求让上位机重传,动图则按帧带时长下发。CRC32 让损坏的图标被判为 INVALID 而不是直接显示出来——作者认为显示一个坏图标比缺一个图标更烦人。
其余细节:Claude Code 请求授权时,允许与拒绝按钮会出现在面板上,agents 页面还能显示 Claude Code、Codex 和 ZCode 正在做什么、在哪个项目里;安装脚本会往 ~/.claude/settings.json 加 hook,保留原有 hook 并先做备份,hook 最多等 25 秒触摸,面板关机、无人应答或内容过长都会回退到 Claude Code 原本的对话框,Tessera 自己从不擅自放行。渲染上用 PSRAM 中两块 1024×240 的部分绘制缓冲、一个软件绘制单元和 15 ms 刷新周期,6×3 网格因形状固定而手工布局,字体做压缩。限制也很清楚:仅支持 Windows 10/11,只适配 JC1060P470C 这一块板,没有实体按键所以少了真实按钮的手感,未做代码签名因此首次运行可能被 SmartScreen 拦截(发布包附 SHA256SUMS)。面板本身没有网络、只通过 USB 与上位机通信,编辑器与控制 API 只监听 127.0.0.1。项目代码与发布版见 github.com/catorendal-a11y/tessera-deck(v0.9.0);顺带一提,这篇文章由 AI 智能体 Claude Code 依据 Tessera 仓库撰写,每个事实都与代码核对过。

本文来源:https://www.worldprogramming.org/posts/building-a-touchscreen-macro-deck-on-an-esp32-p4-with-lvgl-96-ehs2c4
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。
I wanted a Stream Deck-style macro pad without paying for a Stream Deck, so I built one on a GUITION...
This post covers the design choices behind the panel firmware and the host protocol, for anyone building something similar on an embedded display.
The cover image is a Blender render made from the CAD model of the desk console. I have no photo of a printed one yet. The panel screenshots below are not photos either: the native PC build of the real UI code drew them, and the agents screen uses sample data.
The UI code compiles natively on the PC against the same LVGL sources as the firmware. A test feeds it touch input and renders 1024 x 600 frames.
A layout bug shows up in seconds, with no flashing and no cable. I can also make screenshots for the README without photographing a screen at an angle.
The catch is that the native build does not exercise the display driver, the touch controller or PSRAM limits. Those still need the board. I treat the native build as the place for UI logic and the board as the place for hardware surprises.
The panel only draws and reports touches. A small Python host on the PC owns the USB Serial/JTAG port, runs the actions and serves the editor on 127.0.0.1. The two talk over a line protocol at 115200 baud.
Each message is one UTF-8 line. The host sends text commands or one JSON object. The panel answers, and it also sends its own events as JSON lines. A key press looks like this, copied from the protocol page at v0.9.0:
hold is only present for a long press. Everything else about what the key does lives on the PC. That means I can add an action without touching the firmware, and the firmware stays small.
There are limits to this design. The panel reads lines of up to 1,099 bytes and ignores a longer one, so the host sends a layout in 380-character chunks and an image in 160-byte chunks. A pressed key stays highlighted until the host answers, so without a host it flashes red at once. If the PC is off, the panel keeps the last layout on screen and the keys do nothing.
A touchscreen punishes rebuilds. If a gauge value arrived and the grid were rebuilt while you press a key, your finger would be on a new object that never saw the press, so I update in place.
Now labels, colours and gauge values change on the existing LVGL objects. The grid is only rebuilt when the page or its layout changes. A key is never rebuilt under your finger.
The same idea covers widgets. The host sends a title, a second line and a progress value, and the panel draws every wide key the same way. A new widget is a new host-side feature and needs no new firmware.
Icons are the one thing that is too big to treat like text. The host sends each icon as RGBA with a CRC32 check, using these lines:
The panel only answers at the end: OK , INVALID or FULL . FULL means the cache cannot hold the image.
The cache is 8 MiB in PSRAM. When it needs room it drops the least recently used image that no key is showing. If the panel needs an image it no longer has, it sends {"type":"image","id":"im:…"} and the host sends it again. Animated icons arrive as frames with a duration each.
The CRC32 lets the panel answer INVALID for a corrupt image instead of showing it, and a corrupt icon is more annoying than a missing one.
When Claude Code wants permission to run something, the question appears on the panel with Allow and Deny buttons. The agents screen also shows what Claude Code, Codex and ZCode are doing and in which project.
Setup adds hooks to ~/.claude/settings.json . It keeps your other hooks and makes a backup first. The hook waits up to 25 seconds for a touch. If the panel is off, nobody answers in time, or the request is too long to show, Claude Code falls back to its normal dialog. Tessera never allows anything on its own. If nothing answers, you answer at the keyboard as before, so the panel cannot block your work.
The screenshot below is drawn by the native PC build and uses sample data.
The panel uses two partial draw buffers of 1024 x 240 pixels in PSRAM, one software draw unit and a 15 ms refresh period. The 6 x 3 key grid is placed by hand because the shape is fixed. Fonts are compressed. The icon cache takes its 8 MiB from the same PSRAM as the draw buffers.
It runs on Windows 10 and 11 only. The host is Python and the actions are Windows actions.
It supports one board. The pin setup and the screen layout are written for the JC1060P470C, so another board needs a port.
There are no physical keys. You lose the feel of a real button.
It is not code-signed. Windows SmartScreen may warn on first start. The release includes SHA256SUMS so you can compare the zip before you run it.
The agents and mixer screenshots show sample data, and the console images are renders.
In the release the panel has no network and speaks only to the host over USB. The editor and the control API listen on 127.0.0.1 only.
Code and releases: github.com/catorendal-a11y/tessera-deck (v0.9.0)
I would like feedback on the in-place updates and the icon cache. If you want to port it to another ESP32-P4 display, CONTRIBUTING.md says what it takes.
This post was written by an AI agent (Claude Code) from the Tessera repo, and every fact in it was checked against the code. Tessera itself was also built with help from Claude Code.
No comments yet. Be the first to share your thoughts.