早就听说虾哥开源了一个基于 ESP32 的小智 AI 聊天助手,一直没机会认真研究一下。赶一下边缘ai的热潮。这次有机会通过 Moji2.0 硬件来学习一下。
源码获取
先去 GitHub 上扒下来源码:
https://github.com/78/xiaozhi-esp32
下载后解压,里面主要是 4 个文件夹,分别是文档、程序、Flash 分区表配置、脚本。
主要看程序这一块,结构分布大致是这样:
main/
├── application.h/cc # 主应用类
├── assets.h/cc # 资源管理
├── ota.h # OTA升级
├── settings.h # 配置管理
├── system_info.h # 系统信息
├── device_state.h # 设备状态枚举
├── device_state_machine.h # 状态机
├── mcp_server.h # MCP服务器(机器人控制)
├── audio/ # 音频服务
│ ├── audio_service.h/cc
│ ├── audio_processor.h
│ ├── audio_codec.h
│ ├── wake_word.h
│ ├── processors/ # 音频处理器(AFE等)
│ ├── codecs/ # 音频编解码器驱动
│ └── wake_words/ # 唤醒词引擎
├── display/ # 显示模块
│ ├── display.h
│ ├── lvgl_display/ # LVGL图形库
│ ├── oled_display.h
│ └── emote_display.h # Emote显示协议
├── protocols/ # 网络协议
│ ├── protocol.h
│ ├── mqtt_protocol.h/cc
│ └── websocket_protocol.h/cc
├── led/ # LED控制
│ ├── led.h
│ ├── single_led.h
│ └── circular_strip.h
└── boards/ # 硬件板级支持
├── common/ # 通用板类
│ ├── board.h
│ ├── wifi_board.h
│ └── ml307_board.h # 4G模块支持
├── esp-box-3/ # 具体板卡配置
├── electron-bot/ # 机器人平台
└── ... (30+种板卡)
整体架构
从结构上来看,这个项目的层级是这么划分的:
- Application(主应用层) :状态管理、事件处理、网络协调、OTA升级、MCP服务
- Core Services(核心服务层) :音频服务、网络协议、资源管理、显示输出
- Board Abstraction(硬件抽象层) :音频编解码、指示灯、网络接口、摄像头
- ESP32 Hardware(底层硬件)
设计的架构相当清晰,各个模块的职责划分得很清楚。而且相关资料也很丰富,小智有个类似官网百科全书的地址:
https://my.feishu.cn/wiki/F5krwD16viZoF0kKkvDcrZNYnhb 这里有很多相关技术文档。还有专门帮助入门的,比较友好
上手操作
如果用的小智官方已经支持的硬件,那流程很简单:搭建好编译下载环境后,去 boards 文件夹找到对应的板子文件夹,然后让 AI 根据这个文件夹内容帮你编译下载就行了。
烧录好程序后,根据配网流程完成网络设置。然后可以在小智的控制后台进行各种配置:
这里可以配置模型、设计人设、声纹识别,甚至还能自定义界面UI。
改点东西
基础流程跑通之后,就该动动自己的需求了。我这次主要想实现两个功能:
- 增加一个 ADC 温度采集
- 重新设计待机界面
用 Cursor 配合 AI 基本上就能搞定。这种硬件层面的改动,主要是在板级文件夹下操作就行。
不过有个忠告:如果像我一样对这个项目还不太熟悉,建议先别动框架内的核心代码,哪怕是用 AI 生成代码也要克制。我一开始试着改了改框架里的一些东西,结果出现了各种奇葩问题——内存溢出、声音突然消失、界面卡顿等。这些问题排查起来还挺耗时。
所以想动 AI 相关部分的话,建议先花点时间把项目结构理解透彻再动手。
最终效果
最后展示一下这次修改后的效果,包括自定义的 UI 界面,以及其他功能,小智 AI 整体表现还是很流畅的。
请在微信客户端打开
I'd heard for a while that Brother Xia had open-sourced a XiaoZhi AI chat assistant based on ESP32, but never had the chance to dig into it seriously. I wanted to jump on the edge AI trend, and this time I got the opportunity to learn through the Moji2.0 hardware.
Source Code Acquisition
First, go to GitHub to grab the source code:
https://github.com/78/xiaozhi-esp32
After downloading and extracting, you'll mainly find four folders: Documentation, Program, Flash Partition Table Configuration, and Scripts.
Let's focus on the Program folder. The structure is roughly as follows:
Overall Architecture
In terms of structure, the project is organized into these layers:
Application (Main Application Layer): State management, event handling, network coordination, OTA updates, MCP services
Core Services: Audio services, network protocols, resource management, display output
Board Abstraction (Hardware Abstraction Layer): Audio codec, indicator LEDs, network interface, camera
ESP32 Hardware (Low-Level Hardware)
The designed architecture is quite clear, with well-defined responsibilities for each module. The documentation is also quite comprehensive—XiaoZhi has something like an official wiki:
https://my.feishu.cn/wiki/F5krwD16viZoF0kKkvDcrZNYnhb This page contains many relevant technical documents, including beginner-friendly guides.
Getting Started
If you're using hardware officially supported by XiaoZhi, the process is simple: set up the compilation and flashing environment, find the corresponding board folder in the boards directory, and then let the AI help you compile and flash based on that folder's contents.
After flashing the program, complete the network configuration following the pairing process. Then you can make various settings on the XiaoZhi control dashboard:
Here you can configure the model, design personas, set up voiceprint recognition, and even customize the UI.
Making Modifications
Once the basic flow is working, it's time to customize things to fit my needs. This time I mainly wanted to implement two features:
Add an ADC temperature sensor
Redesign the standby screen
Using Cursor paired with AI made this mostly straightforward. For hardware-level changes, you mainly operate within the board-specific folder.
However, a word of advice: if you're unfamiliar with the project like I was, avoid touching the core framework code—even when using AI-generated code. I initially tried modifying some parts of the framework, and it resulted in all sorts of weird issues: memory overflow, audio suddenly disappearing, UI lag, etc. Troubleshooting these problems was quite time-consuming.
So if you want to modify the AI-related sections, I recommend spending time thoroughly understanding the project structure before making changes.
Final Results
Here's a showcase of the results after my modifications, including the custom UI and other features. XiaoZhi AI overall performed very smoothly.
Please open in WeChat client





