智能眼镜开源软硬平台!
模块化硬件 + 低功耗处理器 + 事件相机,为可穿戴AI研究提供一个开放的硬件平台
智能眼镜被认为是下一代人机交互的终极载体,它处于用户视线的最前沿,可以感知用户所见、所听。
但要让它真正智能,却面临着三个相互制约的挑战:功耗、算力和体积。
想要功能强大,就得塞进高性能芯片和大容量电池,眼镜就会变得笨重。
想要轻便续航长,又往往只能把数据传到云端处理,引发隐私和延迟问题。
现有的开源硬件平台,要么算力不足,只能处理简单的传感器数据。要么传感器固定,无法灵活更换,要么功耗太高,撑不过几个小时。
最近,ETH Zurich 团队了发布一个完全开源的智能眼镜平台OpenGlass,专为事件驱动型边缘AI设计。
所有硬件设计、固件代码、模型权重,全部公开。研究者可以自由修改、扩展和定制。
一、OpenGlass是什么?
OpenGlass 是一套开源、模块化的智能眼镜参考设计。
它不仅提供了完整的硬件图纸和固件,还训练了一个手势识别的视觉模型,并在眼镜上实现了33.9毫秒的端到端推理延迟。
它采用了事件相机作为核心视觉传感器。与传统相机不同,事件相机只记录画面中变化的像素,输出稀疏的事件流。
整个系统的核心设计思路是:模块化 + 双域架构 + 边缘计算。
-
模块化:将传感器与主控分离,无需重新设计整块PCB就能更换或增加相机模块。
-
双域架构:一个超低功耗的处理器始终待命,一个高性能的芯片按需唤醒。
-
边缘计算:所有AI推理都在眼镜上本地完成,只把最终结果通过蓝牙发出去。
这种设计让原型机在本地持续运行AI推理时,能以330 µW/GOPS的能效运行视觉模型,续航达到了11.8小时。
二、硬件架构:效率域 + 性能域
OpenGlass 的硬件分为两块核心电路板:
主控板:塞在镜腿里,集成了两颗处理器。GAP9主要负责跑神经网络,nRF5340 负责系统配置、电源管理和蓝牙传输。200mAh 的电池藏在另一个镜腿里。
柔性传感器板:通过排线连接主控板,承载了Prophesee GENX320事件相机和一颗备用的HIMAX HM0360 RGB相机。
为了省电,OpenGlass 将电路分为两个域:
1. 效率域始终低功耗待命,可被运动或声音唤醒。
nRF5340协调器:一颗超低功耗的Cortex-M33芯片,负责系统配置、电源管理、蓝牙传输。
加速度计和麦克风:配置为事件驱动的唤醒源。只有当检测到用户有动作或发出声音时,才会唤醒高性能域。
2. 性能域只在需要时通电,推理完成后立即关闭。
GAP9 RISC-V SoC:集成了9个核心和一个AI加速器,能以330 µW/GOP的能效运行神经网络。
Prophesee GENX320事件相机:没手势时,几乎没有数据产生,大幅降低了功耗。
三、手势识别实战,33.9ms延迟
为了验证平台的实用性,团队在一个第一人称视角的事件相机手势数据集上测试了手势识别流程。
在13.3Hz的推理频率下,系统平均功耗65.6mW,200mAh电池可提供11.8小时的持续推理。
OpenGlass 的所有资源在论文正式发布后都会开源,包括PCB原理图、机械图纸等硬件设计;nRF5340和GAP9的完整固件代码,你可以修改电源管理策略、添加新传感器;训练好的神经网络模型权重你可以直接部署,也可以用自己的数据重新训练。
具体可以查看论文及代码:
开源地址:https://arxiv.org/pdf/2606.07431最后,小伙伴们可留言回复‘加群’进入大模型交流群、视觉应用落地交流群!
想涨知识,点击下方卡片,关注码科智能!
//VLM大模型文章推荐超越YOLO-World!Meta发布开放词汇目标检测新框架Meta又一巨作!可用于姿势估计、深度估计、身体分割等任务
CV被大模型解决了?腾讯开源VLM:十余种视觉和多模态任务全拿下!
点个“小爱心”吧
Open Hardware and Software Platform for Smart Glasses!
Modular hardware + low-power processor + event camera, providing an open hardware platform for wearable AI research.
Smart glasses are considered the ultimate carrier of next-generation human-computer interaction, positioned at the forefront of the user's line of sight, capable of perceiving what the user sees and hears.
But making them truly intelligent faces three interrelated challenges: power consumption, computing power, and form factor.
To pack in powerful features, you need to fit high-performance chips and large-capacity batteries, making the glasses bulky.
To keep them lightweight with long battery life, data often has to be sent to the cloud for processing, raising privacy and latency concerns.
Existing open-source hardware platforms either lack sufficient computing power and can only process simple sensor data, have fixed sensors that cannot be flexibly swapped, or consume too much power to last more than a few hours.
Recently, an ETH Zurich team released OpenGlass, a fully open-source smart glasses platform designed for event-driven edge AI.
All hardware designs, firmware code, and model weights are made publicly available. Researchers can freely modify, extend, and customize them.
I. What is OpenGlass?
OpenGlass is an open-source, modular reference design for smart glasses.
It not only provides complete hardware schematics and firmware, but also trained a gesture-recognition vision model achieving an end-to-end inference latency of 33.9 ms on the glasses.
It uses an event camera as its core visual sensor. Unlike traditional cameras, an event camera only records pixels that change, outputting a sparse stream of events.
The overall system design philosophy is: modularity + dual-domain architecture + edge computing.
Modularity: Sensors are decoupled from the main controller, allowing camera modules to be swapped or added without redesigning the entire PCB.
Dual-domain architecture: An ultra-low-power processor stays on standby, while a high-performance chip wakes up on demand.
Edge computing: All AI inference runs locally on the glasses, with only final results sent out via Bluetooth.
This design allows the prototype to run vision models at an energy efficiency of 330 µW/GOPS during continuous local AI inference, achieving 11.8 hours of battery life.
II. Hardware Architecture: Efficiency Domain + Performance Domain
OpenGlass's hardware is divided into two core circuit boards:
Main control board: Housed in one temple, integrating two processors. GAP9 mainly handles neural networks, while nRF5340 manages system configuration, power management, and Bluetooth transmission. A 200 mAh battery is hidden in the other temple.
Flexible sensor board: Connected to the main control board via a flexible cable, carrying a Prophesee GENX320 event camera and a backup HIMAX HM0360 RGB camera.
To save power, OpenGlass splits the circuit into two domains:
1. Efficiency Domain: Always runs at ultra-low power and can be awakened by motion or sound.
nRF5340 coordinator: An ultra-low-power Cortex-M33 chip responsible for system configuration, power management, and Bluetooth transmission.
Accelerometer and microphone: Configured as event-driven wake-up sources. The high-performance domain is only awakened when user movement or sound is detected.
2. Performance Domain: Powered on only when needed, and shut down immediately after inference completes.
GAP9 RISC-V SoC: Integrates 9 cores and an AI accelerator, capable of running neural networks at 330 µW/GOP energy efficiency.
Prophesee GENX320 event camera: Generates almost no data when no gestures are present, significantly reducing power consumption.
III. Gesture Recognition in Action: 33.9 ms Latency
To validate the platform's practicality, the team tested the gesture recognition pipeline on a first-person-view event-camera gesture dataset.
At an inference frequency of 13.3 Hz, the system's average power consumption is 65.6 mW, and the 200 mAh battery provides 11.8 hours of continuous inference.
All OpenGlass resources will be open-sourced upon the paper's formal publication, including PCB schematics, mechanical drawings, and other hardware designs; complete firmware code for nRF5340 and GAP9 — you can modify power management strategies and add new sensors; trained neural network model weights that you can deploy directly or retrain with your own data.
For details, please refer to the paper and code:
Finally, leave a comment with "join group" to enter the Large Model Exchange Group and the Computer Vision Application落地 Exchange Group!
Want to expand your knowledge? Click the card below to follow Makke Intelligent!
// VLM Article Recommendation — Surpassing YOLO-World! Meta Releases Open-Vocabulary Object Detection Framework; Meta's Another Major Work! Capable of Pose Estimation, Depth Estimation, Body Segmentation, and More
Has CV Been Solved by Large Models? Tencent Open-Sources VLM: Mastering Over a Dozen Vision and Multimodal Tasks!
Give us a "little heart"!





