IT之家 9 月 28 日消息,Meta 宣布将于今年秋季起陆续为旗下雷朋 Display 智能眼镜 / Quest 头显及全新轻薄型头显上线“Hologram”全新虚拟形象能力,这项工艺类似苹果 Vision Pro 的 Persona,可以借助生成式 AI 构建与真人外观高度接近的虚拟形象。
事实上,Meta 早在 7 年前就展示了名为 Codec Avatars 的拟真虚拟形象项目,不过当下消费者可以买到的 Meta VR 机型仍主要使用较为卡通化的自定义形象角色。按照当下的规划,后续 Meta 会逐步将使用者的自定义形象角色从卡通化转向拟真写实化。
据介绍,使用者首次使用 Hologram 时,需要打开手机上的 Meta AI 应用,并按照提示转动头部、微笑等。随后平台会提出一些开放式难题,使用者需要开展较长时间的回答。在回答过程中,平台会采集使用者的面部表情和说话状态,用于训练和调整 Hologram 模型,同时还会记录使用者的服装和背景。
整个创建过程只需几分钟,完成服务器端处理后即可生成 Hologram。之后,当使用者佩戴 Meta 雷朋 Display 眼镜接听或拨打 WhatsApp 视频电话时,对方看到的将不再是传统摄像头画面,而是由 AI 生成的使用者面部虚拟形象。
从工艺角度来看,Meta 雷朋 Display 版本的 Hologram 使用运行在 Meta 服务器上的实时生成式 AI 扩散模型,以眼镜麦克风采集的音频作为输入,并实时生成 2D 视频流发送给通话另一端。Hologram 的面部表情主要依据使用者说话时的声音变化生成,而不是直接使用摄像头捕捉真人画面。
而在 Quest 平台维度,相应 Hologram 能力使用 3D 等身立体形式展现,当两名 Quest 使用者借助 WhatsApp 开展视频通话时,双方首先会在一个 3D 窗口中看到对方的 Hologram。点击窗口旁边的按钮后,通话可以切换至全身模式,对方的虚拟形象即会以接近真人的尺寸浮现在使用者面前。
不过,当下版本仍存在一些改进空间,例如人物尺寸可能与真人存在轻微差异,眼睛等细节在部分情况下会显得较为粗糙。同时如果使用者向侧面倾斜过多,虚拟人物就会变形,因而还有待开发人员开展改善。
本文来源:https://www.ithome.com/1/007/767.htm
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。
IT之家 news on September 28: Meta announced that starting this fall, it will gradually roll out the new "Hologram" virtual avatar capability for its Ray-Ban Display smart glasses / Quest headsets and new lightweight headsets. This technique, similar to Apple Vision Pro's Persona, leverages generative AI to build virtual avatars that closely resemble real people.
In fact, Meta demonstrated a realistic virtual avatar project called Codec Avatars as early as 7 years ago, but the Meta VR models currently available to consumers still mainly use fairly cartoonish customizable avatar characters. According to current plans, Meta will gradually shift users' customizable avatar characters from cartoonish to realistic.
According to the introduction, when using Hologram for the first time, users need to open the Meta AI app on their phone and follow the prompts to turn their head, smile, and so on. The platform then poses some open-ended questions, and users need to provide extended responses. During the answering process, the platform captures the user's facial expressions and speaking state to train and adjust the Hologram model, while also recording the user's clothing and background.
The entire creation process takes only a few minutes, and the Hologram is generated after server-side processing is complete. Afterward, when users wearing Meta Ray-Ban Display glasses answer or place WhatsApp video calls, the other party will no longer see traditional camera footage, but an AI-generated virtual avatar of the user's face.
From a technical perspective, the Meta Ray-Ban Display version of Hologram uses a real-time generative AI diffusion model running on Meta servers, taking audio captured by the glasses' microphone as input and generating a real-time 2D video stream sent to the other end of the call. Hologram's facial expressions are mainly generated based on changes in the user's voice while speaking, rather than directly using a camera to capture real-person footage.
On the Quest platform, the corresponding Hologram capability is presented in a 3D life-size stereoscopic form. When two Quest users conduct a video call via WhatsApp, both parties first see the other's Hologram in a 3D window. After clicking the button next to the window, the call can switch to full-body mode, and the other party's virtual avatar appears before the user at a size close to a real person.
However, the current version still has room for improvement, such as the character's size potentially differing slightly from a real person, and details like the eyes appearing somewhat rough in some cases. Also, if users tilt too much to the side, the virtual character will deform, so there is still a need for developers to make improvements.

