IT之家 10 月 18 日消息,据科技媒体 TechSpot 今天报道,索尼最近申请一项专利,描绘以下情形:让 AI 在 GPU 完成渲染前预测下一帧画面,减少游戏画面卡顿,同时避免帧生成工艺带来的额外延迟。
IT之家从原报道获悉,这项工艺可以应对帧生成的缺点,让表明画面不再等待两帧画面完成渲染,才能生成插入其中的中间帧。而是直接利用已经生成的画面预测下一帧,无需等待两帧画面就能插值,协助游戏维持更稳定的帧输出节奏。
同时,索尼使用了联合训练的编解码器,将渲染完成的画面转换为嵌入表示,对图像开展紧凑的数值化表达,让解码器利用这些提示生成预测帧。游戏运行时,该平台会与图形渲染管线并行工作。硬件渲染下一帧时,AI 会处理上一帧,并将预测结果发送至表明缓冲区。因而在下一帧完成渲染之前,预测画面就有可能先行表明。
另外,该平台还可利用运动矢量和玩家近期的手柄操作。使玩家操作角色的历史,成为下一帧动作预测的根基。该平台训练过程使用连续的视频帧。神经网络最初使用随机初始化的权重,以一帧画面作为输入,并尝试预测下一帧。随后,平台会将预测结果与实际的后续画面开展比较,并计算一个用于衡量预测误差的损失值。
不过这项工艺能否发挥实际作用,取决于 AI 能否迅速地生成准确画面,从而在渲染速度下降时填补帧输出间隙。
本文来源:https://www.ithome.com/1/014/615.htm
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。
IT House Oct. 18, TechSpot reported today that Sony recently applied for a patent that depicts the following scenario: letting AI predict the next frame before the GPU finishes rendering, reducing game picture stuttering, while avoiding additional delays caused by the frame generation process.
IT House learned from the original report that this process can cope with the shortcomings of frame generation, so that the display screen can no longer wait for two frames to be rendered before generating an intermediate frame inserted into it. Instead, it directly uses the already generated picture to predict the next frame, and can interpolate without waiting for two frames to help the game maintain a more stable frame output rhythm.

At the same time, Sony uses a jointly trained codec to convert the rendered picture into an embedded representation, and carries out a compact numerical expression on the image, allowing the decoder to use these prompts to generate prediction frames. When the game is running, the platform works in parallel with the graphics rendering pipeline. When the hardware renders the next frame, the AI processes the previous frame and sends the prediction to the indication buffer. Therefore, before the next frame is rendered, it is possible to show the prediction screen first.
In addition, the platform can also be operated using motion vectors and the player's recent paddles. Enables the player to manipulate the character's history to become the basis for the next frame of motion prediction. The platform training process uses consecutive video frames. The neural network initially uses randomly initialized weights, takes one frame of screen as input, and tries to predict the next frame. The platform then compares the prediction with the actual subsequent picture and calculates a loss value to measure the prediction error.

However, whether this process can make a practical difference depends on whether the AI can quickly generate accurate images to fill the frame output gap when the rendering speed decreases.
This article is from: https://www.ithome.com/1/014/615.htm For copyright issues, please write to: 13951431913#139.com or submit a review on the right.