
学术生涯一夜之间被颠覆,学者们将不得不在 OpenAI 继续推进之际,从粗制滥造中甄别出真正成果。
“令人瞠目结舌。”“铺天盖地。”“史无前例。”“恍如梦境。”“完全疯狂。”
这些说法,出自三十多位数学家本周接受《The Verge》采访时脱口而出的评价。他们试图理解 OpenAI 突然向数学界抛出的大量成果。面对这场洪流带来的惊叹、兴奋与不确定感,人们内心深处还涌动着一种焦虑:这一切究竟意味着什么,接下来又会发生什么。尽管反应各异,研究人员普遍认为,仅理解 OpenAI 发布的内容就可能耗时数年,更不用说弄清楚数学家自身在这个正被改变的领域中将处于何种位置。许多人担心,OpenAI 不会等那么久就会继续推进,甚至放出更多成果。
OpenAI 总共发布了近 400 项由人工智能生成的成果,这些成果分布在 700 多份手稿中,涵盖组合数学、几何学的多个分支、数论、理论计算机科学、代数、拓扑学、概率论与统计力学以及数学物理等广泛数学领域。这批成果规模庞大,以至于 OpenAI 认为有必要发布指南,帮助人们在这座庞杂的 GitHub 仓库中检索导航。
成果数量如此庞大,以至于人们甚至难以对其究竟发布了什么作出初步评估。发布后的数小时乃至数天内,多数接受《The Verge》采访的数学家表示,他们仍在努力消化所有内容;几位学者说,仅通读约 40 页的目录和摘要,就花了将近一个小时。康涅狄格大学数学教授阿尔瓦罗·洛萨诺-罗夫莱多说:“光是浏览整份摘要清单,就令人应接不暇。”
在这数百份手稿中,还夹杂着一些用 Lean 完成的“形式化”证明。Lean 是一种编程语言兼证明辅助工具,可让成果通过计算机进行验证。这些形式化证明在评估 OpenAI 此前的一些数学主张时发挥了关键作用,使研究人员即便尚未完全理解其背后的论证,也能确信相关结论在逻辑上成立。
“如果人工智能此刻突然消失,仿佛外星人造访地球后径直离去,我们接下来十年都会研究这一现象,试图弄清一切。”
但各项结果经过验证的程度差异极大。在 GitHub 上,OpenAI 承认这些结果“处于不同验证阶段”,且“许多手稿已被形式化,但并非全部”。截至发稿时,该合集中的手稿似乎只有不到一半被正式描述。OpenAI 表示,在 719 份手稿中,只有 300 项主要结果已被形式化,约为 42%,并称“随着我们获得更多形式化结果,将更新代码库”。
多位数学家向 The Verge 抱怨形式化不足,尤其是在结果数量庞大的情况下;他们强调,即使有 Lean 代码随附结果,评估也并非瞬间完成。研究人员必须检查形式化是否确实证明了结果所声称的内容,这又是一项耗时的过程。一些翻阅论文的研究人员表示,即便提供了可由计算机验证的证明,其质量也参差不齐,证明所验证的陈述并不总是与随附手稿中的主张完全对应。
伦敦帝国理工学院数学教授凯文·巴扎德表示,他在自己的研究领域——代数数论——中发现了大量定理,但只有大约六个立即“脱颖而出”。这些定理中几乎没有在 Lean 中得到形式化验证。他说:“因此,我要么只能阅读可能并不正确的‘垃圾内容’,要么等待别人做同样的事,要么等待有人对其进行形式化,然后才能确定这些结果是否真的正确。”巴扎德的担忧得到了许多其他研究人员的共鸣。
巴扎德对 AI “垃圾内容”的担忧绝非孤例。这一术语是低质量、频繁出错的 AI 生成内容的简称,这类内容在网上——乃至现实世界中——越来越多地出现,学术论文也不例外。与其他领域一样,数学家们告诉 The Verge,近年来他们看到使用 ChatGPT 和 Claude 等工具生成的此类材料大幅增加。其中许多内容令人困惑、难以阅读,并且几乎看不出对主题的理解;在引用其他研究人员方面尤其粗劣。
OpenAI 此前发布的数学文章因粗制滥造遭到专家广泛批评,尤其是引用不当或完全没有引用。在发布前与 The Verge 的交流中,一些研究人员开始将即将到来的论文洪流称为“垃圾内容末日”,或类似的说法。
人们担心的“垃圾末日”是否真的降临,很难说,这在很大程度上是因为发布材料的数量令人眼花缭乱。早期迹象表明,OpenAI 这次在论文方面似乎更加用心,至少在某些论文上如此。几位数学家告诉 The Verge,他们的第一印象远好于预期,尽管他们自己也承认,考虑到该公司此前粗制滥造的出版物,这原本也不是什么高标准。
“这是一团糟。它可能导致学术文化的巨大崩塌,并直接扼杀大多数院系。这是一个社会问题,而 AI 实验室似乎完全忽视了这个议题。”
但“更好”并不一定意味着“好”,更不用说达到通常对提出此类重大主张的学术工作的要求。在许多 OpenAI 的主张缺乏形式化验证的情况下,配套论文的质量就变得尤为重要;它们是数学家确认、理解、审查和定位这些结果的主要途径。
即使在最好的情况下,撰写严谨的数学论文也是一项艰巨的工作。以这种规模做到这一点更是一项艰巨任务。OpenAI 的模型正以令人眩晕的速度、在广泛的专业领域产出数学成果,远远超出了其人类员工的处理能力。该公司在数学前沿领域并不具备足够的专业知识广度或资源来妥善审查其发现。研究人员告诉 The Verge,这一点显露无遗。
许多人描述这些论文艰涩难懂,有时几乎无法跟进。“我了解最清楚的那个问题的论文,快速读完后几乎不知所云,”布朗大学数学教授布伦丹·哈塞特(Brendan Hassett)告诉 The Verge。“如果这是由人写的,我不会再花时间去理解它。当然,这还剩下 721 篇其他预印本!”(哈塞特是在 OpenAI 撤回三篇论文之前说这番话的。)
一些研究人员告诉 The Verge,他们或同事注意到的几篇论文似乎重复了其他数学家已经走过的领域,但在有机会妥善审查材料之前,他们不愿公开这样说。另一些人则指出这些论文异常简短,通常预期要用数百页篇幅展开的结果被压缩到几十页甚至更少。
澳大利亚悉尼大学数学教授纳利尼·乔希(Nalini Joshi)表示,她快速检索这批发布内容后发现,与她自己研究重叠的部分很少,但她指出,她查看的一些论文“参考文献很短”。鉴于外界此前曾批评OpenAI的署名做法,她说自己“担心论文中的引用归属可能没有完整呈现来龙去脉。”
尽管其中不乏粗糙内容和不确定性,但总体共识是,这批发布成果中确实包含一些令人印象深刻的工作。
尽管强调评估如此大量材料的困难,以及妥善验证结果的必要性,但多位接受The Verge采访的研究人员表示,尽管呈现方式存在不足,这些工作看起来仍具有非常高的水准。他们说,在人工智能出现之前,OpenAI的许多成果显然足以在顶级期刊发表,甚至足以让作者奠定学术生涯。其中少数成果被描述为足以让数学家成为菲尔兹奖的有力竞争者,而菲尔兹奖是该领域最高荣誉之一。
“我要么去读可能并不正确的粗制滥造内容,要么等别人先做同样的事,要么等有人把它们形式化之后,我才能确定这些结果是否真的正确。”
斯坦福大学数学家贾里德·杜克·利希特曼(Jared Duker Lichtman)表示,有“数十项”成果可归入这一类,包括黎曼假设方面的进展、霍奇猜想的一个特例,以及四维挂谷猜想的解。这些是数学中的重大问题。黎曼假设——可以说是整个学科中最令人望而生畏的未解难题——和霍奇猜想都属于著名的七大千禧年大奖难题。粗略地说,前者关注素数分布,后者则涉及如何用更简单的构件来理解复杂几何形状。与此同时,挂谷猜想大致是在问:要让一根针或一支铅笔在所有方向上旋转,需要多少空间。三维挂谷猜想的证明是纽约大学数学家王虹今年早些时候获得菲尔兹奖的成果之一。
数学家斯科特·阿姆斯特朗(Scott Armstrong)说:“这些并非没人听说过的琐碎问题。其中许多都是非常著名的问题,很多人已经研究了几十年。”
随着尘埃落定,数学家们不得不面对一个骤然改变的局面。许多人刚刚眼睁睁看着多年心血和精心制定的研究计划瞬间化为泡影,或知晓同事经历了同样的命运。整个学科领域内,无论人们的反应是兴奋、恐惧、绝望,还是介于其间,都弥漫着一种强烈的迷失感。
到了第二天上午,这种情绪并未明显改变。阿姆斯特朗一边在巴黎街头步行,一边通过电话表示,他设想,如果索邦大学没有因持续的学生抗议而关闭,同事们仍聚集在惯常的咖啡机旁,现场气氛恐怕会相当“凝重”。
在苏格兰,圣安德鲁斯大学数学教授科尔瓦·罗尼-杜加尔也描述了同事和学生间类似的阴郁气氛。她说,这场“洪水”般的信息洪流多少令人“惊骇”,但在数周的不确定之后,它的到来反而带来了一些释然。“我有很多朋友和同事,他们的基金申请刚刚被一笔勾销。”
“我有很多朋友和同事,他们的基金申请刚刚被一笔勾销。”
阿姆斯特朗表示,他了解到有一个研究团队,其整个研究计划几乎被这一发布“彻底抹除”。与此同时,曾处于OpenAI此前围绕纳维-斯托克斯问题争议中心的纽约大学数学家特里斯坦·巴克马斯特说,他已经听说“有三个人整个研究计划都被摧毁”。
在《The Verge》与数学家的多次交流中,类似的故事反复出现,尽管冲击主要集中在该领域的某些方向。苏格兰赫瑞-瓦特大学数学教授弗朗切斯科·富尼耶-法乔表示,概率论、组合数学和理论计算机科学领域的研究者似乎“尤其受到震动”,并补充说,他的研究领域群论中的某些方向已被“推土机铲平”。
阿姆斯特朗和其他研究人员表示,某些领域似乎几乎以军事般的精准度被锁定为目标。“确实存在一些目标,”阿姆斯特朗说。他特别指出王今年获得菲尔兹奖的相关领域,以及若干千禧年大奖难题。他表示,数学物理领域的一系列成果清楚地表明,OpenAI 正在研究杨-米尔斯理论——支撑现代粒子物理学大部分内容的数学框架——及其尚未解决的“质量间隙”问题,这是七大千禧年难题之一。
在其他方面,研究人员对自己的工作似乎受到的影响如此之小表现出一种惊讶的宽慰。罗尼-杜格尔表示,她所处的研究领域一隅——主要围绕群论——似乎相对毫发无损。她开玩笑说,幸运的是她从事的是一个“非常不热门的领域”,不过后来她发消息称,在发现自己的一项研究被其中一篇论文引用后,她感到“不确定”。
乔希同样发现自己的工作与之重叠很少;她的研究主要聚焦于可积系统,即能够精确求解的复杂系统。她认为,这或许是因为她的领域较少由长期存在、形式化表述的猜想和定义所驱动,而更多受到随着物理学发展而起伏变化的问题推动。
截至10月8日,该记录已列出大量更正,包括对十几篇手稿的修订,以及因“符号错误”导致论证失效而撤下三篇论文。
无论自己的工作是否受到直接影响,《The Verge》采访的大多数数学家都有一种共同感受:这个领域已经越过某种门槛,不再是一天前的样子。普林斯顿高等研究院研究员康斯坦丁·科格勒称这是“数学史上最重大的单一时刻”;伦敦数学科学学会研究员何杨辉(Yang-Hui He)则将目光投向数千年前,把今年由AI推动的进展比作欧几里得《几何原本》的出版——该学科最具影响力的著作之一。
很少有研究人员的评价如此宏大,但人们普遍认为数学正在快速变化——数学家也必须随之改变。
OpenAI 知道此次发布将带来冲击,并已采取了一些措施来缓和影响、与数学界展开沟通。今年早些时候与研究人员发生数次激烈交锋后,该公司转而向数学家寻求帮助,并与新成立的“数学与人工智能咨询小组”(AGMAI)合作,探讨如何负责任地发布成果。
AGMAI 此前已阐明其认为负责任的互动应当是什么样子。理想情况下,AI 实验室应发布“人类能够理解的论文”,或者为人类理解并吸收其 AI 生成的数学成果、识别其可能应用所必需的额外数学活动提供必要支持。该组织表示,在可行的情况下,证明应当被形式化,企业也应公开用于生成这些证明的模型和提示词。该小组还敦促 AI 实验室不要把数学成果发布当作“推广模型的宣传工具”,并应“停止在专有模型上测试高级数学问题”,因为这些模型无法向更广泛的科学界开放。
“这些并不是那种无人听说过的琐碎问题。其中许多都是广为人知、许多人已经尝试解决了几十年的经典问题。”
OpenAI 采纳了该小组的一些建议。该公司表示,将资助一系列围绕其数学研究工作的研讨会和会议,帮助数学界消化和理解其成果,但没有说明这些活动将以何种形式举行或何时举办。与以往发布相比,该公司披露了多得多的信息——研究人员再次指出,这其实门槛很低——包括其模型尝试了超过 4000 个问题,以及一项典型结果大约使用了三小时的 ChatGPT Pro “思考”算力。该公司还实施了“论文修订和引用协议”,并在 GitHub 上表示将“保留该合集的公开发布历史”。截至 10 月 8 日,该记录已列出大量更正,包括对十几篇手稿进行修订,以及因一个“符号错误”导致论证失效而撤下三篇论文。对于 The Verge 要求提供更多细节的置评请求,OpenAI 未作公开回应。
本文来源:https://www.theverge.com/ai-artificial-intelligence/1008726/openai-mathematics-solutions-chaos
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。
Careers upended overnight, academics will have to separate solutions from slop while OpenAI moves on.
“Staggering.” “Overwhelming.” “Unprecedented.” “Surreal.” “Pure insanity.”
Those were among the descriptions more than three dozen mathematicians reached for in conversations with The Verge as they tried to make sense of the flood of mathematical results OpenAI abruptly dropped on the field this week. Amid the awe, excitement, and uncertainty over the sheer scale of the deluge was a deep-seated anxiety over what it all means — and what comes next. For all their different reactions, researchers agreed that simply understanding what OpenAI had released could take years, let alone figuring out where the mathematicians themselves fit in the field now changing around them. Many feared OpenAI would not wait that long before moving on — or releasing even more.
In all, OpenAI released nearly 400 AI-generated results. These were spread across more than 700 manuscripts and covered a diverse array of mathematical disciplines, including combinatorics, several branches of geometry, number theory, theoretical computer science, algebra, topology, probability and statistical mechanics, and mathematical physics. The collection is so vast that OpenAI felt the need to publish guidance on how to navigate the sprawling GitHub repository.
The sheer volume of work makes even a preliminary assessment as to exactly what the company has released difficult. In the hours and days following the drop, most mathematicians The Verge spoke with said they were still struggling to digest everything; several said that simply working through the roughly 40-page table of contents and abstracts took them the better part of an hour. “Just going over the entire list of abstracts is overwhelming,” said Álvaro Lozano-Robledo, a professor of mathematics at the University of Connecticut.
Sprinkled among the hundreds of manuscripts are formalizations in Lean, a programming language and proof assistant that allows results to be verified computationally. These formalizations have proven instrumental in assessing some of OpenAI’s previous mathematical claims, giving researchers confidence that a claim is logically correct even if they don’t fully understand the argument behind it.
“If the AIs would disappear now, as though there were aliens that came to Earth and then just left, we would be studying this for the next 10 years, trying to understand everything.”
But the degree to which each result had been verified varied wildly. On GitHub, OpenAI acknowledged that the results are “at different stages of verification” and that “many, but not all, of the manuscripts have been formalized.” As of writing, fewer than half the manuscripts in the collection appear to have been described formally. OpenAI said only 300 top-line results out of 719 manuscripts had been formalized, around 42 percent, and that it “will update the repository with more formalizations as we obtain them.”
Several mathematicians complained to The Verge about the lack of formalization, particularly given the sheer number of results, and stressed that even when Lean code accompanies a result, evaluation isn’t instantaneous. Researchers must check that the formalization actually proves what the result claims, another time-consuming process, and several digging through the papers said that even where computer-verifiable proofs had been provided, the quality was inconsistent and the statements they verified did not always appear to map neatly onto the claims in accompanying manuscripts.
Kevin Buzzard, a mathematics professor at Imperial College London, said he had identified numerous theorems in his area of work — algebraic number theory — of which only around six immediately “stood out.” Few, if any, of those appeared to be formally verified in Lean. “Hence, I either have to read possibly-not-correct slop, or wait for others to do the same, or wait for someone to formalise them before I can say for sure that the results are even correct.” Buzzard’s concerns were echoed by numerous other researchers.
Buzzard was far from alone in worrying about AI “slop.” The term is a shorthand for low-quality, frequently erroneous AI-generated material that increasingly crops up online — and in the real world — including academic papers . As in other fields, mathematicians told The Verge they have seen a huge uptick in such material produced with tools like ChatGPT and Claude in recent years. Much of it is confusing, hard to read, and demonstrates little understanding of the subject; it is especially shoddy when it comes to crediting other researchers.
OpenAI’s previous mathematical write-ups were widely criticized by experts for their sloppy nature, particularly their poor or nonexistent attribution . In conversations with The Verge ahead of the release, several researchers had taken to calling the impending flood of papers the “slopocalypse,” or similar variations on the theme.
Whether the feared “slopocalypse” actually materialized is difficult to say, largely due to the bewildering volume of material released. Early indications suggest OpenAI took more care with papers this time around, or at least with some of them. Several mathematicians told The Verge that their first impressions were far better than they had expected, though by their own admission that was hardly a high bar given the company’s previous shoddy publications.
“It’s a big mess. It can cause a huge collapse in the academic culture and simply kill most of the faculties. It’s a social problem and it seems that the AI labs are completely ignoring this issue.”
But better does not necessarily mean good, let alone up to the standards usually expected of academic work making claims of this magnitude. With formal verification absent for many of OpenAI’s claims, the quality of the accompanying papers becomes particularly important; they are the primary means by which mathematicians can confirm, understand, scrutinize, and contextualize the results.
Producing rigorous mathematical papers is difficult work under even the best of circumstances. Doing so at this kind of scale is a formidable undertaking. OpenAI’s models are pumping out mathematics at a dizzying speed and across a broad range of specialities, far outstripping the capacity of its human staff. The company simply does not have the breadth of expertise or resources to properly scrutinize its findings at the cutting edge of mathematics. Researchers told The Verge that it shows.
Many described papers that were difficult, sometimes practically impossible, to follow. “The write-up of the problem I know best made little sense after a quick read,” Brendan Hassett, a mathematics professor at Brown, told The Verge . “If this had been written by a person, I wouldn’t spend any more time trying to understand it. Of course, this leaves 721 other preprints!” (Hassett said this before OpenAI retracted three papers).
Some researchers told The Verge several papers they or their colleagues had noticed appeared to cover ground already trod by other mathematicians, though were wary of saying so publicly before they had a chance to properly review the material. Others pointed to the unusual brevity of the work, with results they would ordinarily expect to be developed over hundreds of pages compressed into a few dozen or less.
Nalini Joshi, a mathematics professor at the University of Sydney in Australia, said a quick search through the release revealed little that overlapped with her own work, but noted that some of the papers she examined “have short bibliographies.” Given previous criticism of OpenAI’s crediting practices, she said she is “wary that attribution in the papers may be lacking the complete story.”
But for all the slop and uncertainty, the overarching consensus is that the release contains some genuinely impressive work.
While stressing the difficulties assessing the volume of material — and the need to properly verify the results — numerous researchers The Verge spoke to said the work appeared to be of a very high caliber, despite shortcomings in its presentation. In a pre-AI world, they said, many of OpenAI’s results would clearly have warranted publication in top-tier journals and could have been enough to secure an academic career for their authors. A handful were described as being the kind of work that could make a mathematician a serious contender for a Fields Medal, one of the discipline’s highest honors.

“I either have to read possibly-not-correct slop, or wait for others to do the same, or wait for someone to formalise them before I can say for sure that the results are even correct.”
Stanford mathematician Jared Duker Lichtman said there were “tens” of results he would put in this category, including progress toward the Riemann hypothesis, a special case of the Hodge conjecture, and a solution to the four-dimensional Kakeya conjecture. These are major problems in mathematics. Riemann — arguably the most notorious unsolved problem in the entire discipline — and Hodge are both among the seven famed Millennium Prize problems . Respectively, they are concerned with the distribution of prime numbers and, very roughly, how complex geometric shapes can be understood in terms of simpler building blocks. Kakeya, meanwhile, roughly asks how little space is needed to rotate a needle or pencil in every direction. A proof to the three-dimensional version of Kakeya was among the achievements NYU mathematician Hong Wang was awarded a Fields Medal for earlier this year.
“It’s not the case that these are just silly problems that no one’s ever heard of,” mathematician Scott Armstrong said. “Many of them are like very well-known problems that many people have tried for decades.”
As the dust began to settle, mathematicians were left confronting a landscape that had abruptly changed around them. Many of them had just watched years of work and carefully laid research plans evaporate in an instant, or knew colleagues who had. Across the field, whether reactions were laced with excitement, dread, despair, or something in between, there was a profound sense of disorientation.
By the following morning, that mood had not altered much. Speaking by phone as he walked through Paris, Armstrong imagined that, had the Sorbonne not been closed amid ongoing student protests and his colleagues were gathered around their usual coffee machine, the atmosphere would have been rather “solemn.”
In Scotland, Colva Roney-Dougal, a professor of mathematics at St Andrews University, described a similarly bleak atmosphere among colleagues and students. The deluge felt somewhat “horrific,” she said, but its arrival brought some relief after weeks of uncertainty. “I have a bunch of friends and colleagues whose grant proposals have just been wiped out,” she said.
“I have a bunch of friends and colleagues whose grant proposals have just been wiped out.”
Armstrong said he knew of one group whose entire research program was practically “wiped out” by the release. Tristan Buckmaster, an NYU mathematician who was at the center of OpenAI’s earlier dispute over the Navier-Stokes problem , meanwhile, said he had already heard of “three people who had entire research programs obliterated.”
Similar stories surfaced repeatedly in The Verge ’s conversations with mathematicians, though the disruption was concentrated heavily in some areas of the field. Francesco Fournier-Facio, a professor of mathematics at Heriot-Watt University in Scotland, said researchers in probability, combinatorics, and theoretical computer science appeared “particularly in shock,” adding that some regions of his field, group theory, had been “bulldozed.”
Armstrong and other researchers said some areas appeared to have been aimed at with almost military precision. “There were definitely some targets,” said Armstrong. He singled out work in the area Wang received the Fields Medal for this year, as well as several Millennium Prize problems. A cluster of results in mathematical physics, he said, makes it clear OpenAI is pursuing Yang-Mills theory — the mathematical framework underpinning much of modern particle physics — and its unresolved “mass gap” problem, one of the seven Prize problems.
Elsewhere, researchers expressed a kind of surprised relief at how little their own work appeared to have been touched. Roney-Dougal said her own corner of the field — largely centered on group theory — appeared to have escaped relatively unscathed. She joked that it is fortunate she works in a “very unfashionable area,” though later messaged to say she was feeling “unsure” after finding her work cited in one of the papers.
Joshi similarly found little overlap with her work, which is largely focused on integrable systems, complex systems that can be solved exactly. She suggested that may be because her field is less driven by long-standing, formally stated conjectures and definitions and more by questions that wax and wane with developments in physics.
As of October 8th, that record already listed numerous corrections, including revisions on more than a dozen manuscripts and the removal of three papers due to a “sign error” it says invalidates an argument.
Whether their own work had been directly affected or not, most mathematicians The Verge spoke to shared a sense that the field had crossed some kind of threshold and was no longer the same as it had been a day earlier. Constantin Kogler, a researcher at the Institute for Advanced Study, called it “the most important single moment in the history of mathematics,” while Yang-Hui He, a fellow at the London Institute for Mathematical Sciences, reached back millennia, comparing AI-driven developments this year to the publication of Euclid’s Elements , one of the discipline’s most influential works.
Few researchers were quite so grand in their assessments, but there was a general consensus that mathematics was changing fast — and that mathematicians would have to change with it.
OpenAI knew the release would be disruptive and had made some effort to soften the blow and engage with the mathematical community. After several bruising encounters with researchers earlier this year, the company turned to mathematicians themselves for help, working with the newly formed Advisory Group on Mathematics and Artificial Intelligence (AGMAI) on how to release the results responsibly.
AGMAI had already laid out what it believed responsible engagement would be. AI labs should ideally release papers “that a human understands” or otherwise provide the necessary “support for the additional mathematical activities that are needed for humans to be able to understand and assimilate their AI-generated mathematical output and identify possible applications of it.” They said that, where possible, proofs should be formalized, and that companies should make public what models and prompts they used to create them. The group also urged AI labs to stop treating mathematical releases as “marketing vehicles to promote their models” and should “stop testing advanced mathematical problems on proprietary models” that are inaccessible to the broader scientific community.
“It’s not the case that these are just silly problems that no one’s ever heard of. Many of them are like very well-known problems that many people have tried for decades.”
OpenAI followed some of the group’s advice. It said it would be funding a series of workshops and conferences around its work in mathematics to help the community process and understand its work, though it provided no details as to what these might look like or when they may occur. The company also disclosed significantly more information than it had in previous releases — again, researchers said this was a low bar — including that its model attempted more than 4,000 problems and that a typical result used around three hours of ChatGPT Pro thinking compute. It has also implemented “protocols for paper revisions and citations” and on GitHub said it “will preserve the public release history” of the collection. As of October 8th, that record already listed numerous corrections, including revisions on more than a dozen manuscripts and the removal of three papers due to a “sign error” it says invalidates an argument. OpenAI did not respond on the record to The Verge ’s request for more details.