今天，AI 领域传出一封重要的公开信。

包括 「ChatGPT 之父」Sam Altman 在内 350 位 AI 权威都签署了这封联名公开信，并对当前正在开发的 AI 技术可能会对人类构成生存威胁担忧。

这封信只有一句声明：降低人工智能灭绝的风险，应该与其它社会规模的风险（如大流行病和核战争）一样，成为全球的优先事项。

加入公开信签名🔗
https://www.safe.ai/statement-on-ai-risk

签署人包括三个领先的 AI 公司高管：

• OpenAI 首席执行官 Sam Altman；

• Google DeepMind 首席执行官 Demis Hassabis；

• Anthropic 的首席执行官 Dario Amodei；

更重要的是，名单里还有 Geoffrey Hinton 和 Yoshua Bengio 两位人工智能的「教父」。

超过 350 名从事人工智能工作的高管、研究人员和工程师签署了这份由非盈利组织人工智能安全中心发布的公开信，认为人工智能具备可能导致人类灭绝的风险，应当将其视为与流行病和核战争同等的社会风险。

如今人们越来越担心 ChatGPT 等人工智能模型的发展可能对社会和就业带来的潜在威胁，很多人呼吁针对 AI 行业进行更强力的监管，否则它将对社会造成不可挽回的破坏。

AI 不断狂飙，但相关的监管、审计手段却迟迟没有跟上，这也意味着没有人能够保证 AI 工具以及使用 AI 工具的过程中的安全性。

在上周，Sam Altman 和另外两位 OpenAI 的高管提出应该建立类似国际原子能机构这样国际组织来安全监管 AI 的发展，他呼吁国际间领先的人工智能制造商应该展开合作，并要求政府强化对尖端人工智能制造商的监管。

其实早在 3 月，就一篇有关叫停 6 个月 AI 研究的公开信火遍了全网。

该信呼吁所有的 AI 实验立即暂停研究比 GPT-4 更先进的 AI 模型，暂停时间至少 6 个月，为的就是把这些可怕的幻想扼杀在摇篮之中。

AI 的进步速度实在过于惊人，但相关的监管、审计手段却迟迟没有跟上，这也意味着没有人能够保证 AI 工具以及使用 AI 工具的过程中的安全性。

该联名信已经获得了包括 2018 年图灵奖得主 Yoshua Bengio、马斯克、史蒂夫 · 沃兹尼亚克、Skype 联合创始人、Pinterest 联合创始人、Stability AI CEO 等多位知名人士的签名支持，截稿前联名人数已经达到 1125 人。

公开信的原文如下：

人工智能具有与人类竞争的智能，这可能给社会和人类带来深刻的风险，这已经通过大量研究 [1] 得到证实，并得到了顶级 AI 实验室的认可 [2]。正如广泛认可的阿西洛马尔人工智能原则（Asilomar AI Principles）所指出的，先进的 AI 可能代表地球生命历史上的重大变革，因此应该以相应的关注和资源进行规划和管理。

不幸的是，即使在最近几个月， AI 实验室在开发和部署越来越强大的数字化思维方面陷入了失控的竞争，而没有人能够理解、预测或可靠地控制这些数字化思维，即使是它们的创造者也无法做到这一点。

现代 AI 系统现在已经在通用任务方面具有与人类竞争的能力 [3]，我们必须问自己：

• 我们应该让机器充斥我们的信息渠道，传播宣传和谎言吗？

• 我们应该将所有工作都自动化，包括那些令人满足的工作吗？

• 我们应该发展可能最终超过、取代我们的非人类思维吗？

• 我们应该冒失控文明的风险吗？

这些决策不应该由非选举产生的科技领导人来负责。只有当我们确信 AI 系统的影响是积极的，风险是可控的，我们才应该开发强大的 AI 系统。这种信心必须得到充分的理由，并随着系统潜在影响的增大而增强。OpenAI 最近关于人工智能的声明指出：「在某个时候，可能需要在开始训练未来系统之前获得独立审查，对于最先进的努力，需要同意限制用于创建新模型的计算增长速度。」我们同意。现在就是那个时刻。

因此，我们呼吁所有 AI 实验室立即暂停至少 6 个月的时间，不要训练比 GPT-4 更强大的 AI 系统。这个暂停应该是公开和可验证的，并包括所有关键参与者。如果无法迅速实施这样的暂停，政府应该介入并实行一个暂禁令。

AI 实验室和独立专家应该利用这个暂停时间共同制定和实施一套共享的先进 AI 设计和开发安全协议，这些协议应该由独立的外部专家严格审核和监督。

这些协议应确保遵守它们的系统在合理怀疑范围之外是安全的 [4]。这并不意味着暂停 AI 发展，而只是从不断迈向更大、不可预测的黑箱模型及其突现能力的危险竞赛中退一步。

AI 研究和开发应该重点放在提高现有强大、先进系统的准确性、安全性、可解释性、透明度、鲁棒性、对齐、可信度和忠诚度。

与此同时，AI 开发者必须与政策制定者合作，大力加速 AI 治理体系的发展。这些至少应包括：

• 专门负责 AI 的新的、有能力的监管机构；

• 对高能力 AI 系统和大型计算能力池的监督和跟踪；

• 用于区分真实与合成内容、跟踪模型泄漏的溯源和水印系统；

• 健全的审计和认证生态系统；对 AI 造成的损害承担责任；

• 技术 AI 安全研究的充足公共资金；

• 应对 AI 引起的巨大经济和政治变革（特别是对民主制度的影响）的资深机构。

人类可以在 AI 的帮助下享受繁荣的未来。在成功创造出强大的 AI 系统之后，我们现在可以享受一个「AI 盛夏」，在其中我们收获回报，将这些系统明确地用于造福所有人，并让社会有机会适应。

在面临可能对社会造成灾难性影响的其他技术上，社会已经按下了暂停键 [5]。在这里，我们也可以这样做。让我们享受一个漫长的 AI 盛夏，而不是毫无准备地冲向秋天。

附录英文

AI experts, journalists, policymakers, and the public are increasingly discussing a broad spectrum of important and urgent risks from AI. Even so, it can be difficult to voice concerns about some of advanced AI’s most severe risks. The succinct statement below aims to overcome this obstacle and open up discussion. It is also meant to create common knowledge of the growing number of experts and public figures who also take some of advanced AI’s most severe risks seriously.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Signatories:

AI ScientistsOther Notable Figures

Geoffrey Hinton

Emeritus Professor of Computer Science, University of Toronto

Yoshua Bengio

Professor of Computer Science, U. Montreal / Mila

Demis Hassabis

CEO, Google DeepMind

Sam Altman

CEO, OpenAI

Dario Amodei

CEO, Anthropic

Dawn Song

Professor of Computer Science, UC Berkeley

Ya-Qin Zhang

Professor and Dean, AIR, Tsinghua University

Ilya Sutskever

Co-Founder and Chief Scientist, OpenAI

Shane Legg

Chief AGI Scientist and Co-Founder, Google DeepMind

Martin Hellman

Professor Emeritus of Electrical Engineering, Stanford

James Manyika

SVP, Research, Technology & Society, Google-Alphabet

Yi Zeng

Professor and Director of Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences

Xianyuan Zhan

Assistant Professor, Tsinghua University

Anca Dragan

Associate Professor of Computer Science, UC Berkeley

Bill McKibben

Schumann Distinguished Scholar, Middlebury College

Alan Robock

Distinguished Professor of Climate Science, Rutgers University

Angela Kane

Vice President, International Institute for Peace, Vienna; former UN High Representative for Disarmament Affairs

Audrey Tang

Minister of Digital Affairs and Chair of National Institute of Cyber Security

Daniela Amodei

President, Anthropic

David Silver

Professor of Computer Science, Google DeepMind and UCL

Lila Ibrahim

COO, Google DeepMind

Stuart Russell

Professor of Computer Science, UC Berkeley

Marian Rogers Croak

VP Center for Responsible AI and Human Centered Technology, Google

Andrew Barto

Professor Emeritus, University of Massachusetts

Mira Murati

CTO, OpenAI

Jaime Fernández Fisac

Assistant Professor of Electrical and Computer Engineering, Princeton University

Diyi Yang

Assistant Professor, Stanford University

Gillian Hadfield

Professor, CIFAR AI Chair, University of Toronto, Vector Institute for AI

Laurence Tribe

University Professor Emeritus, Harvard University

Pattie Maes

Professor, Massachusetts Institute of Technology - Media Lab

Kevin Scott

CTO, Microsoft

Eric Horvitz

Chief Scientific Officer, Microsoft

Peter Norvig

Education Fellow, Stanford University

Atoosa Kasirzadeh

Assistant Professor, University of Edinburgh, Alan Turing Institute

Erik Brynjolfsson

Professor and Senior Fellow, Stanford Institute for Human-Centered AI

Mustafa Suleyman

CEO, Inflection AI

Emad Mostaque

CEO, Stability AI

Ian Goodfellow

Principal Scientist, Google DeepMind

John Schulman

Co-Founder, OpenAI

Kersti Kaljulaid

Former President of the Republic of Estonia

David Haussler

Professor and Director of the Genomics Institute, UC Santa Cruz

Stephen Luby

Professor of Medicine (Infectious Diseases), Stanford University

Ju Li

Professor of Nuclear Science and Engineering and Professor of Materials Science and Engineering, Massachusetts Institute of Technology

David Chalmers

Professor of Philosophy, New York University

Daniel Dennett

Emeritus Professor of Philosophy, Tufts University

Peter Railton

Professor of Philosophy at University of Michigan, Ann Arbor

Sheila McIlraith

Professor of Computer Science, University of Toronto

Victoria Krakovna

Research Scientist, Google DeepMind

Mary Phuong

Research Scientist, Google DeepMind

Lex Fridman

Research Scientist, MIT

Sharon Li

Assistant Professor of Computer Science, University of Wisconsin Madison

Phillip Isola

Associate Professor of Electrical Engineering and Computer Science, MIT

David Krueger

Assistant Professor of Computer Science, University of Cambridge

Jacob Steinhardt

Assistant Professor of Computer Science, UC Berkeley

Martin Rees

Professor of Physics, Cambridge University

He He

Assistant Professor of Computer Science and Data Science, New York University

David McAllester

Professor of Computer Science, TTIC

Vincent Conitzer

Professor of Computer Science, Carnegie Mellon University and University of Oxford

Bart Selman

Professor of Computer Science, Cornell University

Michael Wellman

Professor and Chair of Computer Science & Engineering, University of Michigan

Jinwoo Shin

KAIST Endowed Chair Professor, Korea Advanced Institute of Science and Technology

Dae-Shik Kim

Professor of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)

Frank Hutter

Professor of Machine Learning, Head of ELLIS Unit, University of Freiburg

Jaan Tallinn

Co-Founder of Skype

Adam D'Angelo

CEO, Quora, and board member, OpenAI

Simon Last

Cofounder & CTO, Notion

Dustin Moskovitz

Co-founder & CEO, Asana

Scott Aaronson

Schlumberger Chair of Computer Science, University of Texas at Austin

Max Tegmark

Professor, MIT, Center for AI and Fundamental Interactions

Bruce Schneier

Lecturer, Harvard Kennedy School

Martha Minow

Professor, Harvard Law School

Gabriella Blum

Professor of Human Rights and Humanitarian Law, Harvard Law

Kevin Esvelt

Associate Professor of Biology, MIT

Edward Wittenstein

Executive Director, International Security Studies, Yale Jackson School of Global Affairs, Yale University

Karina Vold

Assistant Professor, University of Toronto

Victor Veitch

Assistant Professor of Data Science and Statistics, University of Chicago

Dylan Hadfield-Menell

Assistant Professor of Computer Science, MIT

Samuel R. Bowman

Associate Professor of Computer Science, NYU and Anthropic

Mengye Ren

Assistant Professor of Computer Science, New York University

Shiri Dori-Hacohen

Assistant Professor of Computer Science, University of Connecticut

Miles Brundage

Head of Policy Research, OpenAI

Allan Dafoe

AGI Strategy and Governance Team Lead, Google DeepMind

Helen King

Senior Director of Responsibility & Strategic Advisor to Research, Google DeepMind

Jade Leung

Governance Lead, OpenAI

Jess Whittlestone

Head of AI Policy, Centre for Long-Term Resilience

Sarah Kreps

John L. Wetherill Professor and Director of the Tech Policy Institute, Cornell University

Jared Kaplan

Co-Founder, Anthropic

Chris Olah

Co-Founder, Anthropic

Andrew Revkin

Director, Initiative on Communication & Sustainability, Columbia University - Climate School

Carl Robichaud

Program Officer (Nuclear Weapons), Longview Philanthropy

Leonid Chindelevitch

Lecturer in Infectious Disease Epidemiology, Imperial College London

Nicholas Dirks

President, The New York Academy of Sciences

Marc Warner

CEO, Faculty

Clare Lyle

Research Scientist, Google DeepMind

Ryota Kanai

CEO, Araya, Inc.

Tim G. J. Rudner

Assistant Professor and Faculty Fellow, New York University

Noah Fiedel

Director, Research & Engineering, Google DeepMind

Jakob Foerster

Associate Professor of Engineering Science, University of Oxford

Michael Osborne

Professor of Machine Learning, University of Oxford

Marina Jirotka

Professor of Human Centred Computing, University of Oxford

Nancy Chang

Research Scientist, Google

Roger Grosse

Associate Professor of Computer Science, University of Toronto and Anthropic

David Duvenaud

Associate Professor of Computer Science, University of Toronto

Daniel M. Roy

Associate Professor and Canada CIFAR AI Chair, University of Toronto; Vector Institute

Chris J. Maddison

Assistant Professor of Computer Science, University of Toronto

Florian Shkurti

Assistant Professor of Computer Science, University of Toronto

Jeff Clune

Associate Professor of Computer Science and Canada CIFAR AI Chair, The University of British Columbia and the Vector Institute

Eva Vivalt

Assistant Professor of Economics, University of Toronto, and Director, Global Priorities Institute, University of Oxford

Jacob Tsimerman

Professor of Mathematics, University of Toronto

Danit Gal

Technology Advisor at the UN; Associate Fellow, Leverhulme Centre for the Future of Intelligence, University of Cambridge

Jean-Claude Latombe

Professor (Emeritus) of Computer Science, Stanford University

Scott Niekum

Associate Professor of Computer Science, University of Massachusetts Amherst

Lionel Levine

Associate Professor of Mathematics, Cornell University

Brian Ziebart

Associate Professor of Computer Science, University of Illinois Chicago

Aza Raskin

Cofounder, Center for Humane Technology, The Earth Species Project

Tristan Harris

Executive Director, Center for Humane Technology

Anthony Aguirre

Executive Director, Future of Life Institute

Sam Harris

Author, neuroscientist, Making Sense / Waking Up

Grimes

Musician / Artist

Chris Anderson

Dreamer-in-Chief, TED

Rif A. Saurous

Research Director, Google

Jose Hernandez-Orallo

Professor of Computer Science, Technical University of Valencia

Paul S. Rosenbloom

Professor Emeritus, University of Southern California

Samuel Albanie

Assistant Professor of Engineering, University of Cambridge

Jascha Sohl-Dickstein

Principal Scientist, Google DeepMind

Been Kim

Research Scientist, Google DeepMind

Cihang Xie

Assistant Professor of Computer Science and Engineering, UC Santa Cruz

Philip S. Thomas

Associate Professor, University of Massachusetts

Pierre Baldi

Professor, University of California, Irvine

Elad Hazan

Professor of Computer Science, Princeton University and Google DeepMind

Katherine Lee

Research Scientist, Google DeepMind

Shai Shalev-Shwartz

Professor, The Hebrew University of Jerusalem

Foutse Khomh

Professor and Canada CIFAR AI Chair, Polytechnique Montreal

Dan Hendrycks

Executive Director, Center for AI Safety

Show More

本文链接：https://my.lmcjl.com/post/19293.html