Open Letter to Sir Paul Nurse, President of the Royal Society

Dear Paul

We, as mathematicians, write to express our extreme concern about the pace of development of AI and the implications of this for humanity and civilisation. None of us have any significant involvement with AI companies[1].

In the last 3 months we have seen the leading models of OpenAI and Anthropic develop from around the level of a strong student to having solved multiple open questions at the forefront of research, including one of the 7 Millennium problems. These models are now operating[2] at the level of the top human mathematicians in many parts of the subject and we must assume there is a significant chance of them developing superhuman abilities within a similarly short timeframe.

This has profound implications for mathematics, but highlighting that is not the purpose of our letter. It is highly likely that these systems are comparably strong in other technical domains such as (for example) cyber security, autonomous weapon control, development of biological and chemical agents and the targeted spread of misinformation. Moreover we must assume that the rate of change of their capabilities will be similarly rapid in these areas.

We have all seen the concerns raised by former employees of the large AI companies, estimating the probability of human extinction to be as high as 10 percent over the next decade. Our witnessing of the rise in capabilities of these models in our domain of expertise persuades us that such statements must not be dismissed as `hype'.

By the time the situation becomes obvious to the wider public it may be too late to act. We believe this is an emergency, and call on the Royal Society to use its influence to convey this view to government and the media.

Signed by 42 Mathematical Fellows and Foreign Members of the Royal Society

Luis Fernando Alday

Martin Barlow

Emmanuel Breuillard

Tom Bridgeland

Mark Chaplain

Kevin Costello

Ingrid Daubechies

Marcus du Sautoy

Toby Gee

Mike Giles

Alain Goriely

Timothy Gowers

Ben Green

Geoffrey Grimmett

Mark Gross

Martin Hairer

Roger Heath-Brown

Dominic Joyce

Frank Kelly

Jens Marklof

Vladimir Markovic

James Maynard

James McKernan

Jason Miller

Robert Morris

Jonathan Pila

Jeremy Quastel

Oscar Randal-Williams

Mary Rees

Richard Samworth

Peter Sarnak

Caroline Series

Peter Scholze

Gordon Slade

Ivan Smith

Endre Suli

Richard Thomas

Jack Thorne

Ulrike Tillmann

Burt Totaro

Claire Voisin

Wendelin Werner

[1] Most of us use AI tools and many of us have been provided access to the leading publicly available models for free. Some of us have informal and unpaid connections with mathematicians working at AI companies, resulting in early access to and subsequent free use of some of their models.

[2] Whilst it is true that the leading AI companies employ top mathematicians, and we cannot be sure of the exact methodology behind the solution of the Navier-Stokes millennium problem, our remarks apply to the outputs of the leading publicly-available models such as ChatGPT6-Astra.

(http://www.autoadmit.com/thread.php?thread_id=5905084&forum_id=2)#50143474)