The Centre for the Study of Existential Risk is an interdisciplinary research centre focused on the study of risks threatening human extinction that may emerge from technological advances. CSER aims to combine key insights from the best minds across disciplines to tackle the greatest challenge of the coming century: safely harnessing our rapidly-developing technological power.

An existential risk is one that threatens the existence of our entire species. The Cambridge Centre for the Study of Existential Risk (CSER) — a joint initiative between philosopher Huw Price, cosmologist Martin Rees, and software entrepreneur Jaan Tallinn — was founded on the conviction that these risks require a great deal more scientific investigation than they presently receive. The Centre’s aim is to develop a new science of existential risk, and to develop protocols for the investigation and mitigation of technology-driven existential risks. 

The Centre for the Study of Existential Risk

The Centre for the Study of Existential Risk is delighted to host Professor Stuart J. Russell (University of California, Berkeley) 

The Long-Term Future of (Artificial) Intelligence

Abstract: The news media in recent months have been full of dire warnings about the risk that AI poses to the human race, coming from well-known figures such as Stephen Hawking, Elon Musk, and Bill Gates. Should we be concerned? If so, what can we do about it? While some in the mainstream AI community dismiss these concerns, I will argue instead that a fundamental reorientation of the field is required.

Stuart Russell is one of the leading figures in modern artificial intelligence. He is a professor of computer science and founder of the Center for Intelligent Systems at the University of California, Berkeley. He is author of the textbook ‘Artificial Intelligence: A Modern Approach’, widely regarded as one of the standard textbooks in the field. Russell is on the Scientific Advisory Board for the Future of Life Institute and the Advisory Board of the Centre for the Study of Existential Risk. 

The Long-Term Future of (Artificial) Intelligence

Human-level (“general”) AI is more difficult to achieve than most people think. One key obstacle is relevance, a conceptual version of the frame problem. Another is lack of the semantic web. Yet another is the difficulty of computer vision. So artificial general intelligence (AGI) isn’t on the horizon. Possibly, it may never be achieved. No AGI means no Singularity.

Even so, there’s already plenty to worry about—and future AI advances will add more. Areas of concern include unemployment, computer companions, and autonomous robots (some, military). Worries about the (illusory) Singularity have had the good effect of waking up the AI community (and others) to these dangers. At last, they are being taken seriously.

Professor Margaret Boden is a world-leading academic in the study of intelligence, both artificial and otherwise. She is is Research Professor of Cognitive Science at the Department of informatics at the University of Sussex, where her work embraces the fields of artificial intelligence, psychology, philosophy, cognitive and computer science. She was the founding-Dean of Sussex University’s School of Cognitive and Computing Sciences, a pioneering centre for research into intelligence and the mechanisms underlying it — in humans, other animals, or machines. The School’s teaching and research involves an unusual combination of the humanities, science, and technology.

Professor Boden has also been an important participant in the recent international discussions over the long-term impacts of AI. She was a member of the AAAI’s 08/09 Presidential Panel on long-term AI futures (http://www.aaai.org/Organization/presidential-panel.php), and also took part in the recent Puerto Rico conference on the Future of AI, co-organised by CSER (http://futureoflife.org/misc/ai_conference); she is therefore uniquely well-placed to discuss near- and long-term prospects in AI. 

Human - Level AI- Is It Looming Or Illusory?

In the coming century, the greatest threats to human survival may come from our own technological developments. However, if we can safely navigate the pitfalls, the benefits that technology promises are enormous. A philosopher, an astronomer, and an entrepreneur have come together to form the Centre for the Study of Existential Risk. The goal: to bring a fraction of humanity's talents to bear on the task of ensuring our long-term survival. In this lecture, Huw Price, Martin Rees and Jaan Tallinn will outline humanity's greatest challenge: surviving the 21st Century.

Professor Huw Price (University of Cambridge)
Lord Martin Rees (Astronomer Royal)
Jaan Tallinn (Co-founder of Skype) 

Existential Risk: Surviving the 21st Century

Will We Cause Our Own Extinction? Natural versus Anthropogenic Extinction Risks

How will humanity go extinct? Is it more likely to be from natural causes such as an asteroid impact or anthropogenic causes such as a nuclear war? Using the fossil record, we can place a rough upper bound on the probability of human extinction from natural causes: all natural causes put together have less than a 1% chance of causing human extinction each century, and probably less than 0.1%. In contrast, it is very difficult to put upper or lower bounds on the chance of extinction from anthropogenic causes. In this talk, Dr Toby Ord advance an argument that anthropogenic causes currently produce ten or more times as much extinction risk as the natural causes, and shows how this suggests that we should prioritise the reduction of anthropogenic extinction risks over natural ones.

About the speaker:

Toby Ord is a Research Fellow at the Future of Humanity Institute, Oxford University & Oxford Martin School. He works on theoretical and practical questions concerning population ethics, global priorities, existential risk and new technologies, and recently contributed a report on Managing Existential Risk from Emerging Tehnologies to the Chief Scientific Advisor’s annual report for the UK government. Dr Ord’s concern for current and future generations also led him to found the orgnanisation Giving What We Can. 

Will We Cause Our Own Extinction?

Writers who speculate about the future of artificial intelligence (AI) and its attendant risks often caution against anthropomorphism, the tendency to ascribe human-like characteristics to something non human. An AI that is engineered from first principles will attain its goals in ways that would be hard to predict, and therefore hard to control, especially if it is able to modify and improve on its own design. However, this is not the only route to human-level AI. An alternative is to deliberately set out to make the AI not only human-level but also human-like. The most obvious way to do this is to base the architecture of the AI on that of the human brain. But this path has its own difficulties, many pertaining to the issue of consciousness. Do we really want to create an artefact that is not only capable of empathy, but also capable of suffering? 

Minds Like Ours: An Approach To AI Risk

“Risks and benefits of gain-of-function experiments in potentially pandemic pathogens. How should we evaluate them, and what alternatives exist?”

Professor Derek Smith, Professor of Infectious Disease Informatics at Cambridge, gave a response.

Abstract: "A growing trend in experimental virology has been the modification of influenza viruses that are antigenically novel to, and virulent in humans, such that these variant viruses are readily transmissible in mammals, including ferrets which are thought to be the best animal model for influenza infection. Novel, contagious, virulent viruses are potential pandemic pathogens in that their accidental or malevolent release into the human population could cause a pandemic. This talk will describe the purported benefits of such studies, arguing that these are overstated; estimate the magnitude of the risk they create, argue for the superiority of alternative scientific approaches on both safety and scientific grounds, and propose an ethical framework in which such experiments should be evaluated. The talk will also explore recent developments following the pause in funding for this research announced by the United States Government in October, and steps towards the risk-benefit analysis called for by the announcement"

Professor Lipsitch is a professor of epidemiology and the Director of the Centre for Communicable Disease Dynamics at Harvard University. He is one of the founders of the Cambridge Working group, which calls for a “quantitative, objective and credible assessment of the risks, potential benefits, and opportunities for risk mitigation” of gain of function experiments in potentially pandemic pathogen strains.

For more on the scientific debate, see “Gain-of-function experiments: time for a real debate”. 

Risks and benefits of gain-of-function experiments in potentially pandemic pathogens

Pushing the Limits - A public lecture with Professor Jane Heal

What do theory of evolution, intellectual history and philosophy tell us about what we human beings are like? And what resources - intellectual, emotional, moral - we can muster for dealing with the existential risks of our current situation? The talk will offer a speculative overview of these topics, which set the scene for the challenging issues CSER faces.

Professor Jane Heal a world leading expert in the Philosophy of Mind. Professor Heal studied for her first degree in Cambridge, reading History for two years and then Philosophy (or "Moral Sciences" as it was called in those days) for another two years. She also took her Ph.D. at Cambridge, working on problems on the philosophy of language. After two years post doctoral study in the US (at Princeton and Berkeley) she was appointed to a lectureship at the University of Newcastle upon Tyne. Having taught there for several years she moved back to Cambridge where she is now a Fellow of St John's College. She was elected a Fellow of the British Academy in 1997. 