Paging Sarah Connor!
After decades as a sci-fi staple, artificial intelligence has leapt into the mainstream. Between Apple’s Siri and Amazon’s Alexa, IBM’s Watson and Google Brain, machines that understand the world and respond productively suddenly seem imminent.
The combination of immense Internet-connected networks and machine-learning algorithms has yielded dramatic advances in machines’ ability to understand spoken and visual communications, capabilities that fall under the heading “narrow” artificial intelligence. Can machines capable of autonomous reasoning—so-called general AI—be far behind? And at that point, what’s to keep them from improving themselves until they have no need for humanity?
The prospect has unleashed a wave of anxiety. “I think the development of full artificial intelligence could spell the end of the human race,” astrophysicist Stephen Hawking told the BBC. Tesla founder Elon Musk called AI “our biggest existential threat.” Former Microsoft Chief Executive Bill Gates has voiced his agreement.
How realistic are such concerns? And how urgent? We assembled a panel of experts from industry, research and policy-making to consider the dangers—if any—that lie ahead. Taking part in the discussion are Jaan Tallinn,a co-founder of Skype and the think tanks Centre for the Study of Existential Risk and the Future of Life Institute; Guruduth S. Banavar,vice president of cognitive computing at IBM’s Thomas J. Watson Research Center; andFrancesca Rossi, a professor of computer science at the University of Padua, a fellow at the Radcliffe Institute for Advanced Study at Harvard University and president of the International Joint Conferences on Artificial Intelligence, the main international gathering of researchers in AI.
Here are edited excerpts from their conversation.
What’s the risk?
WSJ: Does AI pose a threat to humanity?
MR. BANAVAR: Fueled by science-fiction novels and movies, popular treatment of this topic far too often has created a false sense of conflict between humans and machines. “Intelligent machines” tend to be great at tasks that humans are not so good at, such as sifting through vast data. Conversely, machines are pretty bad at things that humans are excellent at, such as common-sense reasoning, asking brilliant questions and thinking out of the box. The combination of human and machine, which we consider the foundation of cognitive computing, is truly revolutionizing how we solve complex problems in every field.
AI-based systems are already making our lives better in so many ways: Consider automated stock-trading agents, aircraft autopilots, recommendation systems, industrial robots, fraud detectors and search engines. In the last five to 10 years, machine-learning algorithms and advanced computational infrastructure have enabled us to build many new applications.
However, it’s important to realize that those algorithms can only go so far. More complex symbolic systems are needed to achieve major progress—and that’s a tall order. Today’s neuroscience and cognitive science barely scratch the surface of human intelligence.
My personal view is that the sensationalism and speculation around general-purpose, human-level machine intelligence is little more than good entertainment.
MR. TALLINN: Today’s AI is unlikely to pose a threat. Once we shift to discussing long-term effects of general AI (which, for practical purposes, we might define as AI that’s able to do strategy, science and AI development better than humans), we run into the superintelligence control problem.
MR. TALLINN: Even fully autonomous robots these days have off switches that allow humans to have ultimate control. However, the off switch only works because it is outside the domain of the robot. For instance, a chess computer is specific to the domain of chess rules, so it is unaware that its opponent can pull the plug to abort the game.
However, if we consider superintelligent machines that can represent the state of the world in general and make predictions about the consequences of someone hitting their off switch, it might become very hard for humans to use that switch if the machine is programmed (either explicitly or implicitly) to prevent that from happening.
WSJ: How serious could this problem be?
MR. TALLINN: It’s a purely theoretical problem at this stage. But it would be prudent to assume that a superintelligent AI would be constrained only by the laws of physics and the initial programming given to its early ancestor.
The initial programming is likely to be a function of our knowledge of physics—and we know that’s still incomplete! Should we find ourselves in a position where we need to specify to an AI, in program code, “Go on from here and build a great future for us,” we’d better be very certain we know how reality works.
As to your question, it could be a serious problem. It is important to retain some control over the positions of atoms in our universe [and not inadvertently give control over them to an AI].
MS. ROSSI: AI is already more “intelligent” than humans in narrow domains, some of which involve delicate decision making. Humanity is not threatened by them, but many people could be affected by their decisions. Examples are autonomous online trading agents, health-diagnosis support systems and soon autonomous cars and weapons.
We need to assess their potential dangers in the narrow domains where they will function and make them safe, friendly and aligned with human values. This is not an easy task, since even humans are not rationally following their principles most of the time.
Affecting everyday life
WSJ: What potential dangers do you have in mind for narrow-domain AI?
MS. ROSSI: Consider automated trading systems. A bad decision in these systems may be (and has been) a financial disaster for many people. That will also be the case for self-driving cars. Some of their decisions will be critical and possibly affect lives.
WSJ: Guru, how do you view the risks?
MR. BANAVAR: Any discussion of risk has two sides: the risk of doing it and the risk of not doing it. We already know the practical risk today of decisions made with incomplete information by imperfect professionals—thousands of lives, billions of dollars and slow progress in critical fields like health care. Based on IBM’s experience with implementing Watson in multiple industries, I maintain that narrow-domain AI significantly mitigates these risks.
I will not venture into the domain of general AI, since it is anybody’s speculation. My personal opinion is that we repeatedly underestimate the complexity of implementing it. There simply are too many unknown unknowns.
WSJ: What proactive steps is International Business Machines taking to mitigate risks arising from its AI technology?
MR. BANAVAR: Cognitive systems, like other modern computing systems, are built using cloud-computing infrastructure, algorithmic code and huge amounts of data. The behavior of these systems can be logged, tracked and audited for violations of policy. These cognitive systems are not autonomous, so their code, data and infrastructure themselves need to be protected against attacks. People who access and update any of these components can be controlled.
The data can be protected through strong encryption and its integrity managed through digital signatures. The algorithmic code can be protected using vulnerability scanning and other verification techniques. The infrastructure can be protected through isolation, intrusion protection and so on.
These mechanisms are meant to support AI safety policies that emerge from a deeper analysis of the perceived risks. Such policies need to be identified by bodies like the SEC, FDA and more broadly NIST, which generally implement standards for safety and security in their respective domains.
WSJ: Watson is helping doctors with diagnoses. Can it be held responsible for a mistake that results in harm?
MR. BANAVAR: Watson doesn’t provide diagnoses. It digests huge amounts of medical data to provide insights and options to doctors in the context of specific cases. A doctor could consider those insights, as well as other factors, when evaluating treatment options. And the doctor can dig into the evidence supporting each of the options. But, ultimately, the doctor makes the final diagnostic decision.
MS. ROSSI: Doctors make mistakes all the time, not because they are bad, but because they can’t possibly know everything there is to know about a disease. Systems like Watson will help them make fewer mistakes.
MR. TALLINN: I’ve heard about research into how doctors compare to automated statistical systems when it comes to diagnosis. The conclusion was that the doctors, at least on average, were worse. What’s more, when doctors second-guessed the system, they made the result worse.
MR. BANAVAR: On the whole, I believe it is beneficial to have more complete information from Watson. I, for one, would personally prefer that anytime as a patient!
The human impact
WSJ: Some experts believe that AI is already taking jobs away from people. Do you agree?
MR. TALLINN: Technology has always had the tendency to make jobs obsolete. I’m reminded of an Uber driver whose services I used a while ago. His seat was surrounded by numerous gadgets, and he demonstrated enthusiastically how he could dictate my destination address to a tablet and receive driving instructions. I pointed out to him that, in a few years, maybe the gadgets themselves would do the driving. To which he gleefully replied that then he could sit back and relax—leaving me to quietly shake my head in the back seat. I do believe the main effect of self-driving cars will come not from their convenience but from the massive impact they will have on the job market.
In the long run, we should think about how to organize society around something other than near-universal employment.
MR. BANAVAR: From time immemorial, we have built tools to help us do things we can’t do. Each generation of tools has made us rethink the nature and types of jobs. Productivity goes up, professions are redefined, new professions are created and some professions become obsolete. Cognitive systems, which can enhance and scale the capabilities of our minds, have the potential to be even more transformative.
The key question will be how to build institutions to quickly train professionals to exploit cognitive systems as their assistants. Once learned, these skills will make every individual a better professional, and this will set a new bar for the nature of expertise.
WSJ: How should the AI community prepare?
MR. TALLINN: There is significant uncertainty about the time horizons and whether a general AI is possible at all. (Though, being a physicist, I don’t see anything in physics that would prevent it!) Crucially, though, the uncertainty does not excuse us from thinking about the control problem. Proper research into this is just getting started and might take decades, because the problem appears very hard.
MS. ROSSI: I believe we can design narrowly intelligent AI machines in a way that most undesired effects are eliminated. We need to align their values with ours and equip them with guiding principles and priorities, as well as conflict-resolution abilities that match ours. If we do that in narrowly intelligent machines, they will be the building blocks of general AI systems that will be safe enough to not threaten humanity.
MR. BANAVAR: In the early 1990s, when it became apparent the health-care industry would be computerized, patient-rights activists in multiple countries began a process that resulted in confidentiality regulations a decade later. In the U.S. as in other places, it is now technologically feasible to track HIPAA compliance, and it is possible to enforce the liability regulations for violations. Similarly, the serious question to ask in the context of narrow-domain AI is, what are the rights that could be violated, and what are the resulting liabilities?
MS. ROSSI: As we have safety checks that need to be passed by anybody who wants to sell a human-driven car, there will need to be new checks to be passed by self-driving cars. Not only will the code running in such cars need to be carefully verified and validated, but we will also need to check that the decisions will be made according to ethical and moral principles that we would agree on.
MR. BANAVAR: What are the rights of drivers, passengers, and passersby in a world with self-driving cars? Is it a consumer’s right to limit the amount of information that can be exchanged between a financial adviser and her cognitive assistant? Who is liable for the advice—the financial adviser, the financial-services organization, the builder of the cognitive assistant or the curator of the data? These are as much questions about today’s world, [about how we regulate] autonomous individuals and groups with independent goals, as they are about a future world with machine intelligence.
Mr. Greenwald is a news editor for The Wall Street Journal in San Francisco. He can be reached at firstname.lastname@example.org.