Starr Forum: Artificial Intelligence and National Security Law: A Dangerous Nonchalance

Starr Forum: Artificial Intelligence and National Security Law: A Dangerous Nonchalance

welcome to today’s MIT Starr Forum on Artificial Intelligence
and National Security Law, A Dangerous Nonchalance. Before we get
started, I just wanted to remind everyone that we
have many more Starr Forums, and if you haven’t
already, please sign up to receive our email updates. We also have fliers
of upcoming events, including our next
event which is on March 15, The
Uncounted Civilian Victims of America’s Wars. The speaker will
be Asmat Khan, who is an award-winning
investigative journalist. She’s also a New
York Times Magazine contributing
writer, and a Future of War Fellow at New America,
in Arizona State University. Her reporting has brought
her to Iran, Egypt, Pakistan, Afghanistan, and
other conflict zones. And I’m Michelle. I’m from the Center for
International Studies. Today’s talk is going
to be followed by a Q&A, so we’ll ask you to probably
just raise your hands, and ask only one
question, due to time. And when you’re called
upon, if possible, please identify yourself and
let us know your affiliation, and so forth. Now I’m thrilled to introduce
the speaker of today’s event, James E. Baker. Judge Baker is a
Robert E. Wilhelm Fellow at the MIT Center
for International Studies. He retired from the
United States Court of Appeals for the Armed
Forces in July 2015, after 15 years of service,
the last four as Chief Judge. He previously served
as Special Assistant to the President
and Legal Advisor to the National
Security Council, and Deputy Legal Advisor to
the National Security Council, where he advised the
president, vice president, national security
advisor, and NSC staff on US and international
law related to intelligence
counter-terrorism, foreign assistance, and military
operations, among other things. He served as general
counsel to the NSC, and advised the Principals
Committee, which included the secretaries
of state and defense, the national security advisor,
the Director of the Central Intelligence, and Chairman
of the Joint Chiefs of Staff, and the Deputies Committee. Judge Baker has also served
as counsel to the president’s Foreign Intelligence
Advisory Board, and Intelligence
Oversight Board. He’s also served as an attorney
at the Department of State, a legislative aide, and
acting chief of staff to Senator Daniel
Patrick Moynihan, and as a Marine Corps
infantry officer, including his reserve
service in 2000. Alas, he has done so much
more, but I will end there. Please join me in
welcoming Judge Baker. JAMES E. BAKER: Well,
thank you very much. Thank you. Thank you, Laura. Thank you, Michelle. Thank you, Center for
International Studies, for allowing me to come
up this year, and try and teach myself
something about AI. You all will be the
judge, in a few minutes, as to whether I have
learned anything. This is the first time
out for this presentation. So let me tell you,
right up front, that it’s somewhere between
seven minutes in length and seven hours. I’m not quite sure how long
yet, so we’ll find out. As you know, my topic is
artificial intelligence and national security law,
emphasis on the law part. I know better than
to come into MIT and purport to lecture anybody
about algorithms or math equations. In fact, I’ve chosen
not to use slides, because if I put up a slide of
a math equation or a game of Go, someone would ask me
to explain the rules, and I would falter immediately. So I am not a technologist. I am a bridger, and by bridger
I mean, not a civil engineer, but someone who bridges
communities, and translates. My job is, I believe,
to translate for you, into plain English,
why it is you should care about artificial
intelligence in the national security sphere. And my job is to explain to
national security policy makers why they should care about
national security law, as it applies to AI, and
why they should care now. So my goal is to try and speak
in plain English on that, and to verify whether I
have succeeded or not, I’ve invited my high school
English teacher here, whose job it is to
validate my use of English. So here’s what I’m going to do. I’m going to start with
a few quotes and data points about AI. I’m then going to talk about the
national security uses of AI. I will then describe
some of the risks, and then in light
of those risks, talk about how law and
governance might address them. Is that a fair way to proceed? Depending on how
we do, we may have some time for questions
and some answers at the end, maybe
mostly questions. So let me start by referring
to your first homework assignment, which
is the Belfer IARPA study on national
security and AI. IARPA the intelligence
community’s version of DARPA, which all of you at
MIT will know of. And the study in 2017
concluded that AI will be as transformative a
military technology as aviation and nuclear weapons were before. So that should get
your attention– as transformative a military
technology as aviation and nuclear weapons. In July 2017, China’s
state council, in theory, their
highest governing body, released an AI plan and
strategy calling for China to become the world’s
leader in AI by 2030, to reach parity by 2020 with
the leading countries, i.e., the United States, to become one
of the world’s leaders in 2025, and to be the primary
AI power in 2030. And they vested $150
billion into this. As you’ll see in a moment,
when you say $150 billion, one, it’s hard to count $150 billion
across the technology field, but secondly, a lot of it
depends on how you define AI. As many of you know–
how many of you actually know about
AlphaGo and Google? OK. Just wanted to get a sense. Right. So in 2016, Google’s
AlphaGo computer, a DeepMind computer at the
time, beat the world’s best Go player. And this was viewed,
and is viewed, by many as a Sputnik moment
in China, and by Sputnik moment, the moment
in the United States when we got notice that we are
in a whole new technological world and place. And in China, this was
viewed as a Sputnik moment, in part because AlphaGo– Go is a Chinese game– and in part because it was a US
Google computer that beat Go. And how many people
watched, live, the AlphaGo play the world
champion in Go in 2016? Anybody in here watch
that, by the way? Right. Not a Sputnik moment
in the United States, but 200 million people watched
the AlphaGo computer play the Go champion,
live on television. So they got the memo
about AI in China. We did not. Lest Russia feel left out,
here are two reassuring quotes from Vladimir Putin. “Whoever becomes the
leader in this sphere, AI, will become the
ruler of the world.” That’s happy. And then my personal favorite,
“When will it eat us? When will it eat “us?” Vladimir Putin asks, and I’ll
explain a little bit more about being eaten in a moment. And then, of course, you’ve
heard Stephen Hawking and Elon Musk, on a daily
basis, say things like this. “In short, the
rise of powerful AI will be either the
best or the worst thing ever to happen to humanity.” That’s Stephen Hawking. So what is it? And again, I’m
conscious that I’m in a room where people not
only can better define it, they can actually
create it, but I’m going to give you a
couple of definitions, and then tell you why
it’s hard to define. First, here’s the Stanford
100-year study definition. “Artificial intelligence
is a science, and a set of computational
technologies, that are inspired, but typically
operate quite differently from, the way people use their
nervous systems and bodies to sense, learn, reason,
and take action.” It’s hard to define, for
many reasons, in part because we anthropomorphize it. I just wanted to use that word
and see if I could say it. But we tend to look at it
as a human quality, right, and by using the
word intelligence, we’re thinking
human intelligence. And what AI really is,
is machine optimization. So don’t think intelligence,
and don’t think of love and kindness, right? Machines do what they’re
programmed to do. And AI is a form of
machine optimization. It also incorporates various
fields and subfields. And that’s another reason
it’s hard to define. More on that in a second. The premise behind
AI is that if you can express an idea, a
thought, or an action in numeric fashion, you can
code that purpose into software and cause a machine to do it. So one question
for technologists is, what exactly are the
limits on coding things? I don’t know the answer
to that question. I sure would like to know. So I define the ability– I like to stick
with AI’s definition as machine optimization,
and get away from that anthropomorphizing. So let me tell you a couple
more things about AI. How long has it been around? It’s been around since Alan
Turing in Bletchley Park. You’ve all seen the
movie, The Imitation Game. The Imitation Game
is Alan Turing’s game where you win the game
when you can get a computer to talk to a human
in another room, and the people in the
other room, the humans, think they’re
talking to a human. That’s The Imitation Game. I never really quite figured out
why the movie was named that. But that’s Alan Turing, and
that’s The Imitation Game. The first AI conference
was held at Dartmouth, it pains me to say, in 1956. They wrote a grant
and it’s remarkable. I mean, it’s
interesting, the hubris. They said, we’re going to have
a conference of 10 scientists and figure out AI. We’re still working
on that piece, but that sense of confidence
can be both a positive force as well as a negative force. If you wade into the
literature on AI, there’s a lot of
talk about seasons. It feels very New
Englandy, and those of you who have read
about AI will know, there’s this constant discussion
about there is an AI winter. And then there’s a pause, and
then there’s another winter, and they keep on having winters. And that basically means a
pause in the funding for AI, as well as a pause in
the progress of AI. But what I’m trying
to tell you, and what I would be telling a national
security official in the United States government, if I could,
is not only are we in a spring now, summer is right
around the corner. And it is going to
be a hot summer. So forget the winter piece. We’re in, at least,
spring, if not summer. There are several
factors for that. One is the growth of
computational capacity. And a iPhone 5 has 2.7 times
the computational capacity of a Cray-2
supercomputer in 1985. I started in the Marine
Corps, as you heard, and we whispered about
Cray-2 supercomputers. They were the most unbelievable
secret tool in the world, and not realizing that the
iPhone would be coming along. Data and big data. AI works, in many
cases, on data, and think about the internet
of things, and all the data you’re currently sharing with
the world on Facebook, Twitter, and every other place. Cloud computing. How many of you have been
working with Fake App? Don’t admit that, by the way. If you read the New
York Times yesterday, you read the article
about Fake App and how you can use it to
morph pictures near perfectly. And of course, because
this is the United States, what are we morphing
near perfectly? Pornography. That’s its best use. But cloud computing,
how’d they do it? The reporter went and he
contracted out with the cloud. He needed
computational capacity. He didn’t have it on
his home computer. You can now get that through
cloud computing, which solves a lot of the problems. Algorithms and software
is the next area that has helped blossom this field. This is why we’ve gone
from winter through spring, and are about to hit summer. Knowledge and mapping
of the brain– in the last 10 years, we’ve
learned more about the brain– I’m making this up– but more about the brain than
50 times what we knew before. That’s all made up,
but in part, because of the study of brain
injuries like TBI, and post-traumatic
stress disorder coming out of the wars, and
also football studies, we’re learning things
about the brain that we did not know before. And a lot of the AI
people are trying to model neural networks
from the brain, some people, literally, by creating
3D versions of brains, and other people,
metaphorically, by creating neural networks. The dotcom Facebook
phase, right? That’s led to people pumping
money into this in a way they didn’t before. You know Napoleon’s
phrase about, in every soldier’s backpack,
there’s a Marshal’s baton? Of course you know that. You all seek to be
Marshals in the front. No. In every dorm
room, we now think, is a billionaire’s
portfolio, if you just come up with the right idea. So that’s part of
the drive behind AI. And then all these
factors come together in what is known as
machine learning. And here, I would encourage you
to go speak to an MIT professor to explain machine
learning to you, but what machine
learning, in general, is, is the use of algorithms,
data, and training of a computer to allow that
computational machine to detect patterns, classify information,
and predict information, better than humans can
in certain spheres. I’ll give you a couple of
examples of those in a second. What are the drivers
going forward? In AI, hardware, right,
supercomputers and chips, data, and algorithms,
and then the power of your commercial sector. We have Silicon Valley. So here, a comment on China. China is not monolithic. It’s interesting to think $150
billion is a lot of money. They have advantages in
focus, centralized authority. They have data. And notice that in 2015,
they passed a law prohibiting any data involving
Chinese persons from being shipped overseas. And that’s not just
a US v Microsoft jurisdictional matter. That is about AI data and
training, among other things. And they have Badoo,
Alibaba, and Tencent. So I would love to say we
have the advantage of market incentive, and they
have the disadvantage of communist centralism,
but they also have the advantage
of market incentive, in the form of
Alibaba and Badoo. But here I would say,
break it down, and look at the particular sectors, like
hardware, software, algorithms, and that sort of thing, and
figure out who’s good at what, and where the edges come. Now finally, a word
about being eaten. Remember Putin? “When will it eat us?” In the field, there’s generally
three categories of AI that people talk about. There’s the current state of
affairs, which is narrow AI, and that’s when an AI
application is generally better than a human at a
particular task, especially pattern analysis. AGI, or artificial
general intelligence, also known as HLMI, human
level machine intelligence, is that point in time– and it’s not a
literal point in time. It’s a phase– where an
artificially empowered machine is generally better than
humans at a number of tasks, and can move fluidly from task
to task, and train itself. And then there’s artificial
super intelligence, and that occurs when machines
are smarter than humans across the board. So that machine that
is smarter than a human can plug into the
internet, fool you into thinking it’s not
plugging into the internet, and do all sorts of things. Now how does that end up
with you getting eaten? It comes in the form of what the
literature calls the paperclip machine, and they picked
the paperclip machine because the emphasis
here is on a machine that neither hates nor
likes, but simply does what it’s trained to do. And that is make paperclips. And eventually the
paperclip machine, which is optimizing making
paperclips, runs out of energy, so it looks around
and says, where can I find more energy
since I’ve already tapped the grid off the internet? And it looks in
a room like this, and it sees sources of carbon,
and then powers other machines to turn those sources
of carbon into energy, to make more paperclips. Now if I were briefing a
national security, an official in the United States
government, would I start with the
paperclip machine? No, I’d be escorted out by
whomever, the Secret Service or the Marshals,
within 17 seconds. What concerns me about Putin’s
statement about being eaten is that it means he’s being
briefed at the highest levels of the
Russian government on super artificial intelligence. Now I’m delighted that he’s
worried about the paperclip machine and getting eaten. That could solve a
number of problems, but it worries me that he is
that far into the conversation, that that’s the level of detail. You follow? So why might Vladimir
Putin get briefed on AI? What are the national
security applications? You will have
heard many of them, but perhaps not all of them. First, military
applications, right? And here people tend to
jump immediately to robots. And the Russians
are indeed making a robot they call Fedor,
which can shoot and carry heavy weights,
basically an infantry– or if they’re really heavy
and they’re really good shots, they’re essentially Marines. But there’s a lot more to it. So we talk about
autonomous weapon systems, and lethal autonomous
weapon systems. That’s what you probably
have contemplated. A couple of points
about these things. First, we’ve had autonomous
weapon systems since the 1970s. So to the military,
at least, the concept of an autonomous weapon
system is not a new thing. Some of you will know the
Phalanx, and Iron Dome, C-Ram. These are all weapons systems
that, in one way or another, have an autonomous
capacity to them. The second, DoD has
gotten the memo about AI. Department of Defense. The Department of Defense
got the memo about AI. They initiated something in
2014 called the Third Offset, and an offset in DoD
speak is the effort to harness the
nation’s technology to offset an
opponent’s advantage. So the first
offset, for example, took place in the 1950s. And it was addressed to the
Soviet Union Warsaw Pact’s advantage in manpower in Europe. And the effort was to develop
tactical nuclear weapons to offset the Soviet
advantage in Europe. DoD came out with the
third offset in 2014, and started implementing it. And what’s notable
about it is that it’s a technological
offset that doesn’t seek to offset the opponent’s
advantage in a weapon, but through the general use of
technology and, in particular, AI. So DoD is putting its
money where its mouth is, and its energy as well. The leader in this area
was Deputy Secretary of Defense, Bob Work. Next point, if you want to
imagine how AI might enable offensive and defensive
weapons, the featured tool here that all sides are working on– the Chinese for sure,
probably the Russians– are swarms. And the use of swarms of
AI birds, objects, robots– call them what you will– pieces of metal,
to work in unison, both as offensive
elements, perhaps to attack a aircraft carrier. You can imagine. Think about kamikaze
pilots, but now think about kamikaze pilots without
the supply problem, or the moral dilemma problem. And you can use AI as chaff. Chaff is metal objects you throw
out the back of an aircraft to try and get a missile to
chase the metal objects, rather than the aircraft itself. You could deploy a swarm
instead, and send the missile on quite a goose chase. So that’s what a lot of the
literature is about right now. You probably aren’t
spending your time, as you were not spending your
time watching Go matches. You might not be spending
your time reading military law journals and doctrinal
publications, but they’re spending a
lot of time on swarmology. But what else can AI do
in the military context? Logistics. You want to plan D-Day? Imagine planning
the D-Day invasion, but now you have ways. You all know ways,
right, for traffic? So it’s ways that tells you
exactly what to load on which ship, and then the
weather comes in and it’s lousy, so you reprogram
ways, and it says, OK, go here instead, on this
day, at this time, with the ship loaded this way. Training. You can use it for war gaming. Testing. You can test weapons with it. Imagine anything that
involves danger, repetition, or something that can be
diminished by human fear, how AI-optimized machine can better
address those situations. So now intelligence, how can
AI be used for intelligence? Remember that we’re talking
about pattern recognition. And one of my favorite
writers in this area, a person named Ryan Calo– Ryan Calo is at the
University of Washington. And one of the reasons he’s
one of my favorite people in this field is because
he does, in fact, write in English. I can understand what he writes,
and he doesn’t provide a math equation after each sentence. And he said the whole purpose
of AI is to spot patterns people cannot see. That’s current narrow AI. So who wants to do that? Intelligence, right? That’s the whole
business of intelligence. Think about image recognition,
voice recognition, sorting, aggregation,
prediction, translation, anomaly identification,
and everything that occurs in cyberspace. Collection. You can collect information. For example, if you are
trying to figure out if a North Korea was
violating a sanctions regime, you can figure out, aggregate
all the information available electronically, about shipping
routes, bills of lading, and so on, and track
them and find patterns. If you’re doing analysis, you
can get all the ISIS Facebook posts, and see if you can
find patterns in them, patterns of message and
patterns of location. With the Fake App from
yesterday’s New York Times, imagine what you can do there
if you’re engaging in an effort to destabilize another
country’s democratic elections. And if you aren’t
scared by AI, and you shouldn’t be scared by it, you
should decide to regulate it. How many of you read the 37-page
indictment from Bob Mueller, from two weeks ago? After you finish reading the
Belfer IARPA study on AI, I would request that you all
read the 37-page indictment, and then ask yourself, how
would AI enable, further enable, what occurred in that context? Do not go to social media
or to cable news networks to answer that question. Go to the indictment,
and then apply whatever you think I said today. Counter-intelligence anomaly,
right, the whole game of counter-intelligence
anomaly is, who’s doing something
out of pattern, and why? AI is perfect for
figuring that out. Now let’s go to
Homeland Security. Think about watch lists. Think about facial recognition. Decision making. The hardest thing to do
in government other than make a decision is diffuse the
intelligence, in real time, to make a timely decision. In my day, the best way
to fuse intelligence was to have a
principals meeting, and see what intelligence the
Secretary of State brought, the Secretary of Defense,
the DNI and so on. That’s not a great method. AI can do this better than
any other method I’ve seen. It can model as well. All that’s great news if you’re
on our side of the table, perhaps. Think about how AI
and these methods can help if you’re running
an authoritarian regime, and you have something called
a social credit system. You all know what I’m
talking about there. That’s the new
Chinese mechanism. If you jaywalk,
you are called out, but lest you think this is just
a Chinese feature, in the UK, if you talk on your cell phone,
handheld, while on the highway, you will receive a license
suspension in the mail, as well as a photo of
yourself on your phone. So AI has wonderful
application for social control. And of course, non-state
actors will want to empower– think about how AI can
be used to enhance IEDs. Take the swarm concept,
and you do not need to use the cinderblock-on-gas-pedal
method anymore. You can use other
methods to empower IEDs. So now, I’ve done the
first two sections of what I claimed I was going to do. I described what AI
is, and then I’ve described some of the
national security uses for AI. Now I’m going to turn
to some of the risks, and then I’m going to
talk about some mechanisms for legal regulation
and governance. Now because I’m in the
business of national security, I talk about risk, and you
tend to do worst-case scenario planning. The record should be
very clear that AI holds great promise for many
things that are positive, including in the
area of oncology. AI-enabled machines can read
tumors and predict cancer better than doctors, period,
time after time after time, by huge percentage points. Why? Because it can see patterns. It can break pictures down
into three-dimensional sectors in a way the human
eye cannot do. It can do pixels in a way
the human eye cannot do. So if you are getting
tested for cancer, you don’t want the
machine to print out a little thing saying,
congratulations, you have cancer and
will die next week, based on probability analysis. But you do want the
machine to tell you whether your tumor
is benign or malign, and so on, and then
talk to a doctor. So that’s an example of
just one promise of AI that is not involving
shopping algorithms. So here are some of the risks
in the national security area. Speed. I talk about the pathologies
of national security decision making, and the
classic pathologies, besides cognitive
bias, are speed, meaning you have too little
time, lack of information, and the national
security imperative, the drive to solve the
national security problem. Clearly. AI is going to help
mitigate the issue about too little information
in too little time. That’s a positive. But it will also,
grotesquely, potentially, shorten the amount of time
people have to make decisions. Think about hand-to-hand combat
in cyberspace, and if that’s AI enabled, and there’s no person
in the loop making a decision. That’s an example. I worry that AI
will both be used and not used, used in a
wrong way, and then not used. It is an advantage to have it
as a probability predictor, and so on, but it will only work
if the people you’re briefing and the people you
are working it with– and that means people like
me, not people like you, meaning people who are less
comfortable with technology– they have to understand why it
is that there’s a 76% chance probability that Jamie
Baker is a terrorist, and not a former secretary
of state, or his son. And you can see how AI,
which is good at telling you what the percentage
possibility is, will be less good
at telling you just which Jamie Baker
you’re dealing with. Policymakers loved to
ask intelligence people, are you 87% sure? Are you 93% sure? And the intelligence
people want to say, it is our informed
judgment that– and I worry that AI will drive
us into math national security decision making. And some national
security making is intuitive, whether
to go or not on D-Day. Plug that into an
algorithm, and we’d still be waiting for perfect
weather and circumstances to make the
cross-channel landing. And then military
command and control. The US military
currently uses what they call the
Centaur model, which is augmentation
of human capacity, rather than completely
autonomous weapon systems. You see the point in
why it’s a centaur. A centaur, part
human, part animal. We’re not prepared to
make it totally animal. And the previous
secretary of defense stated, essentially,
a no-first-use policy, which was, we will not go to
fully autonomous weapon systems in the first instance, but
we will reserve the right to respond in kind. So that’s interesting and risky. The law of intended
consequences. Icarus learned the hard way
that not all technologies work just as intended. But you do not need
to go to mythology. You can go to the Mark
14 and 18 torpedoes. Do you do you remember those? What do you call it when the
torpedo comes back on you? It has a name, right? Other than, oh, my god, and
that’s not what you would say. Yeah. So usually, when
people say I was there, and they’re telling you
a military circumstance, and if every word
doesn’t start with F, you know they were not
there, and it’s not an accurate rendering of the
conversation at the time. But the Mark 14 torpedo
had a propensity to, one, hit the enemy
ship and not explode, which tends to clue the
enemy ship into the fact that you’re there. So that’s an example. I’ll give you a few others. Post-Sputnik, our Sputnik
moment, what did we do? We ran out and launched
two satellites. Both of them
exploded on the pad. Apollo 13, the
Challenger, the Columbia. Stuxnet. Sort of worked as intended,
but then it jumped the rail, didn’t it? And then Sydney Freedberg
and Matt Johnson have pointed out that even when
the technology works exactly as intended, there are enormous
interface issues between humans and technology,
when you hand off, and they use an example,
the Air France 447 flight which crashed. You remember, it
crashed in the Atlantic going from Brazil to Paris? And there’s a couple of
friendly fire instances using the Patriot batteries in 2003,
where the technology worked exactly as intended. But what happened
is the technology passed the con back to
the human in the loop, and they weren’t sure
what was happening until it was too late. Are you all familiar with
the Vincennes incident, which was the incident where
a US Aegis carrier shot down a Iranian Airbus,
commercial airliner? And that has been studied
as both an example where technology worked,
and the interface did not, and also a place
where AI-enabled machines would likely have prevented
that from happening. One of the issues
was, was the aircraft ascending or
descending, ascending indicative of commercial
air, descending indicative, potentially, of an attack. But if you really want
to stay up tonight, remind yourselves about
the Petrov incident. How many of you are familiar
with the Petrov incident? And this is an example
of the importance of having a human in
the loop, and the fact that technology does not
always work as intended. And this is an
incident from 1983, where Lieutenant
Colonel Stanislav Petrov was the watch officer at the
command and control center for Soviet rocket forces. And they got indication
of a US first strike, and he was on the clock. And under doctrine, he was
required to immediately inform his chain of command,
and up to the Politburu for responsive action,
and he sat on it. He said it does not look like
what it should look like. And he sat on it. And he was being
yelled at, yelled at. He was a lieutenant colonel. It takes a lot of
guts, especially in the Soviet system,
as a lieutenant colonel, to not follow direction. He did not, and eventually
the machine corrected itself. It was a technical error. Lest you think this is
good old Soviet technology, go to 1979, exact same scenario. National Security
Advisor Brzezinski is woken up in the
middle of the night by the DoD command
center, and told it’s a Soviet first strike en
route, and you have x minutes– he was told the specific
numbers of minutes– in which the president had
to be informed, and respond. He sat on it until the
system corrected itself. AI is instantaneous,
one of its advantages. One of its national
security advantages is, they can
instantly do things. Now think about that. Foreign relations impact. Larry Summers, who sometimes
gets things right and sometimes does not, has predicted that by
2050, one third of the world’s 25- to 54-year-old
male population will be out of work,
as a product of AI. Doesn’t matter if
he’s right or wrong. If a significant portion of
the male populace of the world is out of work because of AI,
that is a source of instability and, therefore, a
national security issue. AI can divide and further
divide the north-south divide. It can give new power
to smaller states, a version of the
Singapore effect, perhaps, supply chain and
counterintelligence issues. Think about today’s news
story about Qualcomm. Jurisdictional issues. Think about USB Microsoft. You can ask me about
that if you really care. Asymmetric threats, like
interference in elections, and the benefit to
authoritarian regimes. So that’s risk number whatever,
is foreign relations impact. Then we have the arms race. The arms race. Here I would say that
there is a community of AI think tanks that in great,
good, and honest earnestness, hope that we will not
be in an AI arms race. One of the SLMR
principles, and that’s one of the ethics rules on AI, is
that the principle is in fact– let me see if I have it here. Doo-bee-do. Ah, I do. The principle is that an arms
race in lethal autonomous weapons should be avoided. That is the principle. Goodbye. We are in an arms race, right? That’s the national
security imperative. The AI think tanks still wish
and talk about a Baruch plan. A Baruch plan was Bernard Baruch
at the end of World War II. The theory was to turn over
nuclear weapons capacity to the United Nations,
and have them control it. You are not going to
have a Baruch plan in AI, because there is too much
to be gained by it, too much security advantage. And who, by the way, is going
to trust Vladimir Putin, who thinks that whoever controls
AI will control the world. So we are in an arms race. And there are all sorts of
aspects about arms races that we will need to address. More on that in a moment, if
we have time and you wish. And then there’s the
existential risk, and this is where a lot of
people have spent their time. And this is where Elon
Musk spends his time. And I am not going to
spend my time on it, but let me just tell you
there’s three camps here. And this is the part that
people love to yabber about. And this is, when you get to
super artificial intelligence, will it be the end of
humanity or not, right? This is the paperclip
monster issue. And there are three camps, and
the first thing you should know is, there are generally
three camps here. Camp one is represented
by James Barrat, who wrote the book, Man’s Last Invention. It’s our last invention
because you’re gone afterwards. The institute at
Oxford that studies AI is called The Future
of Humanity Institute. That’s the stake they
think that’s in the game. So you have the camp that says,
not only is this a bad thing, it will be the end of humanity. I can talk about
this, but I’m not going to for a reason
I’ll tell you in a second. Second camp. Well, we’re not quite sure. It could be friendly, or
it could be unfriendly. This is the
fork-in-the-road camp. This is where
Stephen Hawking is. And the technologists say things
like, we’ll figure that out. Don’t worry. We’ll fork right, or
left, whichever way we want to go at the time. And then there is
the third camp– these are my camp notations– which is the
stay-calm-and-carry-on camp, right? And that’s largely
your technologists. That’s everybody
at MIT, probably, and that’s the people who have
the optimism that this is all going to work out, and
that’s because they’re going to write the algorithm
at the last minute that will make it work out. But one thing I picked
up on, partly because I’m a lawyer and a
judge, and I kind of notice funny words
sticking out– here are some of the
leading statements from the stay-calm camp. This is the Stanford
100-year study on AI. “While the study panel
does not consider it likely that near-term AI
systems will autonomously choose to inflict
harm on people, it will be possible for
people to use AI-based systems for harmful, as well
as helpful, purposes.” Contrary to more fantastic
predictions for AI– Elon Musk– in the popular
prose, the study panel found no causes for
concern that AI is an imminent threat to mankind. I’m like, wait. It’s not an imminent threat? So I can go to work tomorrow? And then back to my
friend, Ryan Calo. “My own view is that AI does not
present an existential threat to humanity,”– awesome– “at least
not in anything like the foreseeable future.” So that’s the optimistic camp. But I don’t worry about
the existential threat because I think we’re going to
do ourselves in before that, because we’re going to mishandle
AI as a national security tool and instrument. So I really don’t
worry about getting to the existential threat piece. So now this gets to us. How should we respond
to these risks? Are you happy with the timing? Should I talk faster? PRESENTER: We’ll have
30 minutes at the end. JAMES E. BAKER: Yeah, we will. That’s about where we’ll end up. OK. OK. So how should we respond? And here I have a
helpful 157-part program I’d like to present to
you, but instead I’ve decided to go with two
themes, because my goal here is to get your attention,
not necessarily satisfy it, because that would
just take too long. So how should we respond? First, we should start
making purposeful decisions. National security policy
should not be made by Google. That’s wrong, as a
matter of democracy. It’s wrong as a matter of how
the equities are balanced. And by the way, I don’t think
national security policy should be made by
the FBI either, which is my next point,
which is we should not make national security
policy by litigation, which is where we’re headed. If you want to know
what will happen if we don’t fill the void
on AI national security space with law, you will get
endless iterations of the FBI Apple litigation from 2015, over
the San Bernardino shooter’s phones. And litigation is a terrible
way to make informed policy decisions. Yes, it can serve as
a forcing mechanism, to do things, make
you make decisions you don’t want to
make, but here are some of the problems
with litigation. First, the Apple FBI
thing should also tell you that where national
security is concerned, the government will make
novel uses of the law. Do you remember what law was at
stake in the Apple FBI dispute? If you get the answer, something
very good happens to you. Ken will buy you dinner. AUDIENCE: The All Writs Act. JAMES E. BAKER: You’ve
got dinner right there. The All Writs Act. You’ve got to finish
the sentence though. AUDIENCE: No, that’s all I got. JAMES E. BAKER:
That’s all you got. All right, and you did not give
the complete and full name. What you meant to say was
the Judiciary Act of 1789. Get that? The Judiciary Act of
1789, probably not intended to address an
iPhone dispute in 2015. But that was the law that
was at stake in that case. Part of the Judiciary Act
is the All Writs section which gives the courts
the power to issue the writs necessary to
enforce their jurisdiction. So litigation, there will
be litigation, right, because unlike Apple FBI– Apple FBI is Little
League, compared to the litigation
that will occur in AI, where the stakes
are so much higher. Think about the amount of money
Google has invested in it, Microsoft has invested
in it, and so on. Litigation accents voices,
the interests and voices of a few parties,
not society at large. And the government process
for responding to litigation is very different. The government tends to– it gives you lawyers
making arguments that will win the case,
rather than lawyers making arguments that will serve
society best, going forward. And even in the
FBI Apple case, FBI did not necessarily reflect
the views of the United States government. The largest critics, the most
vocal critics, of the FBI position, in the
end, turned out to be members of the
intelligence community, who disagreed with FBI’S
perspective on encryption. And I think it’s
safe to say, or we can stipulate for our
purposes, that Silicon Valley, as a metaphor,
and the government work least well at moments
of crisis and litigation. It’s called the adversarial
process for a reason, and if you want informed
and sound policy, you need to do it in calm
moments, not in litigation. Ethics is not enough. There’s some ethical
codes out there. I will leave it to
the MIT professors in the room to tell
you whether they feel constrained by the
professional ethical codes that bind them. But I read to you one of the
principal statements of ethics from the SLMR code, and that’s
not going to do the trick. And I know, from
my own Model Rules of Professional Conduct
for lawyers, that defines the basement of conduct. Basically, don’t steal
from your client. And that’s not going
to do the trick. Law serves three purposes. It provides essential values,
national security as well as legal. It provides a central
process, and it provides the substantive
authority to act, as well as the left and
right boundaries of action. I’m going to give you
an example of each, which is about all I can do
here in the time we have. But this is where
I want you to start thinking about how
the law can serve to guide, in a proper
way, in a wise way, in an ethical way, the use of
AI in national security space. Starting with values, the most
important national security and legal values in this area
come from the Constitution itself. The values that will be debated,
and litigated, and tested, are the values in the First,
Fourth, and Fifth Amendments, right? We all know that. Perhaps you know that. The First Amendment,
why the First Amendment? Think about analyzing
Facebook postings for Russian interference. The government analyzing
Facebook postings for Russian
interference, how that might lead to a First
Amendment challenge, that the government is
impeding, restricting, or chilling your voice. Think about the AI
researcher who’s traveling to the United
States to work at MIT, and can’t get in because
there’s a travel ban. And think about
disputes over funding. I’m going to give you
your government funding, or I am not, depending on
what you’re doing with AI. All have First
Amendment dimensions. The Fourth Amendment, right? Everybody familiar with
the Fourth Amendment? In essence– so
that was silence– how many of you could
recite the Fourth Amendment? If you’re working
in this space, you ought to know the Fourth
Amendment by heart. The summary of it is, the
Fourth Amendment essentially protects you from
unreasonable searches and seizures by the
government, and in many cases, but not all, requires
the government to have a warrant
before it engages in a search or a seizure. Critically, it protects
you from unreasonable. So the whole debate here
is, well, what’s reasonable? And since 1979, the
answer to that question has come in the form of
the third-party doctrine. Just by chance, has
anybody in this room ever heard of the
third-party doctrine? For real. You’re not just playing it safe? I won’t call you. So the third part–
and this is critical, because think about all the
data, all the information out there on which
AI depends, depends, in part, in part depending
on how we interpret the Fourth Amendment, on the
third-party doctrine, which posits that if you share
information with a third party, you do not have a reasonable
expectation in its privacy. It is like, but not similar to,
the attorney-client privilege. If you have a privilege
between you and your attorney, but you tell someone
else the information, you are said to have
waived that privilege. But the third-party
doctrine is very different. It says, if I have a cell
phone, and I call someone, and the cell phone company
needs the information, obviously the numbers to stick
through, I’ve surrendered that information voluntarily
to a third party, the phone company. I now have lost my
reasonable expectation in it, from a privacy standpoint and
a Fourth Amendment standpoint. Never mind that the case this
all depends on dates to 1979. It’s a purse-snatching case,
and the piece of information they were garnering there
was information about who had called the witness. All it was was a trap and
trace on the phone call. It wasn’t content. It wasn’t all this data. It was just, was the guy who
was accused of the crime calling the witness to cause trouble? And the Supreme Court
said, nope, that gets in. That evidence is
permitted because you had no reasonable
expectation in its privacy. That is still the law. That’s good law today,
and it covers everything that you share. So that’s point one. Point two, the US government
generally takes the view, and correctly so in my
view, that if it lawfully comes upon your
information, it may use it as it sees fit, right? So once it has the information– perhaps it lawfully
obtained it this way– it now can use it for
other purposes as well, provided two other
things have not happened. First, provided the
government has the authority to use the information in
the first place, which is not a Fourth Amendment
question, but an authority question, and provided
that Congress, that the law has not otherwise
regulated that information. So, for example, you have
no Fourth Amendment privacy protection in that third-caller
information, or in the data you send through a
ISP, because you’ve shared it with the company. But Congress, in 1986, passed
the Stored Communications Act, which gives you an additional
right, a statutory right, to privacy. And that’s currently
under dispute right now before the Supreme
Court in the US Microsoft case, involving the storage
of data in Ireland. Why does all this matter to you? Because the whole AI
system is predicated on data and information
that you’re currently sharing with private parties,
and with the government. And if the United
States Supreme Court, which is hearing a case
right now called Carpenter, which could put this in play– it shouldn’t, but it
will because they’re very nervous about the
aggregation of information– it’s just not the
right case to do it. All of this could change
the AI playing field. The other thing is,
technology could solve some of the
privacy problems, and more on that in
a second, or not. OK. And then the Fifth Amendment. The Fifth Amendment,
for those who have not studied it recently,
the Fifth Amendment essentially provides
for due process of law, and it has the Takings Clause. How might both of
those come into play? Due process of law, essentially,
the theory behind it is, the government cannot
deprive you of your liberty, your liberty interests, or
financial interests without due process of law, an opportunity
to be heard and to make your case. Fair enough? Part of the AI algorithms,
with deep machine learning, is based on the concept
of the black box. And the black box, it turns
out, is the neural networks within the computer. So you put the input in, and you
get the output, and somewhere between the input
and the output, which predicts whether you have
the tumorous cancer or not, the algorithms wait and
classify the information in the pictures,
thousands of times through the neural networks. And so I’m thinking,
OK, I get it. I don’t understand
a thing about this. Let me go ask this smart
person how that works. And they say, oh,
it’s the black box. I say, this is
really interesting, but how does it work? We don’t know. We just know that the output
is accurate 93.2% of the time. Try explaining that to a judge. That’s not going
to work, and that might violate your due process. So that’s where that comes in. Takings Clause. Everybody familiar with
the Takings Clause? The Takings Clause
say the government can seize private
property for public good. There’s no question
they can do that. That’s how they build highways. But they have to pay
just compensation for it. How might that come
up in the AI context? What if you’re a company
in the United States, where most of the
research is occurring, and you invent a great
AI algorithm that can do great things, that
have national security value? Maybe the government will
want to have access to that. OK, so you can see
where that comes in. Process. I’m running out of
time, so let me– my processing is, the
second purpose of law is to provide a central process. You can’t have process without
structure and organization. And here, if I were to
pass one law right now, it wouldn’t be a
substantive law. It’d be a structure for the
United States government to organize itself around AI. Right now, there
is no structure. The lead bill in Congress
creates an advisory committee to advise on what the
structure should be. This train has left. Time to get organized and
actually make decisions. I’ve lived through this
with the cyber thing. In 1996, we were debating
who should be the lead agency in the government for cyber. What authority should they have? And these are the same
debates we’re having now. Skip the debate part. Get to the decision part. And the question
is how to do it. And if we don’t
do it, we’re going to have a DoD-centric AI
policy, which is great if you believe that
national security should drive national AI policy. But if you believe that all the
equities of the United States should come to bear
into the AI equation, you want a more unified
decision mechanism. Who’s going to go
out to Silicon Valley and talk through these issues,
and do so in a credible manner? My first pick would
not be the Director of National Intelligence or
the Secretary of Defense. It’d be someone who could speak
for all of the United States government. That’s my point. So I have some suggestions
here how to do that, if you’re interested, or not. So then substantive authority to
act, right and left boundaries. The lead law right here is
the Stored Communications Act from 1986. I’ll give you a hint. It’s not up to date, and it
will not be up to date for AI. Don’t respond in this
area with substance. Respond with
process, because law will always chase technology. The law never can keep up
with Moore’s law, for example. Note the use of law–
that was very clever. That’s an MIT joke, and I’m
very pleased with myself. What statutory law provides
the substantive authority? I’m talking about the
Defense Production Act. How many of you have
heard of that act? Good. You are now one of the leading
experts in the United States on AI law, because that’s the
essential act, in some regards. Why? Because partly that’s
where you get CFIUS from, the Committee for Foreign
Investment in the United States. As you read in the paper
today, one of the issues is whether a Singapore-based
chip company can acquire a US-based chip company,
and whether there is national security
impediments to doing so. That’s CFIUS. That’s the Defense
Production Act, and I can talk about that
all day long if you’d like, but the act has not been
updated to address AI. And if I wanted something from
the technologists in this room, among other things, I would
want to know, what should we be worried about in the AI sphere? I get it, if the foreign company
wants to come in and build a factory next to the DoD base. I get it, if they want to buy
Northrop Grumman’s aviation component. I need to know what I should
be worried about in AI. So I have a whole
section on arms control, but I’m going to skip that
so that we can get to Q&A. And then I want to leave you
with a couple of thoughts. The reason this
presentation, in theory, was called a
dangerous nonchalance was because here we are, with
what the Belfer study said was probably the
most transformative military technology,
as much as aviation, and as much as the
nuclear weapons. And it’s not what we’re
studying and talking about. I can guarantee
you it’s not what people are learning about and
studying in national security law courses in law school. And if you will recall a
time during the Cold War– and I grew up in Cambridge. You couldn’t graduate from
fourth grade to fifth grade without being able to
talk nuclear doctrine and throw-weight, right? Everybody knew it. This is what you grew up with. Nobody’s talking about
AI doctrine, and such. We should be. When will AI exceed
human performance? Remember, this is the
artificial general intelligence. A group out of the Future
of Humanity did a poll and asked that question
of AI researchers, and industry people and
academics, two years ago. The median answer for
AI-ologists from China was 28 years. The median answer for American
specialists in AI was 76 years. They’re optimistic
about something that we’re not as
optimistic about, but everyone’s
optimistic in the field about getting to this
artificial general intelligence. We need to talk and bridge
now, government to industry to academics, technology
policy to law, and need to do so
in plain English. I asked someone at
the Future of Humanity to explain to me how you could
do a due process algorithm to make something fair. And they gave me a 40-page
article with math equations. I can’t explain that to
anybody, including myself. If you’re a technologist
and you actually want to influence
policy, and shape it, you need to speak and
write in plain English. So that’s one thing. We need to make
purposeful decisions and we need to ask
the big questions now. One of the big
questions, for example, is what is the responsibility
and role of a US corporation? There is no common
understanding, as there once was. I’m not saying there’s
a correct answer. I’m saying, right now
the answer appears to be default,
company by company, and commercial interests. But we can’t have
that fight now, every time we have
a national security San Bernardino iPhone issue. Do corporations have
a duty, a higher duty, in the United, States
to national security and to the common good? The answer might
be no, no, no, it’s all about shopping algorithms,
and that’s just fine. But we ought to purposely
make that decision and make that an
informed choice. So that’s it for now. I do want to leave time
for Q&A, or at least one Q and maybe an A. So thank
you very much for your time and attention. I hope that’s not your question. OK. All right. Are there any questions? Yes, please. AUDIENCE: So Jamie,
you began your talk by noting that AI was
going to have effects– JAMES E. BAKER: Hang on. You violated Rule 9, name,
rank, and serial number. AUDIENCE: Kenneth Oye, Program
on Emerging Technologies, MIT. So Jamie, you began your talk– JAMES E. BAKER: Trying to
distract them right now. OK. That’s chaff. AUDIENCE: You’d expect
that AI would have effects that were on a par with,
or even greater than, long-range aircraft
and nuclear weapons. JAMES E. BAKER: I said
the Belfer study concluded that that would happen. AUDIENCE: Ah, notice the careful
distancing of the good word. If you go back to 1945– JAMES E. BAKER: Right. One of the things
I’m trying to do– I want this debate– so I don’t really care
what the answer is. I care that we have an informed
debate about the answer. And if the only thing out
there is the Belfer study, and everybody is
like, OK, thanks, let’s move on to shopping
algorithms, not a good result. OK. So go ahead. AUDIENCE: If you
go back to 1945, expectations on the effects
of those technologies were really bad. That would be talking about
being eaten, or eating. And, consequently also,
in domestic politics, so you had McCarthyism and other
fear-driven domestic politics taking place. What happened was more
benign, over the long term, with the emergence
of a period of– I wouldn’t call it
peace, but the absence of central systemic war, in part
by virtue of the technologies and the doctrines
associated with them. JAMES E. BAKER: OK. Question now. AUDIENCE: Question. Can you give us
ways, or discuss how we might avoid
potential bad things, recognize on certain
wrong predictions, and develop those doctrines
that we’re talking about? JAMES E. BAKER: Sure. So thank you. One thing, or five
things, I didn’t say by skipping the arms control
section was, one, the doctrine that we now– we’ll just stipulate. You don’t have to agree with
this– but the doctrine that at least provided stability,
and if not having nuclear war is the measure of
success, was successful, known by some as mutual
assured destruction, some, as others, of
assured destruction. By the way the Chinese never
bought into either concept. They simply said, we
have enough missiles to make you think and know
that you will get hammered. That’s enough. And the number of
ICBMs they’ve had– obviously, they’re not in the
business of telling you exactly how many, where, or when– but was always, at the
beginning, was in the 20s and 30s, and then at most was
in the 200s, based on open– I don’t know anything,
and you don’t know. So you don’t have to buy into
complete doctrinal unification, but here’s what I’d say
about the doctrine piece. At the end of the
Cold War, we sort of accepted and assumed no
one would do first use. Even if you had a declared
first-use policy or you didn’t have it, it was sort
of understood you wouldn’t. Of course, we didn’t know
about the Soviet Dead Hand at the time, which sort of
undermined the whole concept of mutual assured destruction. And secondly, if
you go back and read the development of the
doctrine, and here, I encourage you to read Fred
Kaplan’s wonderful book, The Wizards of Armageddon,
we didn’t wake up in 1947 with mutual assured destruction. We woke up in 1947
with Curtis LeMay, and what did you get
with Curtis LeMay? You got a policy of first
use, as in, nuclear weapons weren’t nuclear weapons. They were a better weapon,
more powerful weapon. And secondly,
Robert McNamara, not everybody’s favorite
secretary of defense– got that– but when he came
in as secretary of defense, he asked to see what was
then the PsyOps 1962, which was the second single integrated
operations plan, which is the nuclear response plan. And he was stunned to
find a couple of things. One, he found it was a plan. It was one option. Like pick from the following
options, A. And he was like, A? And what was A? A was, nuke the Soviet
Union, and as an added bonus, all of China too. And he was in the tank. And he said, China? And David Shoup, the
commandant of the Marine Corps, was the only of the
chiefs who said, what’s China have to do
with a Soviet first strike? We’re going to kill 150 million
Chinese that have no connection at all to the Soviet action? That’s what was in the plan. So doctrine looks OK, maybe
possibly good, at the end. But doctrine’s not born correct. So one of my things is,
if you’re writing doctrine when you deploy the
system, whatever the system is, you got a problem. You got to think
that through now, and you got to go through
your ‘let’s nuke China’ moment without it being on the table. I’m joking. That’s a joke for the record. That was national
security humor. I do not want a repeat of
PsyOps ’62 briefing to McNamara. I want it to be
nuanced, informed, and AI’s going to
speed everything up. So doctrine is going to
be even more important. At least with nukes,
you had 22 minutes to have a very thoughtful
and deliberative process. Not necessarily so with AI. So one doctrinal development,
which is what you can do here. You can build in, and
again, I’m out of my league, but I would want to know whether
there are technological ways to build in fail safes
and circuit breaks, so that we can choose
to have them or choose not to have them. But right now, I can
describe that concept. One of the key concepts in the
area of AI is man in the loop. And everybody is
going, oh, you need to make sure there’s
a man in the loop. Well, that doesn’t
mean anything. Try putting that into
an arms control treaty. There shall be a
man in the loop. They had years. It took them years to negotiate
what a delivery system was in the context of arms control. That’s easy, right? It’s a bomb, or it’s
an ICBM, or it’s a Polaris missile off a sub. That took three years? Try defining man in the loop. That’s the sort of thing
we should be doing now. What does it look like and what
are the options, so that when we agree that the law
should say there shall be a man in the loop, or the two
countries or the five countries agree, there should
be a man in the loop, we know exactly what
we’re talking about. Is this sort of responsive? AUDIENCE: Yes. JAMES E. BAKER: And I had some
other really interesting point, but I lost it. AUDIENCE: An accentuating
defense of the [INAUDIBLE].. JAMES E. BAKER:
Yeah, but hard to do. AUDIENCE: I understand. JAMES E. BAKER:
Yeah, hard to do. Yes, sir? AUDIENCE: My name
is [INAUDIBLE].. I’m from the law school. JAMES E. BAKER: MIT
has a law school? AUDIENCE: No, the
other one, the– JAMES E. BAKER: Oh, BC? AUDIENCE: The other
end of the room. The question is if it’s perhaps
blasphemous in this context of American national security
people, but I come from Europe. And there is some
discussion in Europe that’s a broad question,
that the United States is not as reliable anymore
as it used to be. And also there’s now a
question of the very companies and the very technology
that you outlined is strategic and important,
and that the Europeans, first of all, need to compete
more effectively, which is, I think, a lost case,
but also that they are now restricting the transmission
of data to Europe. And I’d be interested in
whether [INAUDIBLE] protection regulation that’s now being
discussed in the European Union, and whether these issues,
particularly with the European Union, are at all, and inform
at all, your thinking on this. JAMES E. BAKER: Yes. Next question, please. OK, fair enough. Let me just add the one
thing I just remembered. One of the areas we
can develop doctrine in is the principle of
command responsibility. So the law of armed– I’m not ducking your question. I’m just, I want to clean
up my mess over here. The law of armed conflict
provides some parallels which, I think, could
usefully be applied to AI, not because AI could
be a weapons system, but to control, to give it
structure and governance. So the doctrine of
command responsibility, among other things,
holds commanders responsible for
things their units do, that they should have
known they were doing, whether they intended
them to be done or not. This is the Yamashita principle
from the Tokyo war crimes trial. It’s one of the
fundamental principles to come out of World War II
law of armed conflict, which is, you are accountable for
what your troops do, and fail to do, or whatever, and
whether you intended it or not. And there are other
aspects to it. There’s a component of
civilian command responsibility as well, that has come out
of the Balkan conflict. So the point is not that it
applies mutatis mutandis, in all relevant parts
corrected, and to make it work, but there’s a principle
there that could be applied. There’s also a principle in
the law of armed conflict that requires, and
100 out of 100 lawyers would agree to this
and 100 lawyers don’t agree on anything,
that any new weapon has to be reviewed for
compliance with the law, and that’s not just the weapon,
but also the mode and method that it is being used for. Why not apply and adopt
that, in some manner, to other aspects of AI? It could be done. And then the law
of armed conflict also requires training,
and this is, in fact, done, and you can say, is
it done successfully? That’s a different question. In the US, generally so, but you
could pass principles and rules that required training on AI
if you’re going to use it. Again, it doesn’t fit perfectly
but those are the kind of things that we can talk
about and develop now, and you cannot do in
the heat of the moment. So on data, one of the drivers
of AI is, of course, data. And that has a lot to do with
machine learning, and also pattern recognition, and
I’m not wonderful at math, but obviously, the more data
you have, the more accurate your predictions are
going to be, right? If you only have
one person who’s bought underwear
and socks, that’s not going to kick underwear
and socks to you every time you buy t-shirts. But if that’s the pattern
with millions of sales, then your algorithm’s
going to predict that. In Facebook algorithms,
likes and don’t likes, there have been
studies indicating they’re better at predicting
your social behavior and your purchasing things
than even your spouse, and that’s based
on volume of data. And the Chinese have
regulated their data, to a certain extent. Now, do you want me to
talk about USB Microsoft, because there’s all sorts
of places to go with this. AUDIENCE: You’re the boss. JAMES E. BAKER: I’m the boss. I mean, you ask. I mean, what do you
want me to talk about, because Europe and the US
have very different concepts of privacy. And when I talked about the
Fourth Amendment as a value, and what was reasonable
and unreasonable, the US– and I’m going to grossly
simplify, so for the record, I’m aware I’m
grossly simplifying– US concepts of privacy generally
run us versus government, right? The privacy line is between what
the government knows or has on you, and what
everybody else does, which might have made
sense when you had– US v Katz was a case
about putting suction cups on a telephone booth
and listening, right, but it doesn’t
make as much sense when people are
sharing everything they’re sharing with
Google, whatever, fill in your favorite place. And using LINQ analysis
and algorithms, you can figure out
where you worship, what you had for
dinner, who you’re married to, virtually
everything about you, and you’re worried that
the government might get that to prevent terrorism? I mean, come on. The European grossly
simplified model is, privacy is about your
individual right vis-a-vis the rest of the world, not
just vis-a-vis the government. But my right, so for example,
the right to be forgotten, have something erased
from the internet, it’s not the
government’s database you want to get it out of. It’s the internet itself. And if you talk to the
people at Facebook, for example, who do the law
interaction around the world, in certain states, they have to
allow certain information out. In certain states
they have to block it. And this gets to the corporate
responsibility thing, which is, and not to put too
fine a point on it, if China is doing a social
accountability program, do you have an obligation,
because you want to have access to 800 million consumers, to
give them all the information they want to get? And should you? And in Europe, they would
never allow that kind of personal
identifying information to be shared in that manner. So different apple
and orange concepts. Both have the
concept of privacy, and to a certain extent,
government regulation and control. And then it emerges in
the US v Microsoft case, which is currently pending– oral argument was last
week in the Supreme Court, and could I just
generally say, and I think I’m entitled to say
this as a former judge, we did a lot of
Fourth Amendment law. Why did we do a lot of Fourth
Amendment law in my court, probably more than any other
court in the United States? Because of child pornography. Who looks at child pornography? 18 to 40-year-old men. Where have we collected 18 to
40-year-old men in one place and given them access to
computers and the internet? The military. And in the military,
you’ll get caught. And then there’s a cat
and mouse game, right? You get caught once looking
at it on your work computer– and that’s the lance corporal
you don’t want anyway, because they’re so dumb they’ll
fire the weapon the wrong way– so everybody is then
figuring out ways to hide the child
porn in black space, on computers, all this
unallocated space. And so, how to say this
politely with great respect for our judicial system? A 57-year-old judge
is not going to be on the cutting edge of social
media knowledge and technology. QED– that’s an
MIT thing, right? And so I cannot impress on
you enough, the importance, if you’re in the government,
or if you’re in academia, to articulate to a larger
audience, in plain English, why principles like the
third-party doctrine do or do not apply to new technology,
because if you don’t do that, the Supreme Court
will, or my court will, and common law courts know
to walk in baby steps, as Learned Hand said. But supreme courts don’t
because they’re supreme. And I don’t know this, but
I can be pretty confident, when they’re deliberating
US v Microsoft, they’re not talking about– but what about machine
learning and the future of AI? Actually, the US
v Microsoft case is not going to be that
exciting, I would project, because the real
principle in that case is, does the Stored
Communications Act– so I’ll give you two
seconds on the facts. Drug investigation
in the United States, a drug dealer,
alleged drug dealer, who’s being investigated,
has e-mails and other data on his computer. But the data is
stored by Microsoft, not in the United
States, but in Ireland. And this is normal process,
because you go where the computing capability is. This is cloud computing, right? And so you can bet a lot of it’s
in Iceland, for energy reasons, and so on. And now you can also bet
that companies understand that if they move their
data all over the world– and I think Google
actually splits it up. Microsoft, at least, puts
the whole message in Dublin, but Google is like brzzt! And so if you’re investigating
a drug, a narcotics thing, and you want the guy’s
email, which is normal– there’s a search warrant. There’s probable cause. So this is not a question
about whether there’s a Fourth Amendment issue– they then go to Microsoft
and say, congratulations, you have a subpoena. Please produce these things. And the question is, is
that in the United States or is it in Ireland? And the normal process by
which you get evidence overseas is through the mutual legal
assistance treaty mechanism. One of my first jobs at
the State Department, among others, was
to be the MLAT guy, and it literally is bows, like
tying ribbons on packages. It takes forever. It is a lousy way to go
about getting evidence, but it is the agreed
upon mechanism. Why? Because MLATs are
based on treaties, and when you agree
to the treaty, you can agree that that
country has a court system that is fair enough, and is close
enough to what we would expect, and why we would share
information with them. So we wouldn’t share
certain information with the Soviet Union. I meant it historically, because
we know that would be misused, and so on. So what happened in the
US v Microsoft case is, Microsoft came back and said– first of all, the district
court ordered them, upheld the magistrate’s
order, and said, produce the information. The Second Circuit
then split 4-4 on whether they had to
produce the information, and the case is pending now. It was argued before
the Supreme Court. Why do I say 4-4, and
why is that important? This is what happens
when you use litigation to decide policy. You get 4-4 splits, and
it’s very, very hard to have a national policy
if the Second Circuit has one view of the law, and
the First Circuit another, and so on. And so the Second
Circuit essentially punted, and then moved it
up to the Supreme Court, but the issue there is
really whether the Stored Communications Act has
extraterritorial effect. So people are all very
excited about what’s going to be decided
here, and it’s going to resolve all
future data issues. And the Supreme Court
will likely just say, the general
principle of law involved with extraterritoriality is that
unless Congress has expressly stated that something
applies extraterritorially, the presumption is
that it does not, because you do not want
to unconsciously create foreign relations
friction unless Congress has intended that you do so. So the Supreme Court
could duck the issue, and it’s not really a duck. It’s answering the legal
question presented, which is, does the Stored
Communications Act apply? That was an example
of a law that requires a certain
mechanism, and allows for the subpoena of particular
evidence, and the question is, does it does it apply overseas? So the Supreme Court
could come back and say, nope, doesn’t, but
Congress could certainly pass that law. And there is a law
pending, as you know. I’m sure you know there
is a law pending, which would deal with the issue of
shortcutting the MLAT process. Ireland and the UK,
for example, could go directly to a US carrier,
in certain situations, and not have to go
through the MLAT process. You see how complicated
this stuff is? And why you can’t decide
it at the last minute. And you don’t want it
decided in litigation. Yes, sir? AUDIENCE: I’d be interested
in your opinion on the patent office playing a collective
role in the extraterritorial use of patents that are granted
in the artificial intelligence world. JAMES E. BAKER: All right. It’s fair to say, I do not have
a particularly informed view. You can guide me,
if you would like. There is something called the
Invention Secrecy Act, which is a Cold War-era act, which
I have not, but intend to, look at with greater care. But the premise behind
the Invention Secrecy Act, a lot of the law, the
Defense Production Act, the Invention Secrecy
Act, were passed in the– the DPA, the Defense Production
Act was passed in 1950– in a Cold War context where
there was a unity of role and a unity of what a
corporation in the United States ought to do, which
probably doesn’t exist now. The DPA is renewed
every five years. There’s a vehicle to
address these issues, but we choose not to. Now the Invention
Secrecy Act says that if there’s a patent,
a request for a patent that is filed, and it is one that
has national security impact, and it is a patent that the
United States government does not want to see on the market,
or that it needs to have security regulation too– So let’s say it’s
a patent to build– I’m being silly,
but this is what they had in mind– a patent
to make a better nuke weapon, right? You don’t want to let
someone patent that like, forget plutonium,
forget uranium. It turns out you can
do it with apples. That’s a patent, you don’t
want like, great, you’re going to now publish that, and
the Invention Secrecy Act says, no you’re not. And the Takings Clause
says, you will now get just compensation
for us taking your patent and sitting on it. Thank you so much for
bringing it to our attention. And what I don’t know, and
thank you for the question, but what I don’t know– it’s a noble fact
I don’t know it– is how often, if at all,
the Invention Secrecy Act has been invoked, and for what? AUDIENCE: About 6,000. JAMES E. BAKER: 6,000 what? AUDIENCE: About 6,000. JAMES E. BAKER: Covered patents? I’ve done detailed
research on this topic. And it turns out that there are
about 6,000 so covered patents. To be more precise, 5,000. AUDIENCE: Thank you. AUDIENCE: What? Wow. AUDIENCE: Only five [INAUDIBLE]. JAMES E. BAKER: Well, that’s one
of the issues with the Takings Clause, and this comes up
in eminent domain, which is, what is just compensation? Guaranteed that comes
up, and so, of course, the government always has the– So here’s how it would
come up in this context. Here’s this AI thing, and
I’m the inventor, and I say, this is going to be
the best algorithm in the history of the world,
and it will make $700 billion. And the government says, this
is an invention that is really a intelligence
aggregation device, and it’s going to make nothing,
and it never did sell anything. Prove to us that you
would have made any money, and you can’t prove. You’re like, well, I never
sold it to Microsoft, and I never sold it to Amazon. So that’s worth $10. And so there’s a
compensation issue, just like when they
take your back yard to put the highway in. You say, nobody’s going
to ever buy my house. The house was worth a million
dollars, and it’s in Cambridge, so it’s worth $10 million. And you took my
backyard, so now it’s worth nothing, because nobody
wants to live on a highway. And the government says, no, no. That’s one-eighth of an acre
and one-eighth of an acre in Cambridge is worth $7. It’s not the house we’re taking. It’s just this one– so you get into all these
just compensation issues. Believe me you’ll get into
them with an invention secrecy issue, because it’s prospective. So that’s all I got
for you on that, because it’s a fair question,
and I will continue my research offline. Yes, ma’am? And then over here. AUDIENCE: I want to
ask you a question about your last statement in
view of your previous answer from the law school. It’s not a great example, but
it’s one that you keep hearing. The people who build
nuclear power plants are not responsible for the
nuclear weapons and so forth. In your social
responsibility, why do you feel that AI companies
have more responsibility than nuclear companies? JAMES E. BAKER: I don’t. My goal here was to identify it
as an issue, not to answer it. So the question is,
should an AI company have more social
responsibility as a corporation than a nuclear energy company,
than a anything else company? And I didn’t come here today
to espouse a particular view, other than I don’t
want to resolve that in the context of
FBI Apple litigation. I want to resolve that in
the context of dialogue between Silicon Valley and
the government, and academia, in the calmness
of a private room, without posturing, and
without litigation at stake. That’s all. I want to hear the arguments. I switched my view on
encryption, for example, by listening to the arguments. And if I’d come out
and just came out swinging in litigation,
I would have come out on one view on encryption,
but having listened to the arguments, and done so
in the glaze of San Francisco, I came away in a very
different place on encryption. And that’s what I’m asking
for, not a particular outcome, but a purposeful and
informed outcome. So we have a question here,
and then a question there. I’m happy to stay. I once did five
straight hours of Q&A, so you’re not going to wear me
out, but there may be rules. So yes, Perry. AUDIENCE: I’m Perry
[? Shtotson. ?] I’m an educator, and so I
consider this kind of the NPR question, meaning, could you–
for the intelligent normal person driving in their car
trying to understand all of this– say, at this juncture,
and it seems very urgent and critical, given that– if you could go to
Washington– and you’re very experienced in
Washington– and you could get, around this table,
the people from government, and industry, and
perhaps some ethicists and humanitarian people
interested in the well-being of the planet, who would
you have at the table? And what would you tell
them, and what would you do? Who would you want
to discuss this? JAMES E. BAKER: Right, right. AUDIENCE: Is that
too general, James? JAMES E. BAKER: No,
it’s fairly general, but ultimately, this
is a conversation that needs to occur. If it’s going to find its
way into US law and policy, it has to be a conversation
that includes Congress. It has to be a
conversation that includes the President of the
United States, which is a complex thing, I understand. And it’s hard to avoid it, but
this is not the time to say– it’s an awkward time to say, we
need the government involved. But we need the
government involved. The Obama administration
was on the path toward creating a structure, but
did not complete the process. So I’d have the chairmen
of relevant committees, and the problem is, this
is not one of those things. It’s like trying to explain
the law of armed conflict to a commander at 2:00 AM when
you have 17 seconds to make the strike call. You’re not going to
get the opportunity to say what you need to say. You have to have done
it ahead of time. And so my sense of urgency is
not one of imminent urgency. The imminence is trying
to trigger the process, not resolve it,
because heck, we have somewhere between 26
years and 74 years, depending on how you look at it. Now I’m joking, but so the
president, members of Congress. Silicon Valley is a metaphor. I understand Microsoft
is not in Silicon Valley. I get it. A lot of Google’s work on AI
is happening in Pittsburgh. That’s where the
driverless car program is. So those are the people. And then academics,
and especially academics with oversized
voices, whether they should have oversized voices or not. If you’re from MIT,
you have a louder voice than if you’re from some other
school, to be perfectly honest. I don’t know how to say that
other than to say it that way. And here’s an example
what happens when you don’t have this conversation. When you don’t have this
conversation, all of a sudden, you’re looking at
a tax bill that doesn’t understand
the importance of fundamental research. And there’s nobody better
at articulating that than the president of
MIT, but it’s too late to articulate it the
night before the bill’s going to get passed. That is not how policy is. You got to be relentless,
and say it often, often and early, like Curly. And so that’s the third
prong of the conversation. There are people who
are looking at this. It’s a fascinating topic
for me, because it’s more complex than any topic
the government has addressed before. It’s much more complex
than climate change. It’s more complex than Ebola. It’s more complex than
Deepwater Horizon, all crises the government was not
organized or prepared for, and that were anomalies,
or black swans as they say. This is the deluxe. This has everything, which
is both why it’s interesting and why it’s so hard. It has everything,
starting with an inability to even say what it is. And then secondly, it cuts
across horizontal organization. It cuts across
vertical organization. It has an academic component,
like others do not have. And so it is that much harder. So I don’t know if
that’s a full answer, but it’s a fascinating
problem, and it’s got to be someone who can
speak with the authority of the President of
the United States, because if you go to
Silicon Valley and say, here’s the deal we want to
cut, or here’s our concern, or trust me, this
really isn’t about blah, it’s useless if that
has no standing.

3 thoughts on “Starr Forum: Artificial Intelligence and National Security Law: A Dangerous Nonchalance

  1. Defining intelligence as a universal constant, so-called AI will reach it long before humans (in the general sense) ever could. Incidentally, the same is true of consciousness and mind. The very fact that this comment will elicit little attention is proof positive of that very fact.

Leave a Reply

Your email address will not be published. Required fields are marked *