thanks better
[Music]
[Applause]
[Music]
hello welcome back i am patrick tucker
technology editor at defence one our
next panel is
titans of technology winning the future
of defense
joining me is emmanuel brasson advisor
for
cyber ai and disruptive technologies
within the french ministry of armies
in the directorate general for
international relations and strategy
and dr heather roth senior research
analyst at the johns hopkins university
applied physics lab welcome to you both
uh quick reminder to our audience we do
want to hear from you we're very anxious
to
incorporate your questions into our
programming today use the globe sec app
to send us questions and i'll be able to
see them here uh later on i might uh
ask the room if they have questions
please raise your hand and then i'll
call on you and then you can approach
the microphone uh
so please keep that in mind we do want
to hear from you
let's okay
uh one second uh emmanuel can you hear
me okay
let manuel fix this sound problem real
quick let me turn
instead to you heather real quick how
can the international community
strengthen the regulatory framework when
it comes to the ethical use
of ai in theater operations
thank you pat and thank you everyone for
it
i'm very honored to be asked to be here
today and i just want to let you know
that i'm speaking in my personal
capacity
so these are uh personal opinions they
do not represent
um johns hopkins or the department of
defense or anybody else or brookings or
anybody else
um i think one of the big things about
strengthening
ethical implications around ai in
theater operations
even when tactical operations
um joint operations i mean any really
any operations generally
um is that we have to start with the
technology and we have to start with
worse
warfighter center design um
so when we're thinking about how to best
research design develop deploy um
we have to think about what the exact
technology is
is going to be used for and then uphold
certain types of principles so
the u.s department of defense for
example has adopted a set of five
ethical principles
um governable responsible equitable
um there is a there's a variety of
things that are
that are that are in that um and a lot
of that have to do with
designing for traceability and
auditability ensuring that
warfighters are center to design and
deployment
um and that human responsibility is at
the core
of all operations and so i think when
we're thinking about how we
deploy ai within defense of operations
we have to think about what the ai is
intended to do
is it a decision aid is it an isr
application
is it a strike application do we have
the right kind of data is the data
properly curated
do we have uh appropriate testing
evaluation verification validation
regimes in place to ensure that those
systems are working as intended
do we have some way to observe those
applications as they're deployed to
ensure that they are
working within the bound of intended use
the boundaries of intended use and that
they are not learning
behaviors outside of those um that
intended use scope
do we have some measure of auditability
how do we how do we train and equip
officers and operators to
continually understand those systems
how do we certify and license
autonomous technologies um to ensure
that their lifetime use
is is in accordance with what we want
them to do
and and that they are not learning um
unintended behaviors outside of that
so this is a very big endeavor
that that really spans doctrine
organization
training logistics design development
um this is a huge endeavor that i think
all militaries really have to
to take into consideration from
everything from policy to
how many maintainers um you need
for a given platform and what types of
training those individuals need
how you train lawyers and
judge advocate generals to ensure that
they are making appropriate
proportionality calculations
and understanding the implications of
using particular assets in the use of
armed force
this is this is not a one-shot problem
this is a
huge problem that all militaries are
ultimately going to have to
spend a great deal of time thinking
through
so in in covering this space uh one
thing that's been very clear to me and
to a lot of other people that are
watching the effects of emerging
technology on military technology
uh is the accelerating effects
of things like machine learning and big
data on the pace of
operations potentially on the pace of
warfare
and that seems to put a lot of potential
future stress on commanders
and others to make decisions ever faster
with an ever
shrinking uh observe orient decide an
act or ooda loop cycle
so uh having an ai principles list at
the department of defense is a huge
accomplishment
it's uh i think something that
international allies are very excited to
see
but is that sufficient to your mind to
deal with
the threat posed by accelerated
operations
due to the advent of new technologies
like machine learning is having
a principles list enough to offset
the potential harm there
well no you you can't just have a
principles list and say job done
right um what really matters is the
application and the institutionalization
of those
principles um the joint ai center within
the dod right now
is is working very hard on figuring out
the ways in which
they are instituting those principles
um again like so go back going back to
testing and evaluation for example like
what does that look like
um you know we don't have a very robust
testing regime for
for machine learning systems right we
can't exhaustively test
the space of machine learning we can do
simulations we can try to find edge
cases
but in reality we can't fully test these
systems like we have tested
prior previous legacy systems and so
there's a there's a fundamental shift
there
um but i think you know even with the
speeding up of operations
there is there is an inherent limit
right i mean
we are talking about i mean cyber is one
thing we can talk about cyber and how
that
um is a bit of a different space but
when it comes to physical operations
right we we are still
limited by time space
logistics you know how many how many
worships do i have
um in an area how many planes do i need
to
to complete a mission what does my
resupply line look like
how do i do joint operations with nato
right how do i
do my coms lines how do i divide air
space there are limits
um to operations given the physical
constraints of warfare
but we do have a huge amount of data
that's coming in
um we are pretty much
observed all the time i mean when we
think about about
uh space assets and isr uh there's just
you know your adversary can see you all
the time and what you're doing and what
movements you're planning um and so that
becomes its own challenge
um but what i would say is that
you know the the ooda loop is speeding
up
but it can only speed up on the quality
of data just because you have data
doesn't mean that it's useful
it has to be curated it has to be
appropriate for
the mission it has to be representative
of your adversary's intentions or
beliefs or assets
and when we're in warfare you know
your adversary is not going to tell you
exactly what they're going to do
they have incentives to dissemble they
haven't
they have incentives to use ruses
um and so the data that we're getting is
not clean it is not perfect
um and even with massive amounts of it
uh we there's a lot of noise in that
data
let me turn to you emanuel uh thank you
for joining us the same question
on the one hand we know that artificial
intelligence and emergency technologies
are going to be a force multiplier
they're going to allow militaries to
have
much greater effect with fewer assets at
the same time they also contribute to an
accelerating pace of operations
which can be uh very difficult for
commanders particularly commanders
that live and operate in societies
where there's a strong sense of uh
government accountability
command and control and where individual
officers are held accountable for their
actions as opposed to
uh other types of governments so can you
describe a little bit of that friction
from your perspective uh from a military
perspective how
on the one hand do you use technology
like ai
to be a force multiplier to have as
maximal effect on the battlefield as
possible while at the same time
allowing commanders to deal with the
accelerated pace of operations
can you hear me yes we do
[Music]
within the ministry of defense we have
identified a
number of fees where the embedding of
artificial intelligence can help in
this multiplying effect that you just
described
whatever it's of the battlefield
whatever it's in the cyber space for
intelligence information whatever it's
on the materials for logistics
maintenance
uh on the medical for rescue evacuation
and so on
and uh of course uh uh don't forget
the the the i should say the masterpiece
of all these fields
uh which is the uh the the actions of uh
planifications decision makings and uh
situation awareness in all these
in all these domains the artificial
intelligence
is uh is a factor of uh
accelerate acceleration uh
multiplication of
efficiency and uh improvement of the
precision
uh meaning with this
taking better decision taking more
precise decision or at the
at the better moment or
calibrating more appropriately the
resources and the materials you need
in the on the field
of course embedding such
algorithms and artificial intelligence
and data everywhere can lead to
increased complexity for the commandment
this is also one of the reasons for
which we have
identified the uh as other
critical uh as a criteria of
critical importance to uh to remain
within
a rigorous chain of commands when uh
performing the operations
and in particular uh the question of
uh maintaining at at every moment and
at every level a sufficient uh
human control on all these operations
uh excellent let's go to zoom and take a
question from jim townsend in the united
states
hello everyone uh this is jim townsend
uh just a sound check can you hear me
i hear you good uh well thank you very
much i found both of those
uh presentations absolutely fascinating
uh
i uh my background is i've spent about
35 years working nato and europe
at the pentagon so i i'm not a technical
guy
but you've got to keep track with
technology particularly dealing with
nato
and make sure there's interoperability
and that's my question for both of you
nato yesterday uh this heads of state
and government agreed to establish
uh an innovate an innovation accelerator
which is a great name uh as because
they're
concerned the alliance that um that nato
allies keep on top of the technology
changes and i know some allies are
uh but what does this mean for you what
how do you coordinate with nato
yourselves
uh dod i mean i worked in the pentagon
for many years
there is some coordination with nato
through the regional offices there and a
little bit on the acquisition side but
on the tech side
um with this uh this new innovation
there's also a fund
that nato is going to put towards r d
and tech
uh how are you going to negotiate how
are you going to uh
to inter-operate and to to communicate
and to
coordinate with what nato is doing as an
institution
and what nato allies are going to be
doing individually or we can be doing
things in stove pipes
thanks i'll take that to you emanuel
first
oh and then and then heather yeah
okay thanks uh this is a very
interesting questions
how should allies coordinate with uh
between them and at the alliance level
to
uh incorporate and to take the best uh
the better the best advantage of
such technologies uh so first of all i
should say that
uh the the most important thing is to
to understand ourselves mutually uh on
uh what we're talking about
it looks like basics but in case of uh
usage responsible use of artificial
intelligence and especially in the
defense sector in the military domain
this is extremely important
because we know that the the the limits
are not
easy to not uh not easy to define
and to uh i would say to translate
into concrete actions programs and
developments uh
also about the developments and the the
military program one of the
question i i think is still to to be on
the table
is that these technologies whatever we
we talk about uh ai in itself
or our other
disruptive technologies uh the point is
that these technologies are
uh uh inherently dual
uh dual civil and military
which means that we have to decide uh uh
between between a member of the alliance
what will be
uh to be uh to to be led by the alliance
either the military alliance
and well as to be developed and under
the responsibility of
of states i think this is
important especially when we when we
talk
about making use of these technologies
in a military context
in a context of of uh potential uh
conflicts when we have a command
reference which is the application
the application of the existing uh
international human humanitarian law
for which we have to uh to think of
course on how
and how concretely and by which means
the ihl can apply to the to the military
use of these technologies
heather you yeah i
jim thank you that was a wonderful
question um i think there's
multiple ways in which nato and the
alliances
um and not just nato i think even for
the us case and dod the five eyes
um i think there's there's a there's a
lot of things that need to be worked out
and one of the the i think most pressing
things
and one of the most uh big elephants in
the room is that
you know even though that we're gonna
start using ai and different types of
applications
um and different militaries are gonna
use different a
different types of systems because of
their of their national defense posture
so we can think of
you know britain might be leaning more
into
naval operations the u.s might be trying
to do more
um ai and autonomy applications at the
get-go within the air force
and so there's going to be kind of
there's going to be variation between
the the member states about what types
of applications that they're going to
pursue and then develop
and then deploy within their own
militaries
but we are also always hamstrung by our
networks and our communications
and so you know until we can figure out
a better way to do
those types of networks and
communications and interoperability
between members um and train we have to
train with our members
um these are these are going to
the legacy systems are going to always
hamstring
the new systems right we are not going
to
um kind of overturn the apple cart and
say everything's new
everything's not new we're going to
still have link 16 we're still going to
have different types of communication
systems
that we have to work with we're still
going to have to divide air space in in
such a way
so you know my patriot battery is going
to be over here and you know
your missile defense system is going to
be over there and so
i think we we have to continually think
about how we
we train and we and we equip our
fighters now
and then how we feather in these new
types of systems into
our existing uh forced postures and then
what types of missions that they are
going to be appropriate for
and to and to emmanuel's um you know
points
you know yes ihl is is an overriding
concern
right we were we are not going to field
systems that are indiscriminate
um that do not um comply with with the
laws of armed conflict
um but how we prove that they are
discriminate
how we prove that we have sufficiently
tested them
um is going to start to shift and going
to start to change because
of the way in which these systems
operate they're they're fundamentally
different so
i think as we are rolling out new um
applications and new
platforms that utilize ai we're gonna
not just see these as machine learning
systems that
you know are just learning all the time
no we're going to put
systems that have a hybrid architecture
out that have different types of
governors on them
you know that um that are not machine
learning all the way down right that are
going to have expert based systems that
we can fully
uh test and validate that you know
ground collision avoidance systems
they're going to have geo-fencing
there's there's just going to be a
different
um kind of stepwise approach to how we
how we feather those in
and and the allies within nato are going
to have to just be much more robust and
transparent about
what those systems are able to do and
how they've tested them and their
communications networks and how they
train with them
um it's just going to be even harder for
for allies to do that
going forward because it's it's not the
same old technology that i can say this
bomb is going to explode
in this location at this time every
single time i drop it it's it's going to
be it's a behavioral shift
more technologies more challenges in
that interoperability space for allies a
very important point
uh new question coming in from zun
from maurizio michoulan from mexico
maurizio hello
great to be here and and very very
interesting discussion
from mexico uh let me ask you this we
we have been hearing from
uh from these issues since long ago we
have been reflecting
on the ethics of artificial intelligence
since i remember
however we are subject to artificial
intelligence mechanisms when we use our
cell phones or our computers
every day just to get profit from
different
companies so how how serious
is this reflection when we speak about
national interests when we speak about
enemies when we speak about uh
interests and agendas how how
how should we approach this in
war scenarios that are happening in the
world
uh every day what what would you
what would be your approach to this
manual yeah
thank you very much this is a
very important question in the sense
that
we are talking about uh what especially
what we should do or not do and what our
potential adversaries
would allow to to themselves to do
[Music]
our goal is to uh to stay within the
uh strong ethics in our in our actions
and in our use of these technologies
so basically we want to uh to maintain
a core of uh of a fundamental principle
uh uh i took previously about the
respect of the
international humanitarian law the
respect of the chain of command
and the uh the the the the fight of
having a sufficient human control at
every level
and at every moment
[Music]
giving this principle we have to uh to
build on uh
some uh we have to do some i would say
some
doctrinal exercises to understand
uh how we should respond and how we
should behave
if at some point we would be facing an
enemy
that would not respect uh the law and
the principle
like uh the same the same as we do
what do we want to do for our soldiers
we want to protect them in their actions
but at the same time we want to remain
exemplary in the way that we are
performing our operation
so this is a reflection which is
uh very important to to incorporate at
the
from the very beginning in our
development in our programs and in our
operations
very very to to to simplify this this
the the ethic uh
the the ethic of our behavior is to be
taken into account
at every step since the beginning
in order to be as performant as we can
but
staying within the limit that we have
defined previously
sorry to cut you off emanuel i didn't
mean to heather did you have a thought
on this
um yeah so you know i was involved with
the creation
of the dod's um ai ethics principles i
was the special governmental expert
attached to the defense innovation board
writing those principles and so
you know that was a very long exercise
and what i can say is
that um you know to emmanuel's point
you know there is a long history of
of attempting to have a professional
code of ethics
um of how to how to fight
within the bounds of armed conflict um
and
most modern western militaries take that
that professional code very very
seriously and so
um where it changes with ai i think is
we have to think about what's different
right like this is
you know how are we going to fight
differently and how are we going to
uphold those same ethical codes
um of you know command and control of
um of of upholding loac
of of upholding um principles of
distinction and necessity and
proportionality
um we have to think about how we design
and develop
and test and train um and these are all
very big questions and and they're not
easy
problems to solve um this means that
all of the militaries going forward have
to take this as a serious commitment
you know you're not just acquiring a new
piece of kit
and then disseminating it throughout the
force and everything's going to be fine
this is the way in which you interact
with systems that learn
the way that you interact with systems
that behaviors might be slightly
unpredictable within a set of bounds
you have to you have to equip commanders
and operators with the appropriate
knowledge
and how to use that when to make the
appropriate judgments of when to use
that type of
of kit you know if i'm looking at a
loitering munition or i'm looking at
a sub hunter or i'm looking at um
a swarm all of these things are going to
be
based in a very big set of human
judgments
with intelligence uh reports coming in
with satellite imagery dual
authentication of all sorts of things
and so humans have to deal with an
increasingly complex space
and make the appropriate judgments in
line with their moral
sets of values but i do think that
having
been part of that process of enumerating
what those principles are
the militaries are very much committed
to doing that
um and even if our adversary says i
don't want to play by these rules
that's what sets us apart as
as as modern militaries who do comply
with a set of normative values in
international humanitarian law if we
throw out those
principles then we have no moral
standing
on the international stage and civilians
will ultimately suffer
not just our warfighters but civilians
will suffer and so it's very important
to maintain
that set of values to minimize force and
to protect human life as much as
possible
if we have to start using force
i think it's a vital point and if you
aren't familiar with the
ai ethics principles i encourage you to
check them out because uh
uh in many ways they are a blueprint for
the way
uh very large corporations should be
constructing ai principles
uh the military's ai principles the
united states
uh runs several pages dozens of pages as
opposed to
uh some that you see coming out of
silicon valley that are maybe 800 words
so uh it's a real feat but uh we have
time for maybe one last question and so
i'll uh turn it to you heather
and if you want to chime in a manual go
ahead but
watching this space over the years
there's lots of different ways to invest
in artificial intelligence you can
invest in
new startups that are developing new
algorithms you can
invest in more data sets and cloud to
host that data
you can invest in more testing and
experimentation uh
to help reveal new plateaus of human
machine teaming so
to your mind where is a military's
research money best spent right now is
it in testing
is it in procuring new flashy
solutions out of silicon valley is it in
perhaps cloud hosting and creating that
uh
technical basis for interoperability
that was just discussed
where does a military invest now to best
harness this future i'll
turn to you heather first that's a great
question pat
um i because everybody it's a
it's it's a zero-sum game right we only
have so much money so where do we put
that money
um infrastructure is is
really the key and and and
it depending upon a state's national
interests
and where they are going to pursue ai
given those national interests
i think infrastructure is key so that's
going to be in how do i
uh create maybe it's cloud
right how do i best secure cloud
computing storage
and and do so um effectively
how do i upgrade my networks how do i
get more bandwidth how do i
make data easy to to be interoperable
and accessible
across a force to repurpose and to
relabel
um how do i then um create that testing
verification validation pipeline so that
i can do
software as a service instead of having
to do
acquisitions in a very old way of
30-year cycles and upgrades and someone
walking around with a thumb drive
and going to each computer so i think
really the backbone
of of any force right now is
is the really not sexy stuff it's the
infrastructure
about data hosting cloud computing
software as a service
networks and communication bandwidth
that will enable
a force to get the shiny bright widget
and i'm afraid we're going to have to
end on that note we are out of time
uh thank you both for being here today
thank you to our audience coming up next
in habsburg at four
we have a session called democracy in
the digital space an alliance for a
healthy infosphere
and on the mariturisa stage european
cooperation with representatives
of ministries of defense from slovakia
the czech republic and sweden
so thank you all for being here thank
you to my panelists
thank you very much
[Music]