hello hey thanks for the introduction
so hello everyone my name is Sheena Tai
from Purdue University so today I'm
going to talk about our project PI
theorem Oh Oracle for the mazes so this
is a joint walk with Professor matheus
pair from epfl
and my advisor in town from UCSD so let
me start my talk by discussing that has
sent a little bit here so there isn't a
security it's really important
especially more and more data a process
and manage inside and within a big data
center in the last few years so the
lesson that has been the place of attack
it's really important to protect our
data in telecenter so let us take a look
what it has in the needs and recent
chance so there are no demands faster
network large memory and also low sip
utilizations that's why our DMA fits
this chain so let me introduce our DMA
here so our DMA stands for remote
directly memory access so in comparison
of traditional classic communications
it's the classic communication goes
through the user space applications a
trap into kernel go to and the request
arrives at the remote site it would
process by the CPU and back to the
memory again
so in our DMA the request is issue from
the user space application goes through
the library and interact with the
hardware directly when the requests
arrive at the remote site you interact
assess the memory directly therefore no
longer kernel and CPU are involved in
our DMA so with our DMA this design can
read and write remote memory it bypassed
kernel at both local and remote I
finally offers memory zero copy because
of Lee's design technique our DMA offers
low latency high throughput and low CPU
utilization because of the performance
benefit of our DMA recently more in
monitoring our centers adopt our DMA
technology to improve the applications
performance for example like Microsoft
Azure
they move to our DMA and also Alibaba
cloud they stopped our DMA so there are
also a lot of our DMA based data center
implications in both academia and
industrial
however most of them either focus on
performance
scalability and usability so then what
about security
it's really important data centers so
let me present you the path eeeh the
first session attack on our DMA so let
me introduce our threat model first
before I go into the detail of our
design so they are two sizing our model
the client side which includes the
victim may also an attacker then outside
server side so server side host data
inside the memory exported to users by
our DMA so the attacker and the victim
can be on different machines and the
attackers want to learn the SS pattern
of the victim for example victim says
get key 50 then the attackers want to
know think the victim really s at least
key so we assume that attackers cannot
observe the network traffic from the
client side otherwise it can tell us the
answer directly so now we are using our
DMA therefore all the process and other
requests will be processed by the how
we're NIC our DMM Inc so let's just look
into what is inside the our DMA Inc so
we can see the classic communication
first so in classic communications
because all the requests are handled by
CPU or processors therefore request will
goes into a regular NIC and pass for CPU
then we let us if you do the address
address mapping information tracking and
resource isolations so because there's
the massive overhead of using classic
communication because CPU is involved
so in our DMA we because no longer we
need CPU here so we need a stronger we
have a stronger argument NIC and also
more complicated so we in our DMA it
relies on a concept which is called
memory region to handle the address
mapping information tracking and all
things so now the card does the address
translation which is handled by CPU
before so in the memory regions there
are two main things the first thing is
our key and L key so our key is a key
who to provide to remote client to let
it do the permission checking
so the Nico would check the key when the
request arrived also the address is used
for the address translation
so let us look into lisa mejia date a
little bit more so our mainly nique
there are three types of metadata the
first type is cue pairs which handles
the connections the second type is
memory regions which does the address
translation and permission checking
Farley is page table entries so thus the
address translations because of the time
limitations
I will mainly focus on page table page
table entries today which is also called
PDE so when our DMA request comes in a
rice at the remote site it includes a
virtual address and a key then our
diamonique would check the keys and does
the address translation and get a
physical address and access the memory
accordingly so let us see this example
when requests come when we request a
rise of the remote machines it would
check that the NIC check the cache by
the metadata it's the heat then you
access the memory so it's really fast
it's really fast so let us look at
another example what request comes in he
found its miss so all Omaha data are not
inside the RDMA cache an aria may need
cash
therefore he needs to fetch from the
host machine to get the PD and other
metadata you'll get a one by one so at
least part creates a little bit slower
after bringing all the men how they have
fetched back to the audio Minnick you
can finally access the memory so these
parts create the timing difference this
is just a channel with this cover
it's a side channel so let me introduce
the PI Thea is a set of sectional
attacks that we introduced on our DMA so
the basic of PI C is a big big and real
o for example like if we let the victim
s as a submissive page through DMA you
will bring the metadata into the area
meaning land attacker evict to clean the
metadata well wait a little bit if the
victim has the page again you will bring
them in our data into the Nick again the
attacker does the real oo operations and
timeless operations if it's fast
it means SS the victim access list page
in the last time window let us look into
another example victim Isis first
attacker evict
victim SS some other pages
well technically low it would be slow
it means the victim didn't as at least
page before so title can read read read
Ulis attack round but again and again
then the attacker can learn the SS in
pattern of the big team so why we do
reverse engineering because we found out
like the basic in V big every low is
slow because now we cross the network
it's already ma so the best thing evn
reload takes around 25 milliseconds not
70 minutes so after our reverse
engineering we improved efficiency by
500 times now we only need 15
microseconds to do one round of pi/3
attack so now I will introduce how we
did our reverse engineering however
because of the time limitation I will
not go into detail today so please check
our paper for more details
so we suppose that our Dominique cash
will be similar to a CPU cache less
there are certain ways so we use
different we use the sets fixed size to
find out in despot and we also use
different eviction set size to find out
the potential ways Tolley we found out
is a K and 128 now we see that we found
a prefetch effect inside the REM unique
when a page when a page is SS you will
also bring the clothes may have data
into the REM unique which facilitated
computation also the SS so at least we
found out that there are 1k set and
we've we also found out that some of the
pages the accuracy when we use our model
to be big is not a high it's only 75% so
if our layer may be another set so we
find we found the high index so the
metadata will either be in the index
who either be in the set pointed or
directed by the law index with high
index therefore in order to improve the
accuracy by Thea attack evict both set
to have a good accuracy so Lee speakers
summarize what we discover with reverse
engineering so this is the completely
out of our knee our Dominique we observe
from experiment so let me show you the
speaker
this is a timing discern this is a PDF
figure which discussed the latency so
the excesses is latency microsecond the
y-axis is person its percentile so the
greater and the right hand side is
greater licensees so we know that what
our DMA request requires page table
entries and also the keys to do the page
translations so if all the metadata is
inside the RDM unique this translation
will be fast so it would be a black line
means heat if page table entries are not
inside the RDM anique it takes a little
bit take some time to fetch it from the
host machines memory you build green
lines a little bit slower finally if all
the metadata include the memory regions
are not inside the RDM Aneke we need to
fetch all that you'll be yellow line so
we can see a clean separation between
these three assets so this is the side
channel we discover so this is not only
in the connect export the NIC lab we use
we also found the same issues on the
different generations of our DMM mix for
example like the latest available our
Dominic's from our next connects 5 we
can see the same things even in the cow
lab which is a public cloud we see the
same issue now we show that pi Thea can
also deploy can also be deployed on Rio
applications therefore we attack the Rio
plication which is called Apache Krell
so CRO is the research project
originated from IBM and it's a Java
based storage platform so we picked a
key value store interface and also the
RDMA communication and to show our path
via attack so how to attack creo
so in our project they are three
different types of attack different ways
to attack Krell the first way is called
Emma base so in my theory my based
attack we need a helper process on the
machine which is co-located with the
cloud service provider so we like the
victim access a page through krell first
you bring them are may have data into
the RDM Annique then the attacker will
in communicate with the helper process
to evict all the memory region may have
data from the NIC the attacker does the
reuleaux these part leaks the SS in
pattern through the side Channel
for the page table entry based attack he
also requires a helper process on the
prowl service provider but now we use a
page table entries metadata's it brings
the page table entry area into our
Dominique the helper process evict
metadata's attacker does a reuleaux
these parts list as SS in patterns
finally we have a chi based attack
pythonic tile based attack on Krell
so in this attack is more powerful than
the previous two so now we don't use
only the Krell client to attack we no
longer need to wrong any helper process
and crayon machines which is fully a
remote attack so in Lisa attack the when
waiting when victim has a page through
crown you bring the meta data into the
our diamonique land attackers just issue
a set of a set of Crayola quest really
request like it can evict the victims SS
Minnow data then reloj then leaks the
ssing pattern so this is the most
powerful one so let me show you the
speaker this is a same figure swallow
illustrate before is still timing these
different speakers so you can see the XS
is the latency even with the Java and
also application overhead we can still
see the timing difference between
different assets so this is the side
channel we discover so based on this
side based on this side channels and
timing difference we have list result so
let me show you the client base that
hacker result so the XSS is timeline in
milliseconds the y-axis is the SS
probabilities so the attacker will keep
predicting the SS probability of victim
whether the victim is a specific page in
the last time window so we let the
victim around zippy and workload a
million or a million pages so through
creo so the inter arrival time of each
SS is based on Facebook keyboard store
work law the inter arrival time so the
Red Cross means the victim assess the
success specific page so the black tile
is the attacker keeps using the PI 3
attack with client base to see whether
the victim is or not so you can see
lesser overlapping
the Red Cross and also the black dog
leashed means the Pythian client-based
attack can really observe the victims as
icing patterns so we introduced the
attack of the RDMs I channel the section
of text on our DMA so let me discuss how
to mitigate least I channel attacks so
the first idea is to do the client-side
mitigations we can introduce noise
inside the software for example because
this is a timing channel attack a timing
difference attack therefore if we put a
noise inside the krail we can mix the
sizegenetics harder this is the first
idea second ways is we do the network
traffic monitoring inside at a network
side it could either be in a hub or in
the switch to see earlier any malicious
section attacks requests so finally is
the server side mitigation
this is stronger than the class I also
the network side because the overhead so
server side has lower overhead and
doesn't requires it can also avoid to
use the Machine which is controlled by
the attacker the class I so a server
size we use the huge we can use huge
page or physical memory registration
with so it means avoid get rid of the
virtual memory registrations so this can
reduce the overhead of the or address
translations which can make their
side-channel attacks harder so the
second idea is to do the hardware
resource isolations on our DMA neke like
like what I mentioned both in the mi
base attack and also the PT base attack
it requires on the helper it requires a
helper process running with the crown
machine on the crown machine therefore
if we can have the resource isolations
we can avoid these two types of tag
which can improve the security finally
the separate resource
between different connections and which
is called protection domain when we
disclosed list vulnerabilities to
Melnick's the audio main providers they
hinted demarest engineers hinted our
list solutions the production domains to
however we found out as far as we know
most of the RDMA based applications they
only use one production domains because
using multiple production domains would
bring the performance loss
in our experiment we can see 15 to 25
percent performance overhead when you
use multiple production domains so let
me conclude today's talk so we know the
Rema performance is really good
that's why data center engineers and
also the managers I want to bring the
Rema into data center to improve the
applications performance however how to
add the security guarantees to our DMA
will be really challenging the main
issues because there's also traders
trade-off between the performance and
security our DMA is attractive because
of its good performance once we add the
security guarantee we add overhead into
the our DMA the argument performance
would be a little bit slower where our
DMS still be attractive to data center
engineers when we have long a little bit
lower performance this is what we need
to think about and to decide the whole
system carefully so thank you very much
and if you are interested in Python code
it will be open source on this github
link so please check out what website
and also our report if you are
interested I'm happy to take the
question now and thank you okay so we
have time for two or three questions
please Simpson University of Washington
great work I'm so excited to see some
attention to our DMA at a security
conference I know that in some of the
papers in the systems and networks
community that they've said oh well if
you know one client that has access to
these servers are key in the data center
gets compromised then game over we don't
have to care about security do you have
a so to execute your side-channel tax
your attacker needs to have our key have
access do you have good answer for the
folks in the other communities who are
saying oh if they get it at all then we
don't have to care so I'm asking about
should we use one out key for all the
applications for all their users or
separated there are cases between
different users I guess another way of
phrasing might be how did your attacker
get an R key to be able to
do the one-sided reads and writes to the
server so in Alfred model because
attacker is another client so it has all
the information that regular clients
have so if the if regular clients can
access some page it has the are key
therefore the attacker has the are key
as other time so it's no extra privilege
for the attackers did I answer your
question i Joe from UC Irvine very
interesting work so I want to ask about
the other like scenarios for our DMA I
recently heard that you can do the RTM
way between GPUs so have you look into
that perspective or do you see any
security issues there so using our DME 2
SS GPU I remember it's called GPU direct
so it's under filter mvvm lnx so we
didn't look into the GPU direct so far
but we said once these issues would
happen not only in regular DMA it may be
also happen a other place because the PI
design you need to catch the metadata
inside you are dominick to handle all
the things but we haven't looked into
the GPU direct it will be interesting to
look at it
thank you ok the last one I had one at
one other question which is so for when
the metadata gets off the SRAM of the
NIC and it's slower
I assume the application developers try
to avoid that since they're wanting to
make the RDMA as fast as possible do you
have any intuitions for in practice how
often the important metadata gets pushed
off the SRAM so I didn't have the number
here but from one paper which is called
form that's as I said through more
memory it's mentioned they have a number
to show how what is the frequency of
metadata will be move out from evicted
from the our DMA link so even in a
regular SS not the attackers once the
memory region sizes of
larger than 16 megabyte and with a
random SS within the 60 megabyte we can
see the metadata is evicted so some
numbers can be seen in that paper also
in the light paper in SOS P okay I think
it's time for us to enjoy the coffee
break so please thank all the speakers
for the great doors