okay so first of all hello everyone I'm
Raz from Israel I'm security is such a
disabled Labs of Bangor University
and today we'll present a little silver
it's an optical sound recovery from
light with reflecting objects
and I'm really thankful for for being
here and I'm excited so let's begin
and so first of all this talk is based
on a research that was covered by Dr
Benassi professors
and myself
agenda for today
and first of all we start with some
introduction and we'll review we will
review the related look for this paper
then we will see the third model of the
attack and we'll discuss some
interesting experiments we did during
the analysis
then we will discuss on how to recover
speech from Optical measurements and we
will hear some nice speech recoveries
and we will review the takeaways of the
talk hopefully we'll have some time for
Q a
so first of all what do you know
regarding the picture presented on the
right side of the slide
and there are some few interesting facts
about it the first fact is that as you
can see it's a picture of an eagle
but the more interesting fact is that it
was given as a gift to the United States
ambassador to the Soviet Union from
Soviet Union in 1945.
but if you really if you review it
internally it wasn't a relative it was
actually the first last New Years
yeah so it seems there is some delay we
see your slides in time uh what is that
one
that monitor is the
what's like do you see now I see it yes
yeah how can we fix this
is it because we see this live correctly
which means the signal is arriving in
this room correctly
exactly so
what
which side is it now
yeah I think another way since we are
actually
yeah I think we're on time exactly so go
ahead maybe click on the next light and
then wait five seconds
in the meantime I try to find someone
for
Okay so
um so as I said if you review it
internally it was actually the first
covered listening device that utilized
passive techniques it contains a passive
device which is considered the first
predecessor of firefighter technology
which was used for review topping
you may know this device by the name the
great deal plug or the thing
now I want to ask you another question
take a look at the picture on the right
side of the screen
you can see there are a few unaltered
reflective lightweight objects which we
call the Little Seal box
do you think that attackers can use
these objects as Optical implants for
the purpose of speeches dropping
so the short answer for this question
question is yes and I'm going to show
you how to work this talk
let's discuss some related eavesdropping
research resources researchers
so first of all in recent news the
scientific Community has offered various
ways to recover sound and these ways are
divided into two main categories
and the internal methods and the
external methods
internal methods or methods that obtain
data data relying on a device located in
a proximity to the victim
for example it was demonstrated how data
obtained from a motion sensor or a
magnetic data obtained from a hard drive
can can be used to recover speech
on the other hand the other category is
external methods which are methods that
rely on data obtained by a device that
is not located in near the victim the
primary methods under this topic is the
laser microphone which will utilize the
laser transceiver to recover speech
visual microphone which uses a high
frequency video camera to analyze the
vibration of objects and recover speech
from them for example a bag of chips
then the last method in this category is
landform which utilize a photodel to
recover speech by analyzing light bulbs
of vibration
in general if I have to summarize this
section of related work each of the
methods I presented to you today is
limited by at least one of the following
limitations
first it might rely on a compromised
device it might be active which makes it
much easier to detect it it can be
applied in real time or it relies on an
uncommon object in the victim's room
now let's discuss the 10 model of the
little silver attack
first of all we assume that the victim
makes a call or attend the meeting from
an office or a room that contains a
little silver in the form of a light
which any object
second we will assume that the sound
from the victim's conversation create
fluctuations on the surfaces of the
light reflective object for example a
nice coffee can or a desk element that
is placed on the desk
in addition we will assume that the evil
detects the photo diode and the light to
reach any object via telescope
the optical signal from the photodiode
is sampled via an analog to digital
converter and after that another view to
recover the acoustic signal
foreign perspective this technology is
external it's passive and it can be
applied in real time it is also based on
really common object that can be used in
the room
now let's analyze the physical
phenomenon
so the first experiment we did is to
measure the vibration of reflective
objects we took a 50 grams anyway and
connected it to a gyroscope then we play
the frequency scan from a speaker that
was placed 10 centimeters from the
weight
at 75 decibels which is important
there is also actually satisfying we can
see in the right in the figure in the
right of in right side of the slide that
the weight vibrates based on the nearby
sound as can be said and that's what we
expected and we go on to the next
experiment
next we wanted to analyze the light
reflected from the work we use the
telescope directed on the weight and we
monitor the photodiode on it then we
sample the photodiode with an ADC and
play the frequency scan in the same
settings as in the previous experiments
we obtain the optical signals from the
photo there three times the first time
was when the light in the room were on
and the way it was uh regularly the
second time was when the weight was
covered with with black tape so it
wouldn't deflect light and the third
time was when the light in the room were
off
there is also there is also as expected
we saw that
um after calculating this enough on the
optical measurements we saw that nearby
sounds can be recovered only when lights
are on and object reflects light which
means the risk of spiritual cover is
only affected the risk of spirit
recovery is only exists unexisted when
the lights reflects from the object
now let's discuss on how to isolate the
sound recovery from the optical signal
so first after we get the whole signal
we follow the we'll follow the next
steps
we will first filter the side effects
that are in the result of sound and for
example side effect that can be out of
the electricity harmonics of 100 Hertz
to filter them we use one we use that we
use the bands to fill those
then we want to enhance the speech by
normalizing the virus of the signal to
the minus one to one range
then we use Vector subtraction algorithm
to denoise the signal
and then after that we'll use an
equalizer to amplify the response of
weak frequencies
now we will evaluate an attacking
comparison to other attacks in this
field
the first attack we evaluated is is a
experimentation sorry the first attack
we evaluated in front of it is visual
microphone
and so we place the speakers five
centimeters from the reflective object
and the volume of the speakers was 95 DB
which is very loud
like in the visual microphone attack we
then played an eftopping equipment 2.5
meters from the object and we play the
same sentences from the teammate
repository that were used by visual
microphone
here we can see the spectrograms that we
obtained from the original sentence and
the optical measurements of values
object that we used in this experiment
you can obviously see that this
spectacle of the original audio is at
much higher quality
but however I want to prove to you that
the signals that we covered are still in
good quality and you will be able to
understand the content out of visual
recovered sound
so let's hear it I hope you will be able
to tell me if you can't
she had your dark suit greasy wash water
all year
[Music]
do you feel anything
okay so next we have added Attack under
the same experiment that set up as long
form
I am just a reminder landform was
presented in 20202 in use next by our
group we demonstrated speech recovery by
analyzing the vibrations of a hanging
light bulb using a photodiode
the experimental setup was duplicated as
follows as follows we placed an eftop
Dave stopping equipment at various
distances
first 15 meters then 25 meters and then
35 meters
the speakers were at a distance of 25
centimeters on the reflective object and
we'll play a statement made by phone
president Donald Trump we are the
speakers at the volume of a virtual
meeting which is 75 decibels
obtained out of the optical signal
recovered from the Rubik's Cube from
various distances you can see that the
quality goes down as the distance goes
up as expected
hopefully you can hear them again
so as you can hear the quality went down
as long as the distance went up but it
was still convenient to heal
now let's review the takeaways from this
talk
so the primary takeaway of this talk is
that as you can understand many shiny
lightweight objects can serve as Optical
implants and might be exploited by each
robot will cover some
most of the cases those those are
completely innocent subjects and in some
cases for example like Financial
implants those objects should increase
policy but can be exploited as the
complete opposite as we showed in a
research
the second takeaway is that
interestingly the area of speech is
looping as advanced as advanced
significantly in the last nine years
ever since visual microphone shows how
to use the open speech using high
frequency camera and other attacks since
then like lamp phone Lido phone the
globe attack and today the little silver
we expect that more Optical image
dropping methods and the three line
Optical sensors will be demonstrated in
the next few years
and for the last takeaway there are
three interesting facts that I'd like to
take into account so the first fact is
that photo diodes are integrated into
most smartphones nowadays and second is
that smartphone manufacturers continue
to increase the smartphone sampling rate
the third fact is that obtaining
obtaining the data from those and so
does not require users permission
currently
all of those facts alone aren't very
important or dangerous but combining
them together we might see any an
implementation of a new attack or
malware or compromised applications for
Speed Shift stopping via smartphone's
lights and so in the near future
and thank you very much for attending
attending the talk I was I really
appreciate it and now if you have any
questions I will again sign them
any questions to ask yes
so I saw that you try different surfaces
did you manage to understand the changes
that you have with curved surfaces that
compare to straight like flat surfaces
when you are reflecting the signal
and so yes light reflected from
um flat surfaces was more
centered into the photodiode like the
photo that could receive the signal
better from
um flat surfaces because it was more
centered and less um distributed
and as you can see in the research it
was it was much much easier to recover
sound from the Rubik's Cube for example
because it was very flat then from other
objects and that's why also the nation
Venetian blinds are very effective
because it's very flat
hello thanks for the talk I really enjoy
your uh final Insight of uh the
probability of turning the light sensor
on smartphones into like during this
attack although I have one question that
is um like the camera like the common
camera stuff has like the uh pinhole
sort of thing but the light sensor
itself does not have focus and it's like
getting light from all the directions
um so that means your significant noise
ratio is going to be super low even if
you have a very high sampling rate so uh
during Invasion any sort of techniques
that could kind of address that problem
thank you
um that's a good question
and currently as you said the
the sensor is not too good
in the future when it will be good I'm
not sure if if that I could still be
able if you still be able to take this
in this direction
because of the distributed
um area
um
I think the if the sensor does Focus for
example on the light bulb always on the
desk and focused on the light reflected
from
you don't need to focus on the surface
of job deck of the object if you if
there's that sort of just
and receive the light reflected from the
objects on the desk it can still work
but it's an interesting question and I
guess that we will be able to know it
only when we have the rights and so then
smartphones
okay
how low can the amplitude of the sound
be before this uh doesn't work
how long
also we this is 95 DB right that's
pretty loud
that's very low that's the sound of
Screams and that's what we used for
comparison with the visual microphone
that's the experiment that has updated
so we've replicated it but we also put
it in
in the in the sound of a virtual meeting
of 75 degrees in the experiment in
comparison to
um landform
so 75 DBS is very similar to Virtual
conversation maybe a little higher but
we showed it from 75 degrees Yeah I
believe that
with maybe with better better setup and
the equipment you might even do it from
less than that but 70 75 degrees is
there is what we proved
oh and you do we have any further
questions