good morning
isn't getting out thanks for getting up for never spoken
on 9 a.m.
for I did not expect this is Wrexham so
is we're going to cover the day um time permitting
given the one hour speaking thirty-minute question format might go
over to get to talk sim 21
making am so it's a part 1 will will be security team 2.0 which is the
content as described in in the AM
in the program and what we're doing here is we're trying and you talk where we
sort of
look at some other things we've done very differently to traditional security
team
when we set up the crime security team I'm eating some things we've done
differently worked out quite well
so I'd like to tell you about them on the course that there is that by
thanks for sharing the things that we've done nah methodology on
you might I want to copy some other things with them when you find yourself
sitting here and security teams in the future
and the second thing is I'm whenever
with a little work done we find that one of the things people most excited about
am another miss questions about his is a vessel wall programs that's why we
and get monetary rewards to researchers he or
things to us that we think a significant and time permitting
will will dive into the history there and some of the statistics
other in statistics relating to the sort of man
with pay with PayPal and so on and also hope to you
devotes a time questions I'm so some brief introductions
the mine I'm Chris Evans informal
try Nancy and im phone tight love troublemakers and he said
um I got my business cards would ever had remote cabin with me and
says trouble-maker something they've done
tended to what's important things going to be talking about
just to give you an idea of where I'm coming from so in 2009 I found it
he the former Prime security thing um
I wanna for sins we didn't wanna the best things we did in retrospect was
relaunched
war program to try I'm get more community involvement
I'm that one so of crime that am
Google wide in the same year we launched a web reward program
that was kind of a big deal with all the time because
inviting searches to serve download your your clients I product in research is
very different mean by
my searches to um researcher life services is not you can download Gmail
insurance or local
machine and research ship that several
how that went I'm following on
later we launched a competition for Michael pentium with larger prizes but
with a
with more difficult challenges will that little bit um
others may consoles work in the past and when I'm not I'm not too busy to injury
if I
when time permits loading a bit of securities his history of the crime
security team
so am I think it all started back in
probably 2007 was the first significant event in life
I'm security team and that was actually an acquisition
I'm people acquire a company called green border on this was a
the company that hedonism some books and technology
and that actually the time worked to some books Internet Explorer browser
and the name report comes in the fact that when you would run
Internet Explorer running inside disagree with technology you actually
see the Internet Explorer window
with the green border I indicating that this some actual samples again on their
am of course a %uh plans with you crime went public at this time but team
expects
obviously this is an acquisition sort of forming a the basis of the crime
security model
which was from some boxing so
do himself on so its public launch in 2008
based course in the green for some books technology on
right out of the gate was wearing in beta we tangled with a few security
issues maybe that one of the motivations to
formalizing crime security team the beginning we've had this role call the
security sheriff
and we still have it Anna number for
in before st. formalize the crime security team
the engineers from the green border position I'll see for a strong engineers
and
know about security so they sort of handle income security issues
and in May 2009 I which they've been
inside google to go in fun prints greeting
so I was very excited I'm and I'll see you just want me is difficult to
get everything you want on silent to the task hiring and it wasn't until October
I'm that I managed to get a first higher and
am as many things from 10 just just really hot hard to hide it security
people in the United months have elapsed
between me starting in getting my first time next
underline we we are in a situation where it's hard to find good security people
and then it was another say six months also forgot mass get a second higher
being the team to three am by name inferno
you since then has probably from a browser bugs anyone on the planet
am South bit higher expect I'm and then after those first he has them
unknown reasons thing sped up and realtor
too great a team quite significantly
so what what we think we we've done well in from skating and them
this is the sales pitch those cases quickly but it's sort of a background
for
name think they've done some things well and then we're going to what we did
cause these things so I'm for the beginning we
am willing to tackle from about a bit software which we thought was for the
biggest problem that was
is causing the damage out there on the internet so was a baked
I know automatic update process into the browser so that you don't have to worry
about
up to date on up on I mention the green border positions so
we'll see with a we think the first puzzle with them
with the samples for the isolated the file system from and from our
on we've since you since looked at the
threats out there and there's a lot of if he is God's love
I'm the bugs around the area Adobe Flash so
across multiple years meet with Pope to that plugins either
the fulcrum samples on
we also think ahead of the curve with the recent Sam
Java issues you may have read about has quite a a little press earlier this year
about him
problems in scares in Java plugin I'm a couple of years
before this we actually were sort of predicting that this might happen and we
were
the plans and defense in that area such as user interaction forum
for launching some in the morning just plugins
we've demonstrated the ability to to a very fast turnaround
mmm because if I mention they were all programs free feel ready
want things I'm most proud of as telling mentioned in the introduction -ism
some the technologies we baked into the browser actually
could have a significant in which is where allegedly
some something Iran that compromised
it in a toast if authority I'm is definite
special originate authorities compromising is alleged that
that some given Iran it done this now of the
has love I'm I love safety consequences potentially for people
in in that area of the world
and we maintain /url Wareham for to be a modest but we
now says one /url on our on a website where we have some unabashed modesty and
sorta
list things we've done keep up to date
okay so into part 1 so
I'm a hypothesis is that some other successes
which we've just nervous list of to me after heated
just some things we've done in setting up the security team that
very different out additional security team might be set up
so let's look at these differences so the way we're going to look at each
difference
-ism festival going to look at how
a more traditional company or perhaps most companies might have the security
team set up
and so more traditional companies donated by this icon agency
I'm have a other seeking titanate more
same a corporate environment Anna
and the first difference going to talk about is fix it yourself
so i think is an object or dirty syringes but
this one with we found is a Withings a particularly big difference to me think
the benefits it is bordered
up on I want the bigger benefits
so by fix it was by fixed itself
what we mean well in this case let's look at what it means is that if not do
things to fix itself
same in what additional security team
on my final the security team operates quite
missus time in a consultancy roles maybe
so looking for issues doing all this Alan
is a founding might just sort of write a report and more I'm
use the term throw it over the wall that's where you sorta found some issues
and shove for David FM team say
a the world's on fire you are I'm the sort out these issues
so that's thats yeah security governing consult your own
yes
can get hurt if the back okay
I your I can france pick up we can meet Mike maybe
I am
so still on for national security team on
you may find that the Thai people employed on a more traditional security
team
maybe the sort of job titles awaited more towards
analyst or researcher I'm as opposed to
more engineering like titles so
but see it as a different way to do things
so this icon he represents so that the more progressive way of doing things
and the way we've done things in the ukraine skating am sincerely
loss thus even tie weaver don't have collated gotta
t-shirt here on the who claim they gonna and a all pretty happy as well as you
can see
say something
we do being often see say the members the crime security team
often actually write code and
commit K representing the fixes for security bugs and get ourselves
and that's that's going to be different so we we might even find about ourselves
and then is that a friend over the wall
say well you know but only just fixity
and of course
more things we fix you find something familiar with the case say
as well as just fixing individual issues you might find ourselves motivated to
to jump in and actually improve areas whether
have been I'm statistically enormous amount of bugs might say well it's just
going to help the rights there
making more robust and ami
knowing that his we do WEA you can empower to sort of plan not just
security fixes but serves
security hardening measures so in effect of this house
is that your service is a part of the product team I'm
and organization that we are actually if you look at our organization which are
actually part of the crime organization not some
not some dedicated security organization most people with hired
the weapon it is you've hyped and anti-people the
engineers a passionate about security
on as opposed to perhaps people to conserve security people
you now if engineering and that sort of helps with the mentality where
when you find something that's it's broken and needs fixing
consider throwing it over the wall with just which dive in and fix ourselves
because we're we like coating when motivated
to dive in and change the code base so putting all this together we really
found
some significant and positive feedback need to develop
yes a the more we get our hands in the code and fix things
more we learn about the case the more we know about the case more likely be our
last
lost on changing things make a disorder use a sick
security powers native do believe kaelin on the site and find additional issues
for fixing one issue
that's one feedback loop that develops another the public that develops
we find is um is
a deep respect from the security people and
me product engineers say if you have never wanted the
they's to the security people in the previous companies maybe they've worked
at
who are suffering problems on them and his buddies
emergency put down my speech you working on a fix is now
and said now it's like a fun about I'm but we
you know we think this patch might fix it would you mind would you mind helping
us with you
and things like things like this and we find that
suddenly went on these hated security people there is causing problems with
these people
engineers his peers at the coming to them with solutions and
and we think down that that
encourages other respect between the two teams and
this respect we found it's actually bankable say the more you do the more
you think someone else's bucks for them
the more so the Moorebank you build up and then when something really naughty
comes in that's a
maybe more for design Aravind just about him and was struggling to fix it
ourselves we can
over to the engine and say hey I'm would you mind helping us with this and of
course since we've
help them so much that have their more than happy to help out
so they take away from this is that the closer you bet your security people
with the product team we find that within the KC
case you get those two groups just a more benefit
and this applies some within for all there is a sphere
in ranging from the initial response to this for proactive work
and so on so the next if
say this on school is all of ok and this is a
this is a mentality we've will try to develop I'm
and say the
the corporate guy he's a uniting some crazy no its not all
all of what he talking about am
what that really means is that as a as a traditional security team yuri
disinclined dispatching disincentive ice tackle
problems outside the product immediately a an to say say say it was good teams
multiple product tax
as we did for a little incentive fee to go in and fix things in in product white
because thats
units know how your it's not I agree it performance review time it's
not how are your manages want you to be doing
save is there a different things
turns out there is so of us here a
is this we've taken a step back and try to apply
there's a company-wide use a focused
culture Salem we sorta look at that and security remote watch the
coming back with is that we think that if any problem we see out there
is a security problem affecting of harming line uses
will just say well you know what is awful and we're gonna
gonna solve this or at least tackle it
for have problems of course week and not yet sold in security
so that mean in practice
it's different from what we might might do if you want thinking it was awful
well this means that
if we depend on any third-party code
in that's not so the valve falls our code hours
our bugs gonna go and fix as anyway regards a few
in a regardless of politics sometimes
analyzes the party code of course
a lot of web browsers these days special events or things like crime is a Firefox
a built-in pop-up love open source code am
I'm since we know collapse directly involve product in bed of course we
consider that
yeah I responsibility if there are bugs that but I am
increasingly we're looking at sort of know my pencils proprietary
non vendor non-google vindicated that has an effect on the security of the web
as a whole
and with considering
well the best since before uses is is help out getting stuff fixed
I'm there are some historical
is expensive sessions in area browsing to do with
I'm for example love security 1.0 X perhaps on the optimal
trying to although all the browsers behave the same way in this regard so
we're trying to
we're trying to tackle some some history there in some legacy that
and Sam you may seem agents paper
earlier conference where she's looking at that some the problems that you act
and how we can
with the web forward their including buddy web sites
so to give one example
that's the lot across site scripting out there my have me this far along here as
we had hoped City
that is such a big problem that we we so say well maybe
if his love cross-site scripting out that maybe it's the fault of the
platform
maybe the browser's of a different platform facilities in a bit less
besides getting out there
so we see that as a problem take said we
we just try to focus on what the right thing to do is to improve the security
of the web
we try to ignore anti-politics your concerns here
and to give some concrete examples
what what's in scope a first look at say to
main categories come to mind one is plugins so we don't
of the major plugins you'll find on the web things like um
flash PDF plugins Java we don't really
own the code base for days but we still try to
where we can help out secure those and increasing in
in modern times colonels so along most of these days you finds
is getting some books and said the attacker
has a sort of break your software and then they find themselves in the sandbox
and then they have to break out over someone's our the most popular and
and in fact dangerous ways to make up for some boxes to attack the underlying
operating system
so actually you with a lot of research to try and improve the quality of both
the windows
panel and the and the Linux kernel by today's
what product on Kb own safe I use is very important that these
those two camels be very solid against against security attacks a
if they suck bugs me just say well you know considered of als
so putting and some sources to try and fix that
okay um this one's more nebulous difference
say I we're gonna look at it was an examples but I go the extra mile
is this the %um to different say I i've just
season of Lost phase where they are but I'm these a
the point of me quoting me is that
you'll sees things like this everywhere every company maverick source project
so if you go and look up like the false fears random Project X
try and find that security force the page it will be fine
the bait him a phrase like we take security very seriously
am have as anyone here
reported about it all to to a vendor tool anyone
10 securities in extent Bhatia maybe
you go back north may deem a fairly quickly and and maybe verbatim it
included a phrase such as
we take security very seriously after thank you for your report so
its or maybe I'll
securities on our top priorities is a is another one I'm if you
go on Google for these for these these quotes using the quotes to get the exact
match your DC
row after row of Cooper corporate website the quote these things
ansonia the I'm getting too is this this is what we Michael talk the talk
%ah this is walk the walk so
I mean I mean you have to say these things I mean what you gonna say we know
securities name maybe eight out of 10 on a Friday list or
or you know is we should have designed this and then
security on later and I've course is gonna because you're gonna say you takes
very seriously
say to go the extra mile you've got even look at actions
you know where the what he'd done that that that basic cause you to
take some invoked in pain upon yourself in order to it improve security
so that's a plus they will will be something even if it's hard even if his
painful
or at least two will try it. them
so had a
glitch that so back to the old school working things
so say and local government you might
you might find generally known security concerns and outweigh security concerns
what I mean by that is say on secure security engineers found this fantastic
new security defense that
really is quite powerful but it Anna maybe it's gonna
cost you attempt some forms hit on some hotly contested benchmark I think you
might guess
you know when it comes to the product review stage he's going to win that you
often find a
few she finds it is my legacies
is impeding progress and that's not the case where well perhaps I'm not going
the extra mile
where up the legacy is usually involves Holdco
painful to change I'm to go the extra mile you've really got us a
dive in and and be loved
lot of work for the benefit you're going from security say
will what might look like if he
have a culture going the extra mile se recovered some say
tackle others problems for them so
I think I think we do go the extra mile when we on the sort of look at
what compromises going on on web large
and we see that from a quite different why is not necessary owned by us is
causing
unit offended if he's a compromise so what we've ended as we can
I'm investment source in thing some security auditing on
why and send these this box along one thing we found it's kind of painful
but we do it because meetings right thing to do -ism is also the pasta what
you found and fixed to security bug
what he do next um
you sort of have the easy option in the hard option where the easy option
is just sort of that fix get picked up in the product
in a couple reese's time when that they can actually get has picked up next
major release
we try doing this is painful I'm this risk to doing it
week we tryin within a very short time frame
grab that fix and back for it
said use a get-tough fixed to the stable version of the product as soon as
possible
and a working stove goal is is a month which is quite aggressive
but that's a month turnaround from above comes in and user
has that that fixed in the stable version of that
for these an important this applies to internal fines as well
so most is about to be fine now we find internally because we've
release you scaled up in and have thousands of CPU causes
doing automated testing and fussing so fun little bugs internally
and then we're faced with you know who do these bugs
a meat-eating exactly the same as if we've received the report externally
saving their internal we we declare we found these buttons that are
you next week on the carpet and we also back for the city
to the extent possible and that's been a painful them you know we get counted out
a lot by the res managers for
an emerging lots of things all the time and that they know
that everything you change the risk of breakage or aggression or boxes if the
balance that we
we think we've got ourselves a seat at the table to discuss
units risk versus reward trade-offs am spending money
when something the extra mile like them to this day I don't have any particular
budget or headcount
limit I'm aware of this that's that's quite nice feeling
will spend money on hardware I mentioned we we do a fucing
I think we have we have we have an in the crime team have about 2,000 CPU
cores at our disposal
the white a skating bouquets and you do
to work for us they've run up to like nine thousand CPU cores to be
to do testing for security at times I think that
and finally do not shy away from changes in behavior
so no on re like to change in behavior
because if you change some small the Nativity King something
pictures like a father someone somewhere is probably going to be upset
I'm am sometimes changing behavior is the right thing to do
so we we should've at least
this about we've come to is that we have to missions it to them
to massive hats
explain myself I went heavily we saw the company's understanding where where
liberty to try things as long as we think so
overstep the line 25 am and you know that you have this liberty
if on occasion he love it if I'm break something a bit too significant and you
know that's happened in the past me
say sorry and scale things back but a K
so just a brief tension here
on kinda for reflection perhaps on
on on our industry am
and I'll say once one season understand this comic-con where it's coming from
and that is actually kinda true it
does change your mentality on on the going the extra mile we found say what
what this is a comic
I'm that was published as part of a series
comics by some people on some morse we say attack side of the industry
and and is coming to the palestinians what it's saying is that name
site mocking tone it saying to you this as some random searches
you know who I found a bug I'm I'm so pleased the basics paint either
you with statistical probability
you when not the first to know about this bug
and yes but it's not was in public so therefore
you know this this but was probably out there known by someone he
using it for nefarious purpose I wants
once the reality of this SM we found that change the mentality forget the
extra mile
say whenever we find about what I i with we've got the thing about this could be
because someone else knows about this but
so that this it strives of Los
um time for a civil war story if you like so
we find that so far too
it's hard to define go the extra mile in SE so full that stories but we think you
have gone the extra mile
and that's what we quickly if you hear
so I do apologize for the sudden an abrupt
changes slight style Ibiza mixing and matching tax here a bit
so is a couple is going out to a company called
view and perform but we time a non-disclosure on us
I mean it I think it's worth talking about full disclosure versus responsible
disclosure anymore
I think that what you have to be worried about his non-disclosure so that's where
back to that comic that's where someone's found a bug and they're not
telling
anyone about it um because in it what he's up active
make love money did which turn implies that that some
something not good going on without bucks a non-disclosure
Anna is what we are about and in fact if someone full discloses on its with the
light it is something we can then immediately fix
which is perhaps a turnaround from more traditional thinking
anyways this non-disclosure was so that was a partial non-disclosure space
me the formalism YouTube video
I'm showing an alleged compromise if the Chrome browser
but i've seen 0 details that would
domestic and fix up but because that's not the business model if this
particular company
a more normal security team do well you know this claim is just a video of a cow
popping up pretty much just to keep young verifiable claims the
he just dnt no rain go away he's come to us for me
when you want to share openly with this security research
you know hand of the PR team to serve its back down the report is a
trying to make a story out a number of a plane I would we d
so we thought well as take this seriously most what we do about 30
minutes
this no details have any useful so to say I'm
this is a freeze-frame from the actual reach you
non-disclosure video am
showing if you made a business this text inside the crime
when themselves in your browser is being own not long of
and something's going on in the background not long after this in the
video thing account data pops up
which is unity miss industry standard for demonstrating compromise to browse a
am say what we did is we salute a frame by frame
view at this video to see if there's any information leaked
you know nor officers in a crowded around a machine for a day
trying to work out if you get anything out this video and this particular frame
is quite instructive we found saying
racism observed in a piece and not how you can see it to
um sit behind the crime when does the Windows Task Manager window
and the reason that's been left office to serve demonstrates
that they freak out a crime samples I'm
as it happens you can see that the crime icon there and just above the crime I
call you can see a couple of pixels have grave
the sort of the to serve program entry fee like above
I'm icon it as well as a significant invalid just that it really is to raise
pixels and this is fifteen villages or it
the only way you'd see that icon is if
even launched if you've launched our day be fast break a process
so immediate telling us why we think this might be
very significant information me we thank you my point about
in on incrimination in flash so
is anything else we we can ascertain yes a
integrity level is shown in the task manager
fortunately wanna 120 members was was at the time
an expert and in this some
in this process via piece of lani happen to know that I am
there's a bug in it way mislabeled something in his column
so he was able to deflect sorta what we what would have
sends out for Spartan tell us yes something is mislabeled and a
songs with what that we're able to looked at their four-game processes that
and the way they were sort of the relationship between them again
indicated a plugin
I'm Flash plugin more am
30 minutes is that them for some reason the
the memory usage of the if the process is in view which is interesting in it
it's on spout half gigabyte which is huge that tells us that there's some
former spring exploit going on since another leak
I'm and also just this the position of the scroll bar on the crime when they
I'm shows at this and some on the show and content and
that's causing a scrollbar to appear in and that's a was vindictive what you
might expect if you
if you've got a.m.
and an invisible plugin rectangle
that's so all this case conference that well
yeah it's a problem not the not with the North crime with flash
bot slinking back already principles you know we're gonna call it off ok because
if we can help we will even that's not necessary arcade
to can fix so have a simply go to this eve
as and when I don't have have have it civil
on a far stronger security engineer so we we borrowed him
to you to go and do some work on on fast security say on
and we will send him a little bit of resorts to to help help him out
I say will he use about 2,000 CPU cores to you
to you I'm performer fighting techniques he
hit by a little earlier which is where the
you sort i've what time it is
is in fussing with excellent code coverage than to do that what you can do
is you can download a white
sample input files and sort of crunch them down into a corpus of files
the DOM that is minimal setup files but represents a
maximum code coverage you can achieve I'm
and fortune a.m. button Google you can access love
large-scale not just these two thousand CPU cores but also you know
a copy of the Internet security
he it's a copy of the Internet and from
from the internet he extracted on
and many many many terabytes of flash files
to perform this analysis and cool lunch this huge set down
on I think the important I
twenty thousand files I think may have I'm of us the inputs and I must be the
output sets I think maybe they're like millions and millions of ash Barty
Crouch down into this
quite dangerous at a twenty thousand files that represents forget code
coverage
was being a good basis for fussing am
anyway he sort of then use the same 2000 CPU cause
to crank out about a about a hundred uni bugs um
in in flax and UVC work for the day be to get this fixed
am also we thought well as an opportunity to you improve after some
boxes of him some deficiencies that
sousa patch things up best week in the short term
including that we believe the sample to skate used in this exploit because it
was Sam
quite discoverable so is likely to be the same on and
that was used related a confirmation unofficially that it was
on I'm excited work on
what's known as paper fax which is long since now and is now long since a plate
maybe it's been out every year it's it's basically
the active bringing that the Flash plugin site some books if the same
strength as the core people some books
with its on the on more significant security improvements we've
we've done in multi-year lifetime if the team
meso went on I am some
some heartening measures this is one of those incidents
where I mention we have the leeway to assertive
step up against the bounds of our security team can do and and go beyond
it and this is missus clinton's when did
when a bit too far I know enthusiasm to respond and we actually am
we actually regressed functioning regressed the
the Flash plugin crime on in quite amusing way if it's possible to you
knowingly people in an amusingly
what happened -ism we put a limit on something
call called g8 pages to try and a try and explain potential
and I when that when the limit hit which we think thought it would not do
we popped up and what happened as a little dialogue box will pop up
I'm with the extreme a bad idea any which is a bit cryptic but we recognize
it right
and was a need it's not recognizing it when people started blogging this
strange dialogue was appearing
if when they were just trying to um distracted
do something in flash so we met the back of that is them
and then yes that's the that's a bad idea in in hex
am sorry if you if you saw that
and finally um go get it more to this but we invited the community turns
around paid on fashion's well by
by bringing Adobe Flash into under the umbrella for reward system
so if you want to if you find a facebook
on for free send it to us and you may get him monthly award if it
this significant for security
yeah gay so next difference so
I'm security number zero sum game so that some
a statement a different size I suppose the difference is that we genuinely
believe
security is not a zero-sum game which is to say
let's look at the traditional view of security on you know if you think
securities
zero-sum game you think that you can wind security expensive others you know
by
by by doing something and not sharing it and saying any we have this protection
or you might its she security as just a PR game you know well
would you say I was in ER maybe it maybe it'll mail settings mine sent same
and some companies um collaboration
me even in the security space for collaboration he's gone important may
even be
discouraged filtered or even outright jazz band
because I'm is I because of concerns about confidentiality maybe
is there a different way of doing things well yes for say I'm
we do in the teen Kasia aberration openly and freely on security features
and standards can have a few standards working on
just just about final call it the IETF level I think for
for certificate certificate authority related harming
redo like it when we can share experiences and approaches such as this
talk with
with other vendors and even since making source project I'm
we have elective share no code if any wants to become come borrow
welcome to do so we've had several projects are some of us on Boxing Day
and this one um surprises people perhaps the most
on on this area but we we actively support to any employees
want researching on Google
products even if they want to know what time
to an example where this this make sense and someone benefit
mention that we've a little fuzzy on the structure and a lot of CPU resource
we can dedicate that say once you've got
the infrastructure running against Inc browser a like friend our brother
justly fairly trivial to briefly point these two thousand Cebu cause a
security fury at any other browser me sort from time to time do that
and is just because easy to do and because
that results in quite a few bugs against all other browsers themselves and
there's a long service
you know so the whole web get safer so we definitely believe our security is
concerned
the whole the whole arm not a zero-sum game
and he says is that a a rising tide lifts all boats that if we can make the
web safer for everyone across all processing technologies
and people use the web more now that's that's good for us as a company
next different so I removed the middleman
say depending on a
bingham which i'm talking about. Security Response
is done can be done very differently
how we do it so i made an attempt at trying to diagram for more traditional
wasting secure response same to creep on from the internet I like
drew a blank actually so had to draw on myself and my my
a meeting in hex much ink has a pretty bad apologize for the stated this
diagram leaving so if you can
you can read it attempt at a flow chart a how
how secure response often works on
so arson run-through like this process a bit pretending I'm a security
researchers is found about him
I'm according to a company offering something like this
feel free to raise your hand if this sounds familiar to an expensive at any
time that an
anyway so who I found about I'm gonna reported
writing today so I sandbagging an ass and you know
and immediate got an email back this is awesome
well is an automated reply just let or somebody you know says that
you know it says it says we take security very seriously I'm
more fired up now and a day later I get a
question yeah some clarification on how to reproduce the the park perhaps a
an email back and I get an email back at a time when we think we've worked out
after the fuses same
annexing happens internally maybe it's a fairly tricky part in that the
organization he is responsible for church
a security people but I think necessarily in essay expert in all the
main so they they bounces back over to their internal
expert team which is a central resource who then
has more questions so that saddam bounces back through the response
organization to me
I found these questions and then eventually
internally the the bug sent over and filed with me
product team to may or may not have
some questions again they make consults internal security experts on how best to
fix something
particularly if it's not just about for the design level floor
am internally them at a three-way disagreement may kick off on
on the severity of the buck the and other products in the verses the
Securities view this is what I
claimed as a researcher on
and so on and so forth so you know week later
sizes in 10 lines and I sent us an email saying hey what's up
yeah we get this fixed so you know
the responsible for that price support team for a status update and sort of
feel threatened send it back to me and so on and so forth so
a lot of communication going on what I've failed to draw on this diagram is
perhaps a big red line
that was the top of the diagram across so the top couple boxes
and the point having I any get visibility into you
a very small part of the process an order this of internal
discussion about the book I found is hidden hidden from view for me
so Mike why be the problems operating in this order mana
one is just latency so
I'm justices round trips going on between people that I get to see any
get involved with you know they it seems that you know I I won't get
status update as often as I might I'm
you know just everyday thats that this latest against that's it that's a day
that the
to take it sort of the to fix gets kicked out like users will
will get fixes later we think because this them significant potential and I
have personally
for lost in translations it's where
it's where you sort of have this middle man whose is your serve case officer:
if you like a understand security not maybe perhaps as much as as the
is the technical person he's actually working how to fix the thing and
you know having a conversation with a technical person you having a
conversation with is this middleman see
be quite cumbersome sometimes am yes
makes our top 10 discussion sometimes I'm significantly a I'm big on
collaboration and I think
the sort of the past if the process that I needed to you don't see a lot of the
discussions going on regarding the back meanie conserve
chime in and help am some bugs specially designed over bugs
con hearts to come up with a a good thick sometimes that's
the debate secured all saying it doesn't break the world so
sometimes being able as the research to be involved in the discussion how to fix
something
is nice so we think we've removed the middle man how does how does it work
how much something me different so report about against crime
you go to a bug tracker and
public the public the access for me on the internet bug tracking you fight
about
and one of the options is this is a security about will talk to you know I'm
will restrict you from public and May 25 if I S
and then we find that the more interesting bucks reported by the
the morning sting researchers used on the same day
like this a three-way Austin high-energy exciting conversation kicks off at
on as in the bug tracker on between the person who sends buck
between remember if the prime security team
and a perhaps the also the engineer who's really the best to fix this issue
and came back from a variety principles
that the engineer fixing the issue in the crime security team member and a lot
of situations water make that the same person
and a on this blog additional experts in
in the area on the cake base to see seed to keep the conversation going
so does lead to Salem though it's definitely latency
so on occasion we've had about coming and the same day we've landed a fix
and take place that's not the same as is going to fix these are for same day
because that would involve
rushing something without queuing and waiting do that but
but I i some the same day fixes that we do inspire people to
to comment in the bug tracker you know as fast and then we
and you know Nexus smile when we see things like
am I justice can be the bug tracker for fast and in a
find boats bikes with this this is this occurred
and his last but this just popped up here and he and his Republican
commitment 'cause
on this is an example of where the the researcher was a sort of
bounce ideas of -ism we bounced ideas of him
you know on on the best way to fix fixed the bug in question and we came up with
something that people were
I was happy with and then we fixed it um
I think our table turnaround time that was about five days in the sixties is
which
when always a fasting as a a fortune Constanza
recycle time in the states that that I am
this process you can be very late see to the system
and then of course we I'm Gator jonsman a monetary reward
at the end of things
be transparent am
so we have sort of an advanced i guess im being able to those projected that
were fundamentally transform but I think there are some
transparencies that that companies with closed source products could engage in
that today said requiring open source
so in if you're not being transparent
how much you behave well of in my dealings with with vendors I often find
the
it feels like communication with me is minimized to the bad things a book
he said I've thought long and hard about why this could be
and I may be that I just think made maybe the many companies are concerned
anything they may say extending my and getting quoted and
with which it might I'm my and
some you story perhaps I don't know say
perhaps as some sort of fate of pounds going on in the risk of saying too much
versus the risk if
limiting coverage may not saying very much its
not been comes back I'll definitely when you find bugs internally
to the company you you'll keep them secret
you fix in the course but you when you do say USA announced that the bug has
been fixed
and that's pretty relating to the fact that you have %um a setback for the fix
a few announce the fix against
the latest frightening you'll be expected to backport it
some this or the lack of transparency might be considered considered dangerous
for security
I'm and when you do announced something fixed
some this does mean time check on
allow I'm minutes left
say you may find that out when when you I'm it doesn't answer his critics
announce
I'm it in very broad a specific terms and they went
doing us a shadowy full rip me original report the course
fix so
is there if my doing things well yes we can say say we mention that
all of report about to see this file a bug tracker
but I am that's just the start say all of us carry bags attract
you know public contractor and that includes stuff we find internally
and reedy weary really do enjoy making
all security bugs after some suitable period of time for the public's you can
go back and look at the history
any security bug in the history of the product
and read them of course that sometimes we delayed
opening the but the public view because love the code we use shared with
with other products that wasn't take the Tampa Bay
I'm will be saved bug tracker box but I think every if you have a big impact on
crime security
its movies about its really is a conversation
any sort of see aberration going on and you can see discussions about how the
fix should be
should be implemented you see their search for jumping in from time to time
is reported as the bug
heehee native had stabbed at something and helpers in the
senator transparency that lets us do that
and I don't know I'm a people that do this but uniqueness me historical gems
available now but they do say so
for example we have a competition format
comment call premium where we actually for lots of money require people to
actually
ex white boxes that is fine bugs and
course the principal transparencies dictates that the full working expertly
did work at some point in time is just there's an attachment
go back and look at those and
and study them and learn from them we
eyepiece that some people do you go in and studies expertise
right papers on the techniques used in someone so like he's about the
collaboration is encourage that
and the best story I could find work demonstrates why transparency with is a
huge win for us is just some other bug
the numbers are random number B its public you can read it
what actually happens is we have researchers and is above its a very good
book
I'm we fix it and
immediate within a day or two the researcher comes back as a comment on
the back and what he says hi and you fixed it
properly is this serves corn a case that still seems to work
and what that enable us to do that trance and of course transparencies or
18 with him to tell us that I'm what saved us the pain of is it's a just the
pain of sort of
fixing the bug releasing and then coming saying it's not fixed
an outing kinda embarrassing you know anything exit security but probably
asylum
but what he needs to do is get it right get it right the first time
is awesome and of course we with them probably doubled his reward
as well as counts wings comes to bugs modest you twice as much for your help
I'm freebie for you know the story
the reason have this story's some people I think
of mister there's a fear that transparency will lead to the
possibility of attacks against you
like if you if you have all your dirty laundry out there for public view that
the fears that someone come in
count how many bugs you hadn't said oh my goodness at 70 bucks or
or someone saw review about database and saying I was an awful lot
and maybe that stream it maybe there's a chance that will happen but but we think
that the
the benefits open transparent cooperation outweigh the risks of taking
some PR hit and actually yeah
to me we found that thing comes back to me very clear positive in
in this recent incident so we face it some
some blog post I'm related to crime re the
that someone disagreed with and have to account for opinion which is which is
awesome
am have the reserve frame that counter pinions around this claim that they had
tons of critical crime issues that were months old not been fixed
obsessive and inflammatory story might get some traction so well look into this
only we found that actually is this is not true there were some bugs
been reported that they were sort of in our opinion on not security box
say there's a transparent /param to respond by posing this link
which is just a aquarian of about database for the name of the company
that it
made this claim listing all of the things they've reported
enabling anyone wanted independent me you know verify lking buses
that claim in the end the a in the end
sooo over quite more than that impacted the block place making the claim was
taken down
handsome modified to be a little more true say
transparency he was actually a a PR when
and this is the final difference on my game
that's when the ones where we think is is more significant
so to a story in that celebrate community
so on motivation company has been known to see the community
in never reports if this
may attempt to vilify members of the community is a No today
see eye to eye in in a on the severity your time line maybe if some bucks
as doesn't seem excited sifter to work with the community
you may try and downplay the significance %uh for such as research
and so on
so as a principle we found that
the morn gauge what we do with the white communities more the more
or something gets a weary time and celebrate community contributions miss
you can say
now we have a Hall of Fame which lists this everyone Sun
got among the few of us know so as on a the as
not quite cross that bob is still send us some useful have a list of people and
we just ran to haiti's
is that list of people in there or sumn and and with we get feedback that that's
helped people and jobs
sometimes that's actually but also with with other companies that's that's quite
appreciated
I'm whenever we fix something that's so important to us we
we declare it now release names I am
just respect supported the transparency principle we saw earlier
also is more server its its eye contact with the crimes gettin is also
less mechanized and and more individualized inhuman I say
when research comes to us and this is bargain its I'm just haven't thought of
anything is really clever you know your your see
myself one of my engine is the first response in the buck masses
well this isn't or sumfin this is really clever have anything to this
and we just fine that's a one on one researcher to research interactions
I'm gets little respect on both sides of course we celebrates
completions financially on on about basis
just because we can sometimes we decided on someone's been so awesome I'll just
issue a surprise monetary water them like sort of a
like a best of the best for the year kinda for what may be
knows only get something sent us this very spectacular
will often do a write up ourselves as a celebration of the quality of the work
some of the pentium entries we've received we've actually written them up
and publish them on a official Google Blog to celebrate the sexes achievement
I'm said this is the next couple of slides are not
not new rewards but just this is just to illustrate that sometimes we just
this last year we just randomly decided to give cover photo reporters an extra
ten thousand dollar top up this way unit working with them
and setting us up for some stuff say a couple slides for presentation I did
last year oh it's just
well just give these to or some people ten thousand dollars each yes as a top
up to the
other wards
so that ends part 1 members can do exactly and check again with Allie
see how much time to outside party won't guarantee
five to 10 minutes K people
infused to go into some more detail in our war program
all and I wanna wants I'm
what I then c2c
so this is the story of people's history with rewards
this is where we when we get good security reports him what we'll do is
we'll
will financially compensate such as the defender state awards
I'm so we'll start with with the statistics that people are you most
excited about which is of course
while how much he I'm to be paid in Table
as the first question you get and a some breaking news or at least it was new on
Monday
with a blog post about this I'm
when our to you million USD across our various war programs
crossed across that bar and interestingly enough I speak down almost
equally a million each
from this or chrome pony im side of things and the Google
website side of things and I suppose I should admit it internally there
as a bit of competition to get a million dollars first
and I was reading for crime and I lost but I week
we crossed a million dollars just just after am
and doesn't bugs as well for that money is back two thousand bags across
everything
slight psyche skewed in favor of more bugs on the website of things
public's because you have a lot more web sites you think and maybe a little
easier to find
certain classes of weapon ability than it is to find memory corruption in a
browser
and as the book basically put out about that
that small
glitch K um every time I talk about or was program
the goal with the changes the talk changes so
at first the call the talks to say hey we've done something a bit different a
little bit crazy was gonna tell you how it went
just share share our story and now
overtime things change dramatically
we now give these talks try and tell people
just engaging the community in his ways is just one hundred percent
fully positivism source emits exceeded our wildest expectations and
and you should launch a vulnerability or program
if you make sense and their couple love
caveats we give this a one is a you must have yet so if you
if you or your company responsible for software which is sort of
local front line in the mail fight say you know installed on
a white percentage of the world's computers or part of critical
infrastructure
you know the world depends on me it sounds dramatic
that but the world does depend on if you're in that position and
on way you can make your Posterous que possa batters to have a role program you
should
but before you do that you wanna have a mature
STL in place like this for war program is its
not a substitute for not bothering with security and in fact we have other
security in order will program probably or be overwhelmed say you need to have
your response organization
so funny chain WellStar using
so the history about bounties I'm
best we can trace it back we think that's jonsman you may have heard of
buying a
Donald Knuth may have kicked his whole thing off in 1981
I'm in a slightly different way but he found a acquits bug in his book
he would send you a check I'm
I know is and when he got a gotta frames check Donald
on their wall you do I i you get a hit in a in a crowd like this
excellent and then of course the first
significant vendor to do this was probably netscape back in 95
when estate became is Ella they carry that on in 2004
and so on and so on and as you can see that this autumn packing density is
these program launches
getting more dense this week it was Martin time and that is more more
is more more to love fact-based
this more more facts and a top this is a guide ACC normal
doing it sensor is slightly some very old
other on small companies you've probably heard of it dived into this game because
it works lol
including most recently Microsoft ISV dip their toe in the water and i think
thats
that's fantastic because they they sort of count on a
on on this summer these goals here they they definitely count on being front
line in the fight against my USA
it done this is awesome some history
I'm I think will skip
skip the sites were alone time but I this is just showing how we overtime
with racks
it up the reward levels and in some way you can see these will programs as an
economy
over time it gets harder to find bugs and
so what you want to do perhaps if you're serious about these programs is
as the incoming reportable trickles down is getting harder he served gives big
not be content with his
in a max payouts it ended up in that we made these people so
see found every time we've done quite a few
quality reward increases another question gotti's
you know what is it what it looks like when you launch a program this is the
crime launch
I'm that he launched and I think Jan 2009 at the beginning of the graph so
in terms have months collapse when a pretty slow do
ramp up for people and I engaging with us
on it wasn't immediately it did happen eventually say
maybe that's a reflection of the fact that research in a browser can be quite
are
need to serve invest a lot of time to learn a lot about a possible for you
necessary
diving in attacking
since on his dick poon Mr
competition format I'm so
to dipped out on the water by by sponsoring
top of the water cram into industry competition in 2011
and iraq has a misunderstanding so I'll just launch are in competition
we've we've try to calibrate things again
looking at this is an economy we try to calibrate insist reward was high enough
that someone was
short enter and got it just right got because men trees
so gentleman by the name of Pinkie Pie one sixty thousand dollars
for the second competition and that is like a
Pentium three I in quite the full price be turned up in and took a forty
thousand dollar when
on and the first only had he turned up for a sixty thousand dollar win is
getting a bit he's at this time say
he said a benchmark one day if we run a penny in competition and he doesn't show
up the collected sixty thousand dollars you know
madame really solid shape because this guy's amazing
am and skip that
say just to humanize the whole pay in competition
is something something mean may say that that's me with with shorter hair
directly shit this is and this is that a tense moment around the desk where
sums turned up and they like I can do this I i've got this
website when you visit it's that well compromise crime in a my sixty thousand
dollars avon's
all eyes on the laptop you may notice as a guy
um so that in the background to the right hand side and has a look at
abject terror on its face and if you can CA
there he's next terrified and the reason for that is
them we actually had in this competition we actually had
the Adobe Flash plugin in scope Anna and this
gentleman is an employee ever date me and
known quite news what's gonna go down yet so this is thought he's three
thinking oh my god the same be a flash but just just
just a and it wasn't so I Z I have the picture to prove it is looking a bit
happier about five minutes later
I'm say we we you know I thought to be about how is it that some successes we
also
something when poking fun at ourselves if we if we fail so
the the penny into sixty thousand dollar win
um involve two pieces have attack against crime like a
difficult piece I'm to
sorta breaking the kind and any though the way this gentleman think my break at
the same box
was just too easy by us and recesses high standards and we went we were
well very happy with it of course we your best foot forward to make some
changes to make correction happen again but in a
I'm we definitely and a double facepalm from performance ever
we're at the same box as he found a way for within the some books to request the
privilege context you just right
this sequence of bytes to this file name: just not have a good
property having any samples am and it was so bad we we just went
demean town with this in SI
you know member Trust I was gonna trust the final aim
to file past and the content of the file it's a great
say long since fixed and the sale went II
went on an internal audit rampage to make those no in other instances of this
happen
but that's what we did the pain in competitions not the value they provide
to be
you really wouldn't have found out about this if we haven't done this you know
60,000 best sixty thousand dollars with
weaves have a spending problem
just shoddy meter stop just beef it's going to the other main
reward program Google Web program more contentious because this is the first
time we know of that a large corporation
is gonna put up security I'm rewards
relative to 'em live production website is there's no other
good way to to do it really say you know you have to talk to you legal people
forty budget people have to get by in from thousands of teams as I've said it
is one product team
I'm the most interesting thing people want a ride is is legal
am lawyers or sumn it turns out that if something's illegal
you can't make it not a legal just by writing a blog post I mean
if the world if that will work like that I think we did
see its we see a lot of trouble in love love pastes
am say you're the sort of see don't lose basically the right to go of the
criminal activity by having a bounty program is this the principal
um also as long as you have
clearly-defined do this do not do that in a wat
you're so dang think lose the ability to chase people civilians they
reading try to damage am
so eagerly actually turns out I i he sort of a headache but it does just fine
am I'm for this launch we formed a formal
Warren like to have our finger readies gonna blog post saying hey
think about single web sites Amaral sat on a warm
and we don't know what's gonna happen so we had a sweepstake on me but getting
first 24 hours and eyes and I and maybe a few
hand I lost I
I lost to someone who sets something like yeah
maybe maybe if f you may be maybe I'm will look at a graph in it
states that so I'm quickly the eligibility
before you launch party you won't be you wanna publicly document
very clearly what is not scope to avoid any misunderstandings
said that I'm
we actually exempt acquisitions for six months
after we acquire them because when you acquire acquisition and
they may not necessarily have the same security posture you do so
yet we think it's reasonable for us to holders of the standard beating them
into shape within six months
challenges were worried about a few things before launch
my going to these cue thank you like but I'm
known as these challenges always came to be
came to be true so describes it we press the buttons
can you tell at what point in the time on this graph we launched the war
program
and I can actually save it as a race this is this is this a wise guy
holders up of being relative but a
you can see this is this the graph above box file duty
external reports week and this %uh villain a small drivel the bugs coming
in the first few weeks represented on the far bottom after the graph and then
bang that's me losing my like that Anna
so without a great night is that it's easier to immediately dive in and do
some
website security assessment is to browse a research
another takeaway is in a if you do this you want your response organization
primed and ready
for you want it wanted overstaffed such that the spec
nasty beyond the current level you're handling
to sort of put some numbers on that spike tacit
ten times biking coming up or strapping on
nor a magnitude-7
it's quite interesting um it then settled down a bit
to a five times late for several months and
we think it's that he stated about three times the load of what historically was
which is in a much more manageable
I'm just a shady
graph illustrating some the economies of the situation
this is of the old now twenty first launch he sorta
had led us to come in and then it's a drops off in I mathematically model this
is probably am
probably a a property maps for exponential decay perhaps I don't know
but it shows you that I am
you want to program and he's fine bucks relative and then over time it gets hot
bucks and that's when you might consider something up for awards
with us what type of what I'm bucks will find a XSS
which is the blue chunk is a percentage dominates number external ports which is
free no surprise to those view
researched all Minister web sites
this is gone cool when we launched the war programs of us he is
initially was that well we really love working with community
so the reward programs a thank you for
for working d %ah but
turns out having a row program will attract new researchers in fact
about 85 percent of people's many bugs a new verses 15
percent for such as that used to work with before the program
University um any 31 6 percent usa-based
the reporters stunning in person go into more detail
we have significant University Justin randomness countries if much we've had
people
come to us from I thought Finland involved because there's just something
about finland's
I don't have in Finland's so
now I can't delegate count the audience but a on the crime program
if every moment turned up the three people from know it turned up from
finland not really known each other just the random
I'm a native Finland and deserve man and tore the house up
you know register made a lot of money and send some more sandbags Finland
I'm missed you have to remain unsold I guess
I'm how charitable the
moving on its say air
the results you get an immediate you get an immediate spike in reports
%uh the signal to noise ratio is better than we thought we thought we were gonna
get some people who only had a profound understanding of security
come to us tryin I'm trained
make money for fan on security issues I'm
you really get a coverage that you just don't get it yet
your internal security net save an internal security number and save
even something big like a hundred how can you compete with you know that
tens of thousands of security researchers around the world both in
terms of Brett coverage
but also in terms of GES down just just sorta
thank you for the corner cases say you're getting some really clever stuff
if you if you want to the wider community
I'm getting some fun but is also this this with someone other
for me serious security but I never seen I'm
so this gentleman claim the issuance transito
transit would give a common specifically trying to transition Portuguese to
Spanish in specifically typing a long sequence apiece
security bug is sounds just like a helicopter
I at I
I was on the rewards panel for the weapon I'm I voted to reward him just
the
being awesome um II was
I was shot down by simpson more some other members of the panel on
apps upset my own like the a like the
like the icon I've been using earlier today yes
I'm not sure we fix that neither in which is a very bad response times came
in
several years ago so summery positives um
say it's not objectify
how how much more secure email users but any good where we found
doing it is some we talk to the people he send a steak is regularly
and we decide have seen every month or we haven't seen for a while saying a is
getting harder and I yes
its very hard now i know i in fact going to
spend my time doing something else it's a hot so you know the
if you don't from these programs it you really can so flush out the
or the qualities bugs that your normal security process my
missed everyone's human we all we all miss security bugs I'm
taking a thinking about the darker side of our industry we from
we've actually a few months back I was verified we've had what we call
collisions
so that's where Whitecap researcher his intent is to make the weapon
the world a safer place ascent is about we fixed it and that is
collided or stepped on the toes a more nefarious intent
you know someone who all sonya is a bug going back to the early a comic
and goodness is a day with it but you know by May 28 in the community
as a has to step up here you know yesterday making SBS
a dent in the dark side of the industry am
nowadays is just a reflection of the fact that since we launched the program
was not ready
had someone drop it Google access are
you know is to have people sale all on the Lexus estimates come with a fix it
I'm now it turns out that yeah few thousand dollars is
more fun and posting a message for
rated as the new people
we've actually hired a bunch of people specifically through introductions made
on a wall programs
I'm out of here security hires is always difficult so
really he's not that aspect cost-effective
there's a paper by dev was presented earlier conference
try to to the objectify some of the cost-effectiveness
which is awesome cuz we display my hands in the air and say it's really cost
effective
and anti-china and the paper comes up with some figures like maybe
maybe a be ten times more cost effective than other ways of getting the same
results
and so on apps
negatives
of launching the program I mean it
specific and slide right but I i we've racked our brains trying
think I've any I'm wrong and it's just all good just so the
hope said from the stage it is not as me rounding up here
I'm others to have jumped into the
into his reward game in positive on same ranging from the payout CISA
you-know-who freely admits he had reservations about the whole concept
but then data the key word being data
has proven him wrong say a that him to launch their program um
and down to the bottom XE security says what I
also believe now that you know this is industry best practice so I think where
heading right before the world by any company who
you know has significant importance of where does not have a rich bounty
program is you know is not following industry best practice
recommendations selling launch
if you do want to launch on these things you would make an invite to lunch with
myself and others
people who think for might wanna start small
if you're a bit bit nervous about the fact that ten times like
so you might want to launched against a subset of your products
am yes we've covered having a response machinery primed and ready
and proactive communication Watson sky
remain respectful we recommend paying for issues in pre-production versions if
you suffer
kind of obvious if you think about it would you rather have a find a bug
before you launch it to the to use as all after he wanted two years
and have gone over too much
any better juncture I'd love to invite any you may have
at Allan Sherman UMBC
how does your security team in Iraq proactively with the design of new
products
on from the very beginning of the design phase yes I have a question said
question was about how we handle design movies
I it differs across company brought I'll talk about my perspective on crime
um what we do is we we have a formal
launch process for new features and security has a seat at that table
say I want something here doing a skit
you feature 12 on the team they talk a security and
at that time immediately look to see if this feature
you know is carries little risk if it does then we'll
assigns and dedicated security people to actually
I'm sitting on the design meetings and be
of Co offers on the design doc say that we have confidence that
at the design level nothing's been done right and as I'm sure wasn't
isn't this in your question if you mess up the design level at the design level
and find out about problems later as you can in place to launch as a really
horrible situation to be in because you make may be faced with launching with
security issues
or also delaying months the project
chris what's actor Sam
am I think you're good neighbor policy is a really good one so you kind of go
above and beyond to solve other people's problems
a but do you get tired of solving problems in Flash
and do you reach out to Adobe to try and
figure out if there's a better way to solve that problem
that's a good question so do we get tired of at helping other people out
I mean I'm yards no cuz we fully enjoy health Wiese
in enjoy a finding in security bugs especially ones that may have a positive
impact not just on
us now uses but the whole web on in terms if I'm
getting getting tired specific projects what we
what we do try and do the define or to make such that we have
computers on the task and they thing ready get tired with humans d
so for example some other work we're doing on on flash
and a lot of our internal I'm k bases we
We have ongoing of causing that basis based will the tech aggressions and also
some tireless the flesh out
the corner cases a bug's I'm
America ID from the receive Utah actor that there was a good lead and
my question so I'm I'm active in fuzz testing and I'm interested to learn a
little bit more
about the was testing the that you apply
this is maybe cluster funds if we could use a little bit more about that and
yes us of the store you mentioned plus the bus which is the
affectionate name we have for our 2000 CPU cores on
so to be effective fuzzing we
we find have to several things one is
want to scale which we we think we have covered although
although there are plans TN bump up the number
computers we haven't and the second is things picking a target say
assertive he should have to know which areas if your product
will benefit from fuzzing if you like and what type of housing
comes as monty the third thing we do some
defense not simple and then get clever so
something for about flashvars Inc say we start a simple
on flash which is better than large corpus a twenty thousand files
mention but I'm mutations may apply to solve
the files to try and finds 3.10 particularly intelligent
in a dose of fitting bits here and there and see if you get crashes
so it is not simple and sort of fuzz heart
until with so I've gotta love the crash is we think we can find
with simple thousand and them with weekend and more creative like
will like 1000 as understand more about the structure of the file for months
they can
for mutations that some surprising to the to the piece of cake
positing the the former but no actual violations a
say the specification that within a cause cause that
the past to stop I'm and more so than
so keep going in that direction we then the gramma based buzz is that
sort of generating thankful on existing test cases and generate from scratch
serve slightly off test miles
along the lines of before the program's expecting but
with me a little bit surprising truant from
anything I have so I'm quite interested in
on than number of security engineers
that are in com team
I'm or more specifically what is the ratio of security engineers
versus the restive the team
yes that's right questions says about coming secured engineers we have sought
a few numbers
I'm fortunately where not quite a secretive is some companies we do we do
you got some numbers in this area said I'm
company-wide number scared people have this -ism
hundreds and hundreds and hundreds sent I don't know is a percentage if
engineers I know that there's a industry recommendations about 1 percent
I know that were significantly above that by choice
I'm in chrome maybe some more specific details so that
the crime security team itself its I accounts about 20 people now
which we think the reason be stuffed team for just just one product
I'm and it's not just 20 S is more than that so we have a
sigh from the twenty secure in his we have we have
experts in SSL and certificate authorities
you know technically on my team
you know the 10 from the network stack team but hundreds and that time day in
day out is
security-related features and pouncing so
maybe the time and the numbers thirty I'm as a percentage
I i think as they know the latest number overall
convention is in sight