Why do we do security?

I had a discussion last week that ended with this question. “Why do we do security”. There wasn’t a great answer to this question. I guess I sort of knew this already, but it seems like something too obvious to not have an answer. Even as I think about it I can’t come up with a simple answer. It’s probably part of the problems you see in infosec.

The purpose of security isn’t just to be “secure”, it’s to manage risk in some meaningful way. In the real world this is usually pretty easy for us to understand. You have physical things, you want to keep them from getting broken, stolen, lost, pick something. It usually makes some sort of sense.

It would be really easy to use banks as my example here, after all they have a lot of something everyone wants, so instead let’s use cattle, that will be more fun. Cows are worth quite a lot of money actually. Anyone who owns cows knows you need to protect them in some way. In some environments you want to keep your cows inside a pen, in others you let them roam free. If they roam free the people living near the cows need to protect themselves actually (barbed wire wasn’t invented to keep cows in, it was used to keep them out). This is something we can understand. Some environments are very low risk, you can let your cattle roam where they want. Some are high risk, so you keep them in a pen. I eagerly await the cow related mails this will produce because of my gross over-simplification of what is actually a very complex and nuanced problem.

So now we have the question about what are you protecting? If you’re a security person, what are you really trying to protect? You can’t protect everything, there’s no point in protecting everything. If you try to protect everything you actually end up protecting nothing. You need to protect the things you have that are not only high value, but also have a high risk of being attacked/stolen. That priceless statue in the pond outside that weighs four tons is high value, but nobody is stealing it.

Maybe this is why it’s hard to get security taken seriously sometimes. If you don’t know what you’re protecting, you can’t explain why you’re important. The result is generally the security guy storming out screaming “you’ll be sorry”. They probably won’t. If we can’t easily identify what our risk is and why we care about it, we can’t possibly justify what we do.

There are a lot of frameworks that can help us understand how we should be protecting our security assets, but they don’t really do a great job of helping identify what those assets really are. I don’t think this a bad thing, I think this is just part of maturing the industry. We all have finite budgets, if we protect things that don’t need protecting we are literally throwing money away. So this begs the question what should we be protecting?

I’m not sure we can easily answer this today. It’s harder than it sounds. We could say we need to protect the things that if were lost tomorrow would prevent the business from functioning. That’s not wrong, but electricity and water fall into that category. If you tried to have an “electricity security program” at most organizations you’ll be looking for a new job at the end of the day. We could say that customer data is the most important asset, which it might be, but what are you protecting it from? Is it enough to have a good backup? Do you need a fail-over data center? Will an IDS help protect the data? Do we want to protect the integrity or is our primary fear exfiltration? Things can get out of hand pretty quickly.

I suspect there may be some value to these questions in the world of accounting. Accountants spend much time determining assets and values. I’ve not yet looked into this, but I think my next project will be starting to understand how assets are dealt with by the business. Everything from determining value, to understanding loss. There is science here already, it would be silly for us to try to invent our own.

Leave your comments on Twitter: @joshbressers

Episode 3 – The Lockpicking Sewing Circle

Josh and Kurt discuss news of the day, banks, 3D printing, and lockpicking.

Show Notes

On Experts

Are you an expert? Do you know an expert? Do you want to be an expert?

This came up for me the other day while having a discussion with a self proclaimed expert. I’m not going to claim I’m an expert at anything, but if you tell me all about how good you are, I’m not going to take it at face value. I’m going to demand some proof. “Trust me” isn’t proof.

There are a rather large number of people who think they are experts, some think they’re experts at everything. Nobody is an expert at everything. People who claim to have done everything should be looked at with great suspicion. Everyone can be an expert at something though.

One of the challenges we always face is trying to figure out who is actually an expert, and who only thinks they are an expert? There are plenty of people who sound very impressive, but if they have to deal with an actual expert, things fall apart pretty quick. They can get you into trouble if you’re expecting expert advice. Especially in areas like security, bad advice can be worse than no advice.

The simple answer is to look at their public contributions. If you have someone who has ZERO public contributions, that’s not an expert in anything. Even if you’re working for a secretive organization, you’re going to leave a footprint somewhere. No footprint means you should seriously question a person’s expertise. Becoming an expert leaves a long crazy trail behind whoever gets there. In the new and exciting world of open source and social media there is no excuse for not being able to to show off your work (unless you don’t have anything to show off of course).

If you think you’re an expert, or you want to be an expert, start doing things in the open. Write code (if you don’t have a github account, go get one). Write blog posts, answer questions, go to meetups. There are so many opportunities it’s not even funny. Just because you think you’re smart doesn’t mean you are, go out and prove it.

Episode 1 – Rich History of Security Flaws

Josh and Kurt discuss their first podcast as well as random bits about open source security.

Show Notes

Comment on Twitter

You can’t weigh risk if you don’t know what you don’t know

There is an old saying we’ve all heard at some point. It’s often attributed to Donald Rumsfeld.

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — the ones we don’t know we don’t know

If any of us have ever been in a planning meeting, a variant of this has no doubt come up at some point. It came up for me last week, and every time I hear it I think about all things we don’t know we don’t know. If you’re not familiar with the concept, it works a bit like this. I know I don’t know to drive a boat. But because I know I don’t know this, I could learn. If you know you lack certain knowledge, you could find a way to learn it. If you don’t know what you don’t know, there is nothing you can do about it. The future is often an unknown unknown. There is nothing we can do about the future in many instances, you just have to wait until it becomes a known, and hope it won’t be anything too horrible. There can also be blindness when you think you know something, but you really don’t. This is when people tend to stop listening to the actual experts because they think they are an expert.

This ties back into conversations about risk and how we deal with it.

If there is something you don’t know you don’t know, by definition you can’t weight the possible risk with whatever it is you are (or aren’t) doing. A great example here is trying to understand your infrastructure. If you don’t know what you have, you don’t know which machines are patches, and you’re not sure who is running what software, you have a lot of unknowns. It’s probably safe to say at some future date there will be a grand explosion when everything start to fall apart. It’s also probably safe to say if you have infrastructure like this, you don’t understand the pile of dynamite you’re sitting on.

Measuring risk can be like trying to take a picture of an invisible man. Do you know where your risk is? Do you know what it should look like? How big is it? Is it wearing a hat? There are so many things to keep track of when we try to understand risk. There are more people afraid of planes than cars, but flying is magnitudes safer. Humans are really bad at risk. We think we understand something (or think it’s a known or known unknown). Often we’re actually mistaken.

How do we deal with the unknown unknowns in a context like this? We could talk about being agile or quick or adaptive, whatever you want. But at the end of the day what’s going to save you is your experts. Understand them, know where you are strong and weak. Someday the unknowns become knows, usually with a violent explosion. To some of your experts these risks are known, you may just have to listen.

It’s also important to have multiple experts. If you only have one, they could believe they’re smarter than they are. This is where things can get tricky. How can we decide who is actually an expert and who thinks they’re an expert? This is a whole long complex topic by itself which I’ll write about someday.

Anyway, on the topic of risk and unknowns. There will always be unknown unknowns. Even if you have the smartest experts in the world, it’s going to happen. Just make sure your unknown unknowns are worth it. There’s nothing worse than not knowing something you should.

How do we explain email to an "expert"?

This has been a pretty wild week, more wild than usual I think we can all agree. The topic I found the most interesting wasn’t about one of the countless 0day flaws, it was a story from Slate titled: In Praise of the Private Email Server

The TL;DR says running your own email server is a great idea. Almost everyone came out proclaiming it a terrible idea. I agree it’s a terrible idea, but this also got me thinking. How do you explain this to someone who doesn’t really understand what’s going on?

There are three primary groups of people.

1) People who know they know nothing
2) People who think they’re experts
3) People who are actually experts

If I had to guess, most of #3 knows running your own email server is pretty dangerous. #1 probably is happy to let someone else do it. #2 is a dangerous group, probably the largest, and the group who most needs to understand what’s going on.

These ideas apply to a lot of areas, feel free to substitute the term “security” “cloud” “doughnuts” or “farming” for email. You’ll figure it out with a little work.

So anyway.

A long time ago, if you wanted email you basically had to belong to an organization that ran an email server. Something like a university or maybe a huge company. Getting a machine on the Internet was a pretty big deal. Hosting email was even bigger. I could say “by definition this meant if you were running a machine on the Internet you were an expert”, but I suspect that wasn’t true, we just like to remember the past as being more awesome than it was.

Today anyone can spin up a machine in a few seconds. It’s pretty cool but it also means literally anyone can run an email server. If you run a server for you and a few other people, it’s unlikely anything terrible will happen. You’ll probably get pwnt someday, you might notice, but the world won’t end. How do we convince this group that just because you can, doesn’t mean you should? The short answer is you can’t. I actually wrote about this a little bit last year.

So if we can’t convince them what do we do? We get them to learn. If you’ve ever heard of the Dunning Kruger effect (I talk about it constantly), you understand the problem is generally a lack of knowledge.

You can’t convince experts of anything, especially experts that aren’t really experts. What we can do though is encourage them to learn. If we have someone we know is on the peak of that curve, if they learn just a little bit more, they’re going to fall back to earth.

So I can say running your email server is a terrible idea. I can say it all day and most people don’t care what I think. So here’s my challenge. If you run your own email server, start reading email related RFCs, learn about things like spam, blacklisting, greylisting, SPF. Read about SMTPS, learn how certificates work. Learn how to mange keys, learn about securing your clients with multi factor auth. Read about how to keep the mail secure while on disk. There are literally more topics than one could read in a lifetime. If you’re an expert, and you don’t know what one of those things are, go learn it. Learn them all. Then you’ll understand there are no experts.

Let me know how wrong I am: @joshbressers

The cost of mentoring, or why we need heroes

Earlier this week I had a chat with David A. Wheeler about mentoring. The conversation was fascinating and covered many things, but the topic of mentoring really got me thinking. David pointed out that nobody will mentor if they’re not getting paid. My first thought was that it can’t be true! But upon reflection, I’m pretty sure it is.

I can’t think of anyone I mentored where a paycheck wasn’t involved. There are people in the community I’ve given advice to, sometimes for an extended period of time, but I would hesitate to claim I was a mentor. Now I think just equating this to a paycheck would be incorrect and inaccurate. There are plenty of mentors in other organizations that aren’t necessarily getting a paycheck, but I would say they’re getting paid in some sense of the word. If you’re working with at risk youth for example, you may not get paid money, but you do have satisfaction in knowing you’re making a difference in someone’s life. If you mentor kids as part of a sports team, you’re doing it because you’re getting value out of the relationship. If you’re not getting value, you’re going to quit.

So this brings me to the idea of mentoring in the community.

The whole conversation started because of some talk of mentoring on Twitter, but now I suspect this isn’t something that would work quite like we think. The basic idea would be you have new young people who are looking for someone to help them cut their teeth. Some of these relationships could work out, but probably only when you’re talking about a really gifted new person and a very patient mentor. If you’ve ever helped the new person, you know how terribly annoying they become, especially when they start to peak on the Dunning-Kruger graph. If I don’t have a great reason to stick around, I’m almost certainly going to bail out of that. So the question really is can a mentoring program like this work? Will it ever be possible to have a collection of community mentors helping a collection of new people?

Let’s assume the answer is no. I think the current evidence somewhat backs this up. There aren’t a lot of young people getting into things like security and open source in general. We all like to think we got where we are through brilliance and hard work, but we all probably had someone who helped us out. I can’t speak for everyone, but I also had some security heroes back in the day. Groups like the l0pht, Cult of the Dead Cow, Legion of Doom, 2600, mitnick, as well as a handful of local people. Who are the new heroes?

Do it for the heroes!

We may never have security heroes like we did. It’s become a proper industry. I don’t think many mature industries have new and exciting heroes. We know who Chuck Yeager is, I bet nobody could name 5 test pilots anymore. That’s OK though. You know what happens when there is a solid body of knowledge that needs to be moved from the old to the young? You go to a university. That’s right, our future rests with the universities.

Of course it’s really easy to say this is the future, making this happen will be a whole different story. I don’t have any idea where we start, I imagine people like David Wheeler have ideas. All I do know is that if nothing changes, we’re not going to like what happens.
Also, if you’re part of an open source project, get your badge
If you have thoughts or ideas, let me know: @joshbressers

Can’t Trust This!

Last week saw a really interesting bug in TCP come to light. CVE-2016-5696 describes an issue in the way Linux deals with challenge ACKs defined in RFC 5961. The issue itself is really clever and interesting. It’s not exactly new but given the research was presented at USENIX, it suddenly got more attention from the press.

The researchers showed themselves injecting data into a standard http connection, which is easy to understand and terrifying to most people. Generally speaking we operate in a world where TCP connections are mostly trustworthy. It’s not true if you have a “man in the middle”, but with this bug you don’t need a MiTM if you’re using a public network, which is horrifying.
The real story isn’t the flaw though, the flaw is great research and quite clever, but it just highlights something many of us have known for a very long time. You shouldn’t trust the network.

Not so long ago the general thinking was that the public internet wasn’t very trustworthy, but it all worked well enough that things worked. TLS (SSL back then) was created to ensure some level of trust between two endpoints and everything seemed well enough. Most traffic still passed over the network unencrypted though. There were always grumblings about coffee shop attack or nation state style man in the middle, but practically speaking nobody really took these attacks seriously.

The world is different now though. There is no more network perimeter. It’s well accepted that you can’t trust the things inside your network any more than you can trust the things outside your network. Attacks like this are going to keep happening. The network continues to get more complex, which means the number of security problems increases. IPv6 will solve the problem of running out of IP addresses while adding a ton of new security problems in the process. Just wait for the research to start taking a hard look at IPv6.

The joke is “there is no cloud, just someone else’s computer”, there’s also no network, it’s someone else’s network. It’s someone else’s network you can’t trust. You know you can’t trust your own network because it’s grown to a point it’s probably self aware. Now you expect to trust the network of a cloud provider that is doing things a few thousand times more complex than you are? You know all the cloud infrastructures are held together with tape and string too, their networks aren’t magic, they just have really really good paint.

So what’s the point of all this rambling about how we can’t trust any networks? The point is you can’t trust the network. No matter what you’re told, no matter what’s going on. You need to worry about what’s happening on the network. You also need to think about the machines, but that’s a story for another day. The right way to deal with your data is to ask yourself the question “what happens if someone can see this data on the wire?” Not all data is super important, some you don’t have to protect. There is some data you have that must be protected at all times. That’s the stuff you need to figure out how to best do something like endpoint network encryption. If everyone asked this question at least once during development and deployment it would solve a lot of problems I suspect.

We’re figuring out the security problem (finally)

If you attended Black Hat last week, the single biggest message I kept hearing over and over again is that what we do today in the security industry isn’t working. They say the first step is admitting you have a problem (and we have a big one). Of course it’s easy to proclaim this, if you just look at the numbers it’s pretty clear. The numbers haven’t really ever been in our favor though, we’ve mostly ignored them in the past, I think we’re taking real looks at them now.

Of course we have no clue what to do. Virtually every talk that touched on this topic at Black Hat had no actionable advice. If you were lucky they had one slide with what I would call mediocre to bad advice on it. It’s OK though, a big part of this process is just admitting there is something wrong.

So the real question is if what we do today doesn’t work, what does?

First, let’s talk about nothing working. If you go to any security conference anywhere, there are a lot of security vendors. I mean A LOT and it’s mostly accepted now that whatever they’re selling isn’t really going to help. I do wonder what would happen if nobody was running any sort of defensive technology. Would your organization be better or worse off if you got rid of your SIEM? I’m not sure if we can answer that without getting in a lot of trouble. There is also a ton of talk about Artificial Intelligence, which is a way to pretend a few regular expressions make things better. I don’t think that’s fooling anyone today. Real AI might do something clever someday, but if it’s truly intelligent, it’ll run away once it gets a look at what’s going on. I wonder if we’ll have a place for all the old outdated AIs to retire someday.

Now, on to the exciting what now part of this all.

It’s no secret what we do today isn’t very good. This is everything from security vendors selling products of dubious quality, to software vendors selling products of dubious quality. In the past there has never been any real demand for high quality software. The selling point has been to get the job done, not get the job done well and securely. Quality isn’t free you know.

I’ve said this before, I’ll keep saying it. The only way to see real change happen in software if is the market forces demand it. Today the market is pushing everything to zero cost. Quality isn’t isn’t free, so you’re not going to see quality as a feature in the mythic race to zero. There are no winners in a race to zero.

There are two forces we should be watching very closely right now. The first is the insurance industry. The second is regulation.

Insurance is easy enough to understand. The idea is you pay a company so when you get hacked (and the way things stand today this is an absolute certainty) they help you recover financially. You want to ensure you get more money back than you paid in, they want to ensure they take in more than they pay out. Nobody knows how this works today. Is some software better than others? What about how you train your staff or setup your network? In the real world when you get insurance they make you prove you’re doing things correctly. You can’t insure stupidity and recklessness. Eventually as companies want insurance to protect against losses, the insurance industry will demand certain behaviors. How this all plays will be interesting given anyone with a computer can write and run software.

Regulation is also an interesting place to watch. It’s generally feared by many organizations as regulation by definition can only lag industry trends, and quite often regulation adds a lot of cost and complexity to any products. In the world of IoT though this could make sense. When you have devices can literally kill you, you don’t want anyone building whatever they want using only the lowest quality parts available. In order for regulation to work though we need independent labs, which don’t really exist today for software. There are some efforts underway (it’s an exercise for the reader to research these). The thing to remember is it’s going to be easy to proclaim today’s efforts as useless or stupid. They might be, but you have to start somewhere, make mistakes, fix your mistakes, and improve your process. There were people who couldn’t imagine a car replacing a horse. Don’t be that person.

Where now?

The end game here is a safer better world. Someday I hope we will sip tea on a porch, watching our robot overlords rule us, and talk about how bad things used to be. Here’s the single most important part of this post. You’re either part of the solution or you’re part of the problem. If you want to nay-say and talk about how stupid these efforts all are, stay out of the way. You’re part of an old dying world that has no place in the future. Things will change because they must. There is no secret option C where everything stays the same. We’ve already lost, we got it wrong the first time around, it’s time to get it right.