Well, we’ve made it to the end. What started out as a short blog post ended up being 7 posts long. If you made it this far I commend you for your mental fortitude.
I’m going to sum everything up with these 4 takeaways.
- Understand the problem we want to solve
- Push back on scanner vendors
- Work with your vendors
- Get involved in open source
Understand the problem we want to solve
In security it’s sometimes easy to lose sight of what we’re really trying to do. Running a scanner isn’t a goal in itself, the goal is to improve security, or it should be if it isn’t. Make sure you never forget what’s really happening. Sometimes in the excitement of security, the real reason we’re doing what we do can be lost.
I always hate digging out the old trope “what’s the problem we’re trying to solve” but in this instance I think it’s a good question to ask yourself. Defining problems is really hard. Staying on goal is even harder.
If we think our purpose is to run the scanners, what becomes our goal? The goal will be to have a clean scan. We know a clean scan is impossible, so what really happens is our purpose starts to twist itself around a disfigured version of reality. I’ve said many times the problem is really insecure applications, or at least that’s the problem I tend to think about. You have to figure this out for yourself. If you have a scanner running make sure you know why.
Push back on scanner vendors
When a scan has 80% false positives, that’s not because your project isn’t built well, it’s because the scanner has a lot of serious bugs. High false positive rates mean the product is broken, it doesn’t mean your project is broken. Well, it might be, but probably not. The security industry has come to accept incredibly high false positive rates as normal. We have to break this cycle. It holds the industry back and it makes our jobs horrible. If you are a scanner vendor go make a sign that says “ZERO FALSE POSITIVES” and hang it on the wall. That’s your new purpose.
Setup a weekly or monthly call with your vendor. Make sure they understand your purpose and goals (remember your purpose isn’t just to run a scanner). Make them help you through rough patches. If you feel pain because of their product, they should feel it with you. A vendor who won’t work with you is a vendor who needs to be replaced. Good vendors are your partner, your success is their success. Your pain is their pain.
Work with your vendors
Now, when you find a scanner that has a lot of bugs, you basically have two choices. You can give up (and sometimes this is an acceptable decision). Or you can work with the vendor. At this stage in the technology, it’s very important we start working with these vendors. Report every false positive as a bug. Make them answer hard questions about the results you see. If nobody pushes back we’re going to see worse results in the future, not better. Products improve because of feedback. They don’t improve if we all just pretend everything is fine. I do think part of the reason code and application scanning seems to have plateaued is because we accepted poor results as normal.
If you are a vendor, remember that reports about false positives are gifts. Make sure you treat false positives like they are important bugs. Like all new markets, there will be winners and there will be losers. If your scanner reports the most results, but most of those are false positives, that’s not progress. The scanners with the most false positives will be on the losing side of history.
Get involved in open source
And lastly, help. This sounds like the old “patches welcome” we all love to throw around, but in this case I’m quite serious. Your product is basically open source. Some of the projects you are working with could use a hand to fix some of these findings. As annoying as you think a huge scan report is, imagine getting one when you’re working for free. It’s insulting and degrading. If you have actual real scan results that need fixing in an open source project, don’t dump it over the fence, get in there and help fix problems. If you include a dependency in your project, sending a patch upstream is the same as patching your own application.
If you’ve never contributed to open source it can be a terrifying. I just spent longer than I want to admit trying to find a nice “getting started in open source” guide. I wasn’t terribly impressed with any of what came up (if you know of one let me know, I’ll link it at the bottom). I’m going to start writing a post titled “How to get involved in open source for security people”, but until then, my advice is just go help. You know how github works, get in there and help. Be patient and kind, apologize when you mess up and don’t be afraid to ask stupid questions. Some say there are no stupid questions. There totally are, but that’s not a reason to be a jerk when asking or answering.
The one ask I have of everyone reading this is to help educate other on this extremely complicated an important topic of security scanners. It’s important we approach others with empathy and understanding. Security has a long history of being ill tempered and hard to work with. If someone is misunderstanding how a security scanner works or what it does, it’s our opportunity to help them understand. These scanners are too important to ignore, and they need a lot of work. We don’t change an industry by being annoying idiots. We change it by being respected partners.
I think we have an opportunity to see composition scanners make things better in the future. Composition security has been a taboo topic for a very long time. Many of us knew what was happening but we didn’t do anything because we didn’t have a solution. Security through obscurity works, until it doesn’t. There’s a lot of work to do, and we have to do it together.
Now go and teach someone something new.