This is a personal post and does not reflect the thoughts or opinions of my employer.
I’ve written and deleted this post multiple times. Mainly because I don’t want the attention or feedback. However, I know there is value in blogs, and posting them for reference and helping others. I’m reminded of my friends Matt Nelson (@enigma0x3) , Matt Graeber (@mattifestation) and countless others who document their research better than me.... The way they write an communicate, encourages me to do better.
Let’s talk about testing software. My experiences and thoughts.
It is been almost 20 years since I’ve been, in some capacity, testing software. My very first gig was at Cisco Systems in early 2000’s. It was here I participated as a lab rat in a project called SAFE - Secure Architecture For Enterprise
That’s me, in another life, the last credit on the last page :). But for me, it was the beginning of learning to test and validate security systems.
“How do we know this is working?”
I was just a Jr. tester on the project, but I still remember well. For those of you new to security, I encourage you to volunteer for any position. I was working in this lab at night building and helping support this research. I learned so much, because I volunteered.
A question we need to keep asking as we deploy our security systems is this…
“How do we know its working, to prevent, detect, alert, etc…?
Later I would move to a role to test and train on SQL Server and Enterprise Data Analytics. SSAS, SSIS, etc… ETL anyone? I learned a bunch of Windows Internals and C# trying to test the efficacy and security of database systems. PS - SSIS Tasks are still amazing ;-).
But I can’t talk about them. :)
I don’t share these things to boast, only to share my path.
One of the books I encountered along the way was this one:
Lesson 21 has always stuck with me. It implores us to use our imagination when we test. It admonishes us to consider tests that the designers of a software system, never imagined.
“Overall, thinking like a tester leads you to believe that things may not be as they seem… Perhaps its that we failed to imagine an entire category of test; testing we would never have found given twice the time and resources…”
I have used this quote in nearly all my public and private talks
And so I’ve tried to apply this when I started testing Application Whitelisting. Some have used my research to say “Whitelisting has bypasses and therefore is no good…”
Nothing could be further from the truth. I’ve tried to demonstrate where the gaps live in Application Whitelisting and what to watch out for…
“Morpheus : This is a sparring program, similar to the programmed reality
of the Matrix. It has the same basic rules, rules like gravity. What you must learn is that these rules are no different than the rules of a computer system...some of them
can can be bent. Others...can be broken. Understand?”
Software Supply Chain Compromise can, and has, proved to be a *very* effective way to affect your influence in organizations that use that software. If you can exploit trusted applications? Then you can probably blend in and evade detction.
People often trust signed binaries, even though signed binaries only ever was meant for Integrity and Identity validation… Not intent…
This has motivated me to find and abuse things that are signed by vendors. Because likely that is what adversaries will choose to do.
Anyway, I close with this.
Test what you have.
Provide assurance to yourself, your team, your company, your community, whomever you are defending, that what you have, or what you will buy, actually works...
I am reminded of “Lesson 68 - Never Assume that an obvious bug has already been found”
I can assure you, many times when I test, I think, or have seen…
“This probably won’t work… but just in case” then, calc pops :) undetected…
(cc: @enigma0x3 Matt Neslon 0007 :) ) <= An amazing find in Device Guard. I remember when Matt found that, and I was like. H O L Y S H - T... :). I love Matt's research and questioning mindset. Check out his latest post on Privilege Escalation.
I’ll close with this.
All software has flaws. And memory corruption is only sometimes that flaw. ;-)
Many of them have not been found, those that have been found are not shared publicly, and then there is a subset of publicly known flaws.
There is value in finding defects, and there is no shortage of defects to be found. Keep at it, ask for help, publish what you learn…
I can assure you, there _are_ gaps in your defenses. Perhaps part of what we do as defenders is find them ourselves, fix them, and move on.
Dispel the illusion that things are working. Challenge assumptions. Question everything, but not necessarily out loud ;)
That’s all I got today.
PS - Free Research Idea/Nudge. TypeLIb Files (.tlb). LoadTypeLib & Assembly and TypeLib Conversions.