Blog Post

Threat Detection: All These Silver Bullets, None of Them Functional

The Silver Bullet

I recently wrote an article for Security Boulevard diving into the nuances of the SOC that make the skills gap even more pronounced. Just to prove to everyone I am not yet another cynical security professional that likes to point out everything that’s wrong with the industry, I’m going to describe something that I think will actually boost us into the future quite a bit. Bear Beer with me, here we go.

My engineering colleagues at Awake recently introduced me to something called functional programming, which itself is based on something called lambda calculus from the 1930s. At first, examples of programs written in functional languages looked like math wizardry to me.  My inclination was to think that functional programming just wasn’t for me and if I had any in-depth questions, I’d ask my more knowledgeable colleagues on the subject. That was before it clicked… and now I spend free time learning Haskell online and giggling like a child when something I write does what I want.

Apparently, I’ve been using a functional language for the past two years and just didn’t know it. In fact, the query language that the Awake Security Platform supports is technically one. Not only that, it turns out that a lot of my skills and passion that make me a good cyber investigator translate surprisingly well to functional programming. I’ve begun to call this similarity in thought process and workflows “functional thinking.”

Those of us that are CLI Warriors that prefer an arsenal filled with grep, sed, cut, and the like are already familiar with a few concepts that translate directly to functional thinking. These include:

  • Expects a certain input and outputs an expected result
  • Standalone tools that each have their own purpose
  • Can be strung together with pipes (|) to reach the desired final result

It also forces you to break apart your problem into smaller (reusable) pieces. In this case, it’s the detection of cyber attacks and threats. These smaller functions are easier to read, debug, and maintain (not my words), which are all extremely important in a field where we simply do not have enough people to keep up with the demand.

Functional languages are also typically strongly typed. Those of you that are like me and come from a python scripting move-fast-and-break things kind of background may lament such a term. However, the advantage this gives us greatly outweighs the initial annoyance of having to be mindful of what you’re doing (what a concept). What this means is that when using a language like this for detection, the detection will not even be accepted as valid unless the data that you’re passing around from function to function are the correct type. In my experience, this means that I get to spend less time scratching my head and tracking down bug patient zeroes through tracebacks, or running detections over and over to make sure that they work correctly, let alone detect what you’re after.

Talk is Cheap

I can sit here and theory craft with you all day, but instead, why don’t I just show you what I’m talking about?

One fun example, based on a blog post my colleague is working on, is used to detect when MS Word is started. The detection consists of TLS sessions to three different server names in quick succession; which is something that is not trivial to detect in 99% of today’s tools on its own.

In order to detect this appropriately, you not only need the ability to chain detections together using time, you also have to be parsing TLS fields and have an accurate understanding of what a ‘device’ is. As I’m sure you’re intimately aware: an IP address is not a device.  It may not seem relevant for these examples occurring so closely together, but any detection that looks at longer periods of time (perhaps low and slow persistent connections over a week) will almost definitely run into this problem.

This is trivial to do with functional thinking (and the right data and analytics under the hood).  To accomplish this, I created a function that allows us to chain detections together by device for a given period of time.  For this example, we simply pass three different detections, one for each TLS session, and a time duration.  Without boring you with source code, it looks something like this:

(TLS to [a-z]\-ring\.msedge\.net,TLS to, TLS to 1m

Simple to make, simple to understand, and we can save all of this as a stand alone function called “MSWordLaunched” or something similar. The only problem here is that someone opening up word is not malicious on its own obviously. Cool, let’s just make a few adjustments.

We can just follow what we already had with the function “after”, which is a simple function that filters the results to only those instances that then match a detection for “recipes.destinations.risky”. This detection uses a variety of algorithms and AI to detect bad destinations including DGA domains, destinations never seen before on the network, destinations that are rare on the network, etc…

All of the sudden we have a much higher fidelity, useful detection. In the same manner, we could use “before” to filter for results that were preceded by something, email access perhaps?

The more abstractions like “recipes.destinations.risky”, or “” that are available out of the box as well as created by the security team, the fewer detections the individual weary security analyst has to actually create.

Back to the Future of Security

A lot of attention is often given to how we display results from security tools for the analysts.  However, it seems that we don’t give as much thought to a user experience that more effectively enables threat hunting or the creation of custom detection. By abstracting different steps of an attack, and creating powerful, reusable functions, we can provide a threat hunting and custom detection mechanism that is both easy to use and understand. It’s also worth mentioning that while testing the above detection it detected something specific as intended by the analyst and nothing else. By starting with a larger dataset and filtering down it appears that we’ve successfully avoided false positives and false negatives in one fell swoop.

What this means for the industry is that threat hunters and custom detection creators can do less, but accomplish more. With better, higher fidelity and easily understood detections, analysts can focus on general ideas of threats without having to know the ins and outs of so many (which can be very daunting for a junior analyst). These junior analysts will also have less false positives mucking up their ticket queue, which gives them more time to actually learn on the job instead of running on the same treadmill with no change in scenery.

They will become more skilled cybersecurity professionals and go on to become knowledgeable managers and detection creators who will continue to improve our current methods well beyond what they are now… and the cycle will continue, but it will be a good thing.

It’s time for us to Func ourselves!

Part 1: How to Break Broken SOC Cycles


If you liked what you just read, subscribe to hear about our threat research and security analysis.

Tory Kei
Tory Kei

Threat Researcher