Blog Post

Seek Observability in Selecting Network Security Analytics Solutions by David Monahan

Packets are the lifeblood of the network and most daily computer operations. In virtually every transaction and operation, we leave a network footprint. Only in the case of a local system login accessing applications and data that are locally resident on that system do we not leave a network trace. If we use the Internet, log in via a domain or realm, access a SaaS or other cloud services, or use network-attached storage, we leave a network trace.

Because the network is the tie that binds business, leveraging network-focused security analytics is a critical part of security monitoring and response for all connected businesses.

All security analytics rely on large datasets, supervised and unsupervised machine learning (ML), and other intellectual property (a.k.a. secret sauce) unique to each vendor. When attempting to detect lateral movement, identify insider threat activities like credential abuse and information theft, or trying to search for other threats within your boundaries and data, consumers need to understand that it is the company’s secret sauce that really makes the difference on creating visibility and observability, not noise.

Noise is a product of both false positive alerts that indicate a problem when none exists, and useless alerts that identify so many normal occurrences that the real threats are not easily discerned or extracted in the overall volume of alerts. Poorly designed systems and/or ML systems with too little data or single-dimension AI/ML approaches are no different than older systems. They can produce high levels of these problematic alerts. However, well-designed ML and AI systems rely on a combination of supervised, unsupervised, and other forms of machine learning to produce not only visibility but observability.

Visibility brings important incidents to the forefront, while observability means they can be measured and acted upon. Observability with physical sciences and the control theory is a measure of how well internal states of a system can be monitored and thus acted upon from knowledge of its external output, which is packet information in this context.

The network has the information necessary to make entity actions visible and observable. With proper placement, observability enables early detection and an effective response. As a consumer of network security analytics, it is the buyer’s job to perform due diligence to understand how the prospective vendor applies their special sauce to ML and AI algorithms to achieve accurate observability. Resist attempts to convince the unwitting buyer that all approaches are the same so the cheapest is the best value, or that the most expensive solution is the best solution. It is most likely just the best-known solution.

It is the secret sauce that differentiates each solution. Solutions that derive their secret sauce from recent innovations in data science enable better defenses, higher detection, and stronger response efficacy. In evaluating solutions, patents and trade secrets can be helpful indicators. Ultimately, however, the best indicator is in the proof of concept on your own network data. Take your time, place the collectors in strategic areas with good access traffic, and let them run for a week or two. Remember, for a proper comparison, try to run them in parallel or use the same packet captures to compare the total volume of alerts and the volume of actionable alerts that drive observability.

David Monahan
David Monahan

Managing Research Director, Security and Risk Management, EMA