Security log analytics is pretty much SecOps table stakes these days. But that doesn’t mean every organization is getting it right. Before, I discussed three reasons you can’t live without security logs—here, I’ll talk about a few common mistakes that could be holding you back from leveraging this goldmine of data to its fullest potential.
Fast Security Logs Are the Key to Beating the Clock
When hackers gain access to a network, they can linger for days or even weeks before you know they’re there. While undetected, they’re gathering admin credentials and doing recon. Security logs can alert you to an intruder, but many aren’t fast enough. Or, they don’t gather quite enough clues to raise a red flag. When they do, it’s often too late and the damage has already been done.
That’s why fast security log analytics is a key part of any defense strategy—the “before” of an attack that is as critical as the “during” and the “after.” Let’s take a look at what fast security logs are, why they matter, and how you can start using them at your organization.
What Are Fast Security Logs And Why Do They Matter?
Security log analytics consists of log data generated around the clock by network systems, end user behaviors and actions, and endpoint activity, and security monitoring systems. Log and event data is spread out everywhere across an enterprise and can be found in files, applications, sensors, network events, virtual machines, operating systems, clouds, security devices, operating systems, and more.
Security log analytics can help make sense of all this data and help to power:
- Rapid experimentation
- Threat hunting
- Purple teaming exercises
In terms of threat hunting, specifically, security logs are only valuable if they’re fast and comprehensive. Simply put: You need your logs and underlying systems to be fast if they’re actually going to help you defend against cyberattacks. Otherwise, you’re looking for a needle in a haystack on borrowed time.
How Do We Define “Fast?”
You can measure the speed of security log analytics in terms of terabytes per second or by a more arbitrary (but also more relevant) measure: “breakout time.”
Breakout time is the time between when an attacker breaks into your system, gaining access as a regular user, and when they elevate that user privilege to “admin.” With admin privileges, they can really start to wreak havoc. We used to measure breakout time in days or hours. Today, it’s down to minutes—about 90 minutes, to be exact. That’s how long you have to find that needle in the haystack.
Speed and performance of security logging systems determine how quickly you’re able to:
- Detect and respond to threats in near real time
- Identify anomalies in historic and behavioral data
- Leverage advanced algorithms and AI to detect never-before-seen threats
However, too many organizations miss out on these capabilities due to a few common mistakes.
Common Fast Logging Mistakes
Effective security logs rely on two things: fast analytics platforms and fast data storage solutions. Trying to detect anomalies without those two components is nearly impossible—but that’s not all. Other common mistakes companies make with their security logs include:
- Not enough logs, and therefore, limited visibility.
Think of your security logs as windows into everything going on within your systems. If you’re looking through a peephole, your view is too narrow. But, see things through a big pane of glass and you’ll get a more holistic view of everything that’s going on. Put simply, everything should be logged at all times.
- Logs that aren’t correlated across networks, endpoints, and end users.
Your logs are only as valuable as they are informative. To be truly useful, they need to be able to correlate events occurring in many different areas of your ecosystem, including cross-referencing across the network, endpoints, and end users.
- A back-end infrastructure that isn’t fast enough to support advanced analytics.
Without a powerful data storage back end, the reality is, even successful correlation queries will be too slow to reveal threats in time. When you’re looking for that needle in the haystack, think of incredibly performant data storage as a magnet to draw that needle to the top. It makes quick work of threat detection and analytics. But if your storage is slow, your data lake too immense, or your queries poorly written, it can take hours or even days of research to pinpoint an issue or potential threat.
When you’re looking for that needle in the haystack, think of incredibly performant data storage as a magnet to draw that needle to the top. It makes quick work of threat detection and analytics. – Andy Stone, CTO-Americas, Pure Storage
How to Supercharge Security Log Analytics
For security log analytics to deliver the firepower you need to combat cyberattacks, they should include:
- Real-time processing with search and query performance that’s reproducible, no matter the log size. This requires high throughput, low latency, and consistent performance finely tuned for any scenario or scale.
- Embedded data reduction and on-demand scaling to log and retain more data for longer, for the richest possible analysis.
- Easy scalability for multiple workloads, concurrent queries, and variable data patterns to accommodate fast analysis of multi-petabyte data sets.
- Multiple logs that correlate data across networks, endpoints, and end users.
- Management simplicity to let you easily build new queries and reduce complexity.
Pure Storage® offers the fastest analytics processing available for fast logs, especially if you’re already using platforms like Splunk and Elastic. With unified fast file and object storage (UFFO) from Pure, you’ll achieve the speed and agility IT and SecOps teams need today by un-siloing data, separation of compute and storage, and elastic, automated infrastructure so you can focus on insights, not operations.
Remember—security log analytics is the most important tool in your toolbox for the before of a cyberattack. Time is limited, so you must make the greatest impact possible in the shortest window possible.
Bottom line: In today’s threat landscape, you can’t afford for your log analytics to be slow.
Boost Splunk by Maximizing Server Utilization with Containers & Pure
