Press "Enter" to skip to content

Observability trends

Cribl Senior Director of Market Strategy Nick Heudecker outlines three trends in observability 

1.Observability moves in-house
There are dozens of products with complex machine learning models aiming to capitalise on observability data. Many assume they’re the only tool in the stack, which makes them hard to integrate with other tools or with adjacent business processes; and many use generic models that may not be applicable to specific problems faced by operations teams. As a result, operations teams will shift away from monolithic, generic automation solutions towards more home-grown implementations built to solve the most pressing security and operational challenges. Rather than focusing on a single tool and data silo, teams will build tools with a mix of technologies accessing data from across the enterprise. 

2.Security teams drive observability maturity
Much of the conversation around observability has targeted developers on the basis that they are also the operators of their code. This view is popular in Silicon Valley, but outside the Bay Area developers are expensive and having them spend time on operational tasks they’re not experts in is seen as a waste of time and effort. Instead of developers driving the observability discussion, cybersecurity teams will take the driver’s seat and lead transformation in their companies. 

Security teams are heavy users of monitoring already, deploying a range of tools to uncover known threats. These tools fall short in three ways. First, they take a one-size-fits-all approach to the data they ingest, ignoring different levels of data quality and value. Second, current pricing models make broad security monitoring cost-prohibitive – ingest-based pricing penalises users for every byte ingested, while workload- based pricing penalises users for every search they run. These limitations hamper investigations and slow remediation. Finally, no tool or platform owns all the data, resulting in a fragmented data picture. An observability-based security architecture must weave all these fragments into a coherent picture. 

Observability helps security professionals uncover governance and compliance gaps; route data to multiple destinations for advanced analysis across a range of tools, e.g. routing the same data to Splunk ES to drive detection and case management and to Google Chronicle for threat hunting; conduct faster, more accurate post-mortems on security events; enrich data with additional context; and filter out low-value data. 

Security teams will demand more accessibility to observability data and better tools to manage it. 

3.Adoption of cloud-based monitoring and observability decelerate
There was a massive expansion of cloud-based monitoring and observability offerings in 2021, spurred by pandemic-driven digital transformation efforts. These frequently came with high costs and unpredictable performance. 

Most monitoring and observability workloads are surprisingly constant and consistent, with data processed growing at a more or less linear rate. This predictability makes them ideal for on- prem processing. The cloud might seem the ideal place to house growing volumes of observability data, but volumes are such that cloud-based object storage can be expensive. Many companies are ingesting over 40TB of data each day; some scale up to 100TB or more. Add in required retention periods, data transfer costs and API calls and it’s easy to see how the cost of cloud- based observability data can run into millions of dollars per year. As costs climb, organisations will experience invoice shock and pump the brakes on cloud-based monitoring and observability migrations. 

Please follow and like us:

Be First to Comment

Leave a Reply

Technology Reseller Magazine & Site is Published by Kingswood Media 2022