September 18, 2018

Lacking a meaningful way to make sense of all the data that the modern technology stack generates, the data can quickly overwhelm an IT operation’s ability to process it — turning that data into the equivalent of radio static.

Amid all the talk about potential business disruptors, security risks, and the challenges of managing a rapidly expanding technology stack, perhaps the most significant threat facing enterprise IT executives is one lurking in the shadows. And, in good Hollywood fashion, this erstwhile threat has appeared on IT’s doorstep not as a foe, but as a friend and savior. Its name?

Data.

To be more specific, this almost unnoticed threat is the noise created as IT organizations generate ever-more data from their systems.

This noise is now reaching a crescendo and enterprise IT executives must deal with it before it undermines their ability to operate at all, let alone deal with all of the other challenges and threats facing them.

The Static is Deafening

I once gave a speech in which, over the course of several minutes while I was speaking, I had the background change from blue to red. Despite being projected on a 50-foot screen immediately behind me, no one noticed until I pointed it out. This inability to see small and subtle changes to our environments over time is what has made IT organizations susceptible to the noise problem.

Over the years, IT leaders have slowly added more technical assets to their technology stacks — and have begun collecting multiple pieces of data from each of them. As the pace of growth increases, therefore, the data that the organization’s infrastructure generates grows exponentially.

While, on the surface, it might seem that having all this data is a good thing, that may not be true.

Too much data can, in fact, make it harder to find the critical information that organizations need to avoid disruptions, identify performance issues, and ensure operational stability — particularly when much of the data that IT organizations generate is incomplete, fragmented, or, at worst, inaccurate. The data can quickly overwhelm an IT operation’s ability to process it — turning data into static.

The Loud Consequences of Noise

The consequences of all this data becoming noise extends beyond the risks to operational performance, stability, and security. The longer-term ramifications of data run amok may represent an even greater and more significant threat to the enterprise’s ability to leverage its technology to compete in a digital world. While the predominant use case of technical data remains operational management, that dynamic is rapidly shifting. The advent of artificial intelligence (AI), and specifically machine learning, has created a voracious demand for data.

The challenge, however, is that to use data to fuel the development of AI models, it must be complete, accurate, and unfragmented. Unfortunately, these are not the words that most enterprise executives would use to describe the current state of their IT operational data.

The great risk, therefore, is that as the technology stack grows increasingly complex, using various forms of AI may turn out to be the only way in which IT organizations can effectively manage it. But without the right data to develop and train these AI-powered systems, IT organizations are at risk of being unable to support the ever-changing and ever-expanding needs of their organizations.

The Intellyx Take – Seek Out Context

Solving the IT noise problem may be one of the greatest challenges facing enterprise executives, yet few are paying it much attention. Part of the reason is the happening before our eyes problem — they just haven’t noticed the color of the screen changing. But even for those enterprise leaders that have recognized the issue, there has not been an easy way to address it.

Ironically, the industry is putting forward AI and machine learning as the way to solve this problem. But, as we’ve covered, you can’t use incomplete and fragmented data to address a data challenge. Instead, organizations must step back and address their noise issue more holistically to make sense of their data.

The solution, it turns out, is something that we, as humans, understand intuitively. When faced with a large amount of data that we cannot make immediate sense of, we naturally seek out context to give it structure and meaning. IT organizations must, therefore, seek to do the same.

This topic, in fact, is the subject of a new white paper I’ve written entitled, Transforming IT Ops with Machine Learning? Apply Context. In this paper, I explore what context means in terms of IT operations and how organizations must approach the application of context to solve their noise problem.

The complexity of the modern enterprise’s technology stack will continue to grow, and the challenges of managing it will continue to multiply. These are unpleasant truths that demand that IT operations teams transform the way they work, change the way they look at data, and embrace the need for context as they do so.

This article was first published on the ScienceLogic website.