Big Data-Driven Innovation: Disruption vs. Optimization

The recipe for successful innovation: begin with a good measure of disruption. Add a heaping helping of talent, and don’t forget to mix in plenty of creativity. Finally, a pinch of intuition. Stir and bake.

Recipe for innovation? Perhaps. But successful innovation? That’s another story. After all, whether an innovation will actually succeed – that is, meet or exceed its business goals – seems to require some unknown, missing ingredient to the mix. The recipe above may include necessary elements, but taken together, they are still not sufficient to guarantee success.

How’s a Digital Transformation professional to determine that secret element, that je ne sais quoi that leads to successful innovations, every time – or at least, most of the time? Simple. Let’s get out our Big Data analytics tools and crunch some data. Surely, boiling down Big Data to a syrup of innovation to that rare sprinkling of insight will provide that missing ingredient. Magnifique!

Driving Agility with Big Data

bananawatchAt Intellyx we consider innovativeness to be the most important, strategic aspect of business agility. Also along for the ride: responsiveness and resilience. Put these core business drivers together and you have a multifaceted picture of how and why businesses deal with change.

Analyzing the information at our disposal, of course, can help any organization deal better with change through a straightforward optimization process. Gather data on anything you’re doing, crunch the numbers, and make recommendations on what to adjust to make the process better. After all, such analysis has been the primary purpose of business information since early managers crawled out from their cave and held the first punch card up to the wan light of morning.

Data analytics in support of human decision making, however, has one flaw – the human. This weak link in the data-driven agility chain becomes apparent as we move to Big Data: as the data grow so too do the results of the analyses, and yet people have a limited attention span and with it, the ability to process information. It doesn’t matter how wonderful the reports your newfangled Big Data tool generate if no one has the time or predilection to read them – or even worse, understand them!

The answer to each human’s exasperating lack of attention, of course, is to say goodbye to the weakest link, and establish fully automated feedback loops. Here’s an example: any business with equipment to maintain, from airlines to factories to data centers, knows that various widgets will eventually fail. Gather enough data to determine each part’s mean time between failures (MBTF), a statistical measure that predicts when a particular doodad will give up the ghost. Consider this statistical analysis Level #1 on our road to Big Data nirvana.

Based upon the MBTF, then, we schedule a replacement before each whatzit is expected to break. However, no one really knows when it’s gonna go, so we now have to balance the prospect of replacing it too early (thus wasting money on a part that still has some useful life to it), and waiting too long, thus increasing the odds it’ll break in production – an even more expensive headache.

The traditional approach to this kind of problem is to generate a bunch of reports, feed the data from those reports into various mathematical formulas, and adjust our maintenance schedule accordingly. Such mathematical machinations would constitute analysis at Level #2.

We’ll get much better results, however, if we automate this feedback loop, thus accelerating our ability to respond to changes in the environment, as well as taking error-prone, lazy humans out of the equation. Now we’ve reached Level #3: feeding back our analysis to improve results automatically over time. In other words, we’re now more responsive to change in the business environment – an essential aspect to business agility.

As you might expect, such automated feedback has been a staple in the equipment maintenance world for a while now. However, there are many other business areas that have yet to take a page out of the broken whatzit playbook. Does your organization, say, optimize its salaries based upon business outcomes – in real-time? Do your security policies balance the cost of security with the corresponding risk of loss, again in real-time? In general, how many of the processes in your organization are subject to continual optimization based upon automated, data-driven improvements?

Driving Innovation with Big Data

Such business-centric automated feedback is perhaps one of the largest potential Big Data benefits in the enterprise today – although adoption of this approach is spotty at best. Let’s say, however, that you’re one of the progressive early adopters in the automated Big Data-driven feedback world. Does this streamlined ability to optimize all aspects of your business improve your ability to innovate?

Perhaps, but not in a straightforward fashion. As I explained in an earlier Cortex newsletter, optimization activities in the absence of disruption will lead to local optima, but only disruption-driven innovation will lead to the most strategic outcomes. Does that mean innovation is always a crap shoot? Or is there some way we can improve our odds of successful innovation, perhaps by leveraging feedback-driven Big Data optimization techniques?

Time for a hypothetical example. Let’s say a leading consumer electronics manufacturer, perhaps named after a fruit, decides to create their first ever wrist-mounted Internet-of-Things device: the Banana Watch. Such devices are a new market for the Banana Company, and in fact, their device is so innovative that it’s essentially creating an entirely new market segment on its own (much as the Banana Pad did a few years ago).

So, is Banana Inc. taking a complete flyer on the Banana Watch? Or do they have data to suggest the Watch will be successful? And furthermore, how do those data help them to optimize their innovation process itself?

One the one hand, entering a new market, especially when you’re creating that market as you go, is always risky. But on the other hand, they do have extensive data about customer buying behavior, price points, design metrics, related markets, and a plethora of other information that feeds an internal analytics algorithm that is indubitably second-to-none.

The reason Banana is so successful in the broader consumer electronics market – and in fact, has been so successful over and over again – is they optimize what they can optimize and they disrupt what they must disrupt. They never make the mistake of trying to optimize what they should disrupt, as that approach would stifle innovation. But they also avoid the mistake of disrupting what they should optimize.

The Intellyx Take

The lack of useful optimization, either because of lack of data, insufficient analytics, or a simple dearth of management will is actually a risk factor for innovation, just as excessive or misapplied optimization is. In other words, it’s a gigantic mistake to believe that you can innovate your way out of a lack of insight – insight into your market, your customers, or your business. Sure, people try it all the time – but most of the time they fall flat on their face. The few people that manage to succeed at such innovation-in-the-dark are simply the lucky ones.

For companies that understand how to balance optimization and disruption, luck is still a factor in how successful any particular innovation will be – just as luck is a factor in whether a particular widget will outlive its MTBF. Taking Big Data-driven optimization to the highest level is how companies make the most of their breakage-prone widgets, just as it is the secret to any organization who wishes to establish innovation as a core competency. Know when to optimize, and know when to disrupt – and above all, know how to tell the difference.

Thanks to Jim Scott at Intellyx subscriber MapR and Raj Dalal at BigInsights for helpful ideas for this article. Please click here to take the BigInsights 2014 Big Data Study survey and enter to win a Samsung Galaxy S5. Image credit: Jason Bloomberg.

SHARE THIS:

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.