By Jason Bloomberg
Real-time streaming data has become a reality for an increasing number of enterprise applications. From the “internet of things” to artificial intelligence, from mobile analytics to a diverse range of big-data challenges, dealing with firehoses of information in real time is rapidly becoming standard fare for enterprise information technology.
Over the years, many technologies have come to the fore to deal with streaming data in various ways. Asynchronous interactions, from traditional message queuing to publish/subscribe to the more modern AsyncAPI standard, have added to the technologists’ toolbelt.
Architectural styles have also kept pace, with the message-oriented, event-driven architecture dating from the 1990s giving way to a revamped, streaming-oriented EDA. Today, the streaming data story is all about massive scale – scale in both dynamic business and technology environments without sacrificing performance, resilience and accuracy.
In other words, streaming data is becoming part of the cloud-native world, since Kubernetes is the only infrastructure technology on the horizon promising the real-time scale that today’s streaming data applications require.
The many paths to cloud-native real-time data streaming
I spoke to 10 vendors with various offerings in the cloud-native streaming marketplace to see how they were approaching this challenging area. I found that many of them had been in the streaming business pre-Kubernetes, and were now leveraging existing technologies and architectural patterns while incorporating them within new cloud-native approaches. Here are some examples.
