Not too long ago, IT was little more than a cost center – do more with less, keep the lights on, and struggle for relevance in an era when IT didn’t matter. The world of enterprise IT comprised legacy systems, physical servers, and inflexible architectures.
Then along came Google. And Amazon. And eBay. Eventually Facebook, Netflix, and scores of other technology-based companies joined in, rethinking what IT was all about. Many of these twenty-first century darlings drove innovation in hardware, networks, software, and the architectures that made everything work.
Welcome to the world of web scale.
In spite of these astounding advances in technology capability, the denizens of enterprise IT continue to struggle with legacy systems, legacy software, legacy…everything. Their world still centers on constrained resources – not enough speed, not enough scale, not enough money to go around. Keep the lights on and maybe – just maybe – you’ll get to do a bit of innovating as well.
The web scale world is just the opposite. These companies had cracked the nut – speed and scale in abundance, with no ball-and-chain legacy in sight. Not coincidentally, their pace of innovation has reached unprecedented levels, as web scale market caps surpass those of traditional enterprises, many of whom spent decades to grow to their current size.
Web scale, therefore, represents both the technology itself, as well as a way of thinking about how to manage and grow an innovative business at scale.
Enterprise executives around the world are now thinking to themselves: we need that. Now we know it works. How do we bring the world of web scale to the enterprise?
As enterprises progress with their digital transformation initiatives, they typically implement elements of web scale within their IT organization. However, you can’t take a century-old bank and expect to turn it into Google – nor would you want to.
Fundamentally, web scale means something different to a traditional enterprise than to the web scale companies themselves. We call this perspective enterprise web scale.
Web scale companies, of course, had the luxury of being able to architect their environment for web scale from the beginning. Enterprises, however, are not so lucky. Such companies scale by adding new capabilities onto existing ones, as retiring legacy assets is rarely practical or cost-effective.
Architecting Enterprise Web Scale
The capabilities enterprises are likely to add to their environment to build out enterprise web scale capabilities are likely to come from a set of relatively new open source projects. Perhaps the best known of these is Hadoop, but Hadoop was merely a harbinger.
Today we have a plethora of open source initiatives that one way or another distill the best practices of web scale companies like Google, Salesforce, and others. On this list: Cassandra NoSQL database, Kafka message broker, Solr enterprise search server, and Storm real-time event computation system, among others (all from the Apache foundation).
The four Apache projects above are in varying stages of maturity, with work going on at a frenetic pace. As a result, enterprises have largely taken a wait-and-see approach to them up to this point.
However, that wait-and-see attitude is rapidly changing, as enterprise decision makers increasingly realize that the risks of not taking advantage of enterprise web scale as their organizations become software-driven is perhaps greater than the risks inherent in implementing such still-maturing technologies.
As a result, enterprise architects must step up to the table as advisors for all things web scale. They must not only keep tabs on the progressing capabilities of each package, but understand how any web scale technology under consideration fits into the existing IT landscape.
Furthermore, becoming web scale for an enterprise typically means integrating web-scale capabilities with existing on-premise assets. This reality presents two core challenges EAs should be prepared to address: first, they must understand the subtleties of integration in the context of web scale to on-premise, and second, they must maintain the big picture of the overall strategy goals of the IT organization and the enterprise at large.
The integration challenge presents what we might call an ‘impedance mismatch’ – connecting fast to slow, or more precisely, integrating elastic, asynchronous infrastructure with existing traditional APIs and database interfaces.
Perhaps an enterprise leverages a public or private cloud in addition to existing on-premise capabilities. Another common pattern is for a company to build out big data capabilities (either in-house or in the cloud) that must now integrate both with the sources of data as well as with the business intelligence, visualization, or other analytics tools that consume and display the results of the big data processing.
Leveraging older integration technologies like ETL or ESBs to solve these modern integration challenges is more likely to cause performance bottlenecks and other issues. Instead, consider integration technology like SnapLogic’s, which is designed to support integration in enterprise web scale environments.
Driving End-to-End Digital with Enterprise Architecture
Integration, however, isn’t the only challenge that EAs are well-positioned to address. Simply adding web scale technologies to an existing environment is the recipe for additional silos – both technological as well as organizational.
However, in spite of the silos so typical of any large enterprise, the digital priorities of the organization are end-to-end. All elements of a customer-facing app, from the user interface all the way to the underlying systems of record, must perform all the time, and at top speed.
In fact, the end-to-end requirement – both technological and organizational – is perhaps the most transformative aspect of digital transformation for many organizations. As a result, EAs should spearhead the effort to avoid making the situation worse by treating newer web-scale efforts as a separate silo.
The secret: EAs must always circle back to the underlying business driver for any IT investment, web scale or otherwise. In the context of today’s digital initiatives, such business drivers are usually customer-focused – and customer needs will not be met by creating a separate web scale silo.
In the final analysis, enterprise web scale requires more than simply adding new technology. It requires both modern integration approaches as well as an end-to-end organizational context that enterprise architects are well-suited to lead.
SnapLogic is an Intellyx client. At the time of writing, no other organizations mentioned in this article are Intellyx clients. Intellyx retains full editorial control over the content of this article. Image credit: Ryan Hodnett.
Comments