Welcome to the API Economy! Now that we’ve worked all the kinks out of RESTful APIs, we now have seamless interoperability among all manner of endpoints, from legacy enterprise web services to microservices.
If only it were that easy. True, vendors have largely risen to the management and governance challenges such APIs face (although more work is still needed around microservice interface governance), but the elephant in the integration room remains: data integration.
Back in the web services days, we offloaded the role of data formats to XML schemas, and leveraged ESBs or XML appliances to perform complex XSLT or XQuery transformations.
This heavyweight, middleware-centric approach to data integration served many of our needs at the time, but today’s cloudified, containerized world needs far more lightweight, scalable approaches to data integration.
To fill this need, along came REST, promising lightweight, web-centric, cloud-friendly integration. Add a REST-centric API management tool, and you’d be a full-fledged member of the API Economy, right?
Just one problem: REST never did handle data integration. After all, REST is web-centric: it provides for Internet media types (formerly known as MIME types) that can include metadata representations of data formats – but leaves the rest to you.
In other words, in the REST world, data integration is largely roll your own. At least in the web services days we had XML schemas to keep us honest. Today, schemalessness is the way to go – providing the lightweight flexibility of JSON, but kicking the data integration can down the road.
The missing piece of this puzzle, just as it was in the web services days, is data transformation. Unlike the ESB-centric 2000s, however, today’s data transformations must be lightweight and scalable. Transformations that can run as microservices themselves, and thus can appear anywhere in a hybrid cloud architecture, as opposed to being locked away inside centralized middleware or physical appliances.
To succeed in the API Economy, therefore, you must look beyond API management to data integration as well – without falling for warmed-over middleware-centric integration technologies from the last decade. Instead, look to vendors like SnapLogic who built data integration from the ground up to be cloud-centric: horizontally scalable and microservices-based.
Complex data transformations have always been processor intensive, which is why we often offloaded them to dedicated appliances. SnapLogic, in contrast, leverages the horizontal scalability of the cloud along with the power of modern transformation technology to offer real-time data transformation suitable even for today’s web scale data streaming applications.
The API lie is that data integration is an easy problem to solve. It’s not. That’s why it’s so important to leverage modern data integration technology for big data challenges, streaming data applications, or increasingly routine hybrid cloud integration.
SnapLogic is an Intellyx client. Intellyx retains full editorial control over the content of this article. Image credit: mhagemann.
Comments