Companies that can quickly deliver new and innovative software that can scale to meet customer demand will inevitably win. The advent of cloud native technologies, with their malleable, modular microservice architectures, holds great promise for companies to deliver on this maxim.
A cloud native development team should quickly generate transformative digital innovations that deliver high-demand customer capabilities and operational value breakthroughs.
So why are so many enterprises struggling to deliver on the cloud native promise? Despite our best efforts to recruit and train the best people, we are experiencing an industry-wide shortage of skilled software development and operations talent–and the inherent complexity of cloud native development is exacerbating the problem.
One thing is certain: we’re not going to hire our way out of this mess.
Why organizations defer cloud native innovation
With an estimated $5B or more than 41K person-years of labor invested in open source coding and architectural innovations that support delivering scalable applications almost anywhere, the software industry as a whole is already moving at a supersonic rate of innovation toward cloud native applications.
Who buys cloud native? Even non-technical executives are starting to understand the basic building blocks of cloud native applications. They know it has something to do with Kubernetes pushing out containers, so the resulting applications are more modular and take advantage of elastic cloud infrastructures.
That’s not an incorrect understanding, per se, but it is a crude oversimplification of the true complexity of the cloud native computing technologies. One gander at this cloud native landscape, with many hundreds of interconnected technologies at various stages of incubation reveals open source projects for configuration, networking, security, data handling, memory, API management, service mesh – and on and on.
And that’s only counting packages on the open source mega-release train. There are also hundreds of vendors offering their own development, management and support tools atop this ever-moving CN platform.
Cloud native expertise can’t be bought. Thousands of interconnected and constantly changing components and tools makes a developer’s learning curve steep and continuous.
Posting a hiring rec that demands developers have “5 years of cloud native experience” – or even demanding “one year of cloud native experience” is tricky, since last year’s stack might no longer be relevant this year. What matters is finding development staff that creatively react to change, which is hard to prove until they work on a real cloud native project.
Companies that leaned heavily on an outsourced development model also find that no longer works, as the few consultants with a real knack for cloud native development are never waiting on the bench for a gig. Furthermore, truly innovative ideas that impact customers represent a competitive differentiator, and should come from within the enterprise.
We can’t hire or outsource our way into cloud native innovation, or out of this skills shortage. The only way to sustainably grow is through building cloud native development talent and capabilities from within, following the introduction and maturation of cloud native projects, as well as keeping tabs on the vendor and end user community at large.
Budgets aren’t keeping up with the rate of change. While the market demand for digital offerings keeps doubling every couple years, IT budgets usually get a budget belt-tightening as a reward for managing and scaling so much technology.
Teams are becoming smaller and leaner, with a mandate to deliver twice the output with half the people. The need for specialized skill positions only increases as concepts like event-driven architecture, machine learning data lakes, real-time analytics, and zero-trust security policies turn into production-grade requirements.
Development teams are becoming more like American football teams, where athletes are recruited and trained for specialized positions, than basketball teams, where talents like passing and shooting are similar for every player on the court.
While expertise and specialization are good things, this trend eventually leaves us with even fewer critical points of failure within development organizations.
Why a developer platform matters
No matter what application or target environment they are contributing to, developers will still spend most of their time coding within an IDE. This hasn’t changed for decades.
Over the years, low-code solutions have often surfaced as potential methods for addressing development skill shortages by lowering the technical bar of entry for development. Vendors proposed everything from 4GLs, to WSYWIG editors, to ‘app builders’ running atop Excel files to get non-technical employees involved in building apps.
Low-code solutions attempt to reduce much of the development labor of software by abstracting coding and operations with prebuilt objects and workflows in a platform that reduces the need for coding in a more modular or reusable fashion. Thus, non-developers can participate in development through drag-and-drop interfaces (to varying extents), while expert developers can do more with less coding of common or redundant aspects of the application.
The low-code concept is now making its appearance now within DevOps tooling for cloud native development circles, but it has proven difficult to translate the deeply infrastructure-focused orientation of CI/CD pipelines into easy wizards that would make sense to less technical audiences.
The proliferation of complex open source tooling, third party service APIs and code that is being mixed and matched from repositories within cloud native is driving teams toward a new platform engineering approach.
Platform engineering practices seek to create shared resources for development environments–encouraging code, component, and configuration reuse.
These common platform engineering environments can be represented within a self-service internal development portal or an external partner marketplace, often accompanied by concierge-style support or advisory services from an expert team that curates and reviews all elements in the platform.
Of particular importance is the need to govern the platform’s self-service policies for access permissions, code, logic, data, and automation at just the right level of control for the business it supports.
A platform with too much flexibility, where roles and rules are loosely defined, will end up creating too much complexity, as stakeholders will not be able to distinguish between the relative value or usefulness of so many unvalidated and poorly categorized components on the platform.
Too much rigidity in policy design can create the opposite problem, where centralized governance and approval cycles for every element can slow down solution delivery and take away the innovative freedom developers need to solve unique problems.
The right approach to platform engineering involves a middle way, or perhaps, a paved road:
“A common metaphor for describing this approach to platform engineering is binding best practice into a ‘paved road.’ There will always be other paths to the goal of deployed, quality software, but one road is the paved one, and hence the easiest and best used path.” – Jason Bloomberg, Managing Partner and Principal Analyst, Intellyx
Distributed systems with microservices architectures are difficult to create a platform for, as by nature they call on unpredictably changing open source and vendor-managed components, and operate using ephemeral systems and workloads. A modern approach to cloud native platform engineering can finally bring the principles of governance, consistency and reuse to the table.
Speedy innovation through infrastructure abstraction
Any critical application within the enterprise estate – no matter what language it was coded in, or what target infrastructure it runs on – shares key service level objectives (SLOs) of reliability, performance, and security.
Cloud native applications naturally share all of these characteristics, but what really sets them apart from conventional apps is the SLO of faster delivery of new features at scale. A need for customer-driven innovation sets organizations on the path toward cloud native development approaches and microservices architectures.
Refreshingly – or maddeningly, depending on how you look at it – there’s no single right way to ‘do’ cloud native. Even Kubernetes, while seemingly the only technology some talk about, is just an abstraction of container orchestration and only one option for going cloud native.
As an open source movement, the CNCF purposely leaves the approach open to interpretation, so that the community can collaborate on new projects that will meet the future needs of different industries and hybrid IT deployment environments. It doesn’t dictate a particular language, or even a specific piece of infrastructure.
That’s excellent, but it also leaves short-handed end user dev teams managing insanely complex plumbing and experimenting with an array of integration options, rather than building new functionality for customers. That’s where platform engineering practices can save the day.
The decision to create a platform is a commitment to help developers of varying skill levels abstract away the complexity of underlying cloud native architectures with interfaces and tools atop readily configured environments. Easier said than done.
Literally, any team could stand up an interface that deploys a configured instance of Kubernetes, Helm and Crossplane, some kind of data store, an ELK stack, yada yada, and call it a platform. And they might even be 80% of the way there for some end users with that. Such is the power of having readily available open source tools, components, and repositories out there.
There’s a big delta between simply provisioning necessary technology resources, and a platform engineering team strategy that can help understaffed teams develop innovative software.
A cloud native platform engineering approach can provide ease of use, elimination of toil, and reduced cognitive load for development teams – without passing up on the competitive advantage of design and technical talent that can bring differentiated innovation to market.
The Intellyx Take
Cloud native innovation is not just about delivering more software, faster, at less cost, to garner more customer revenue. It is about transforming the organization’s overall digital experience, and delivering better digital experience for all developers and all stakeholders in cloud native innovation.
Delivering an innovative digital experience has gone from a competitive differentiator, to a competitive necessity – and there’s no simple shortcut for an enterprise to buy this unique digital experience. The innovative part has to be built from the core expertise and function of the business, and that’s difficult when the supply of skilled cloud native development resources will always be constrained.
A skills shortage can impact the job satisfaction of developers and ops engineers, as well as non-developers who might have to deal with the fallout of misaligned delivery. Fortunately, platform engineering organizations and technology stacks can automate most difficult work of designing and delivering on the needs of API-driven microservices environments.
©2023 Intellyx LLC. Intellyx is editorially responsible for the content of this document, and no AI chatbots were used to write it. Image source: kate.sade on Unsplash
