Brainwave S2-01: Complexity at Scale from Low-Code to Cloud Native w/ WSO2

Intellyx Brainwave – Podcast/vCast for December 14, 2021:

Full transcript of the podcast:

Asanka: Hello, everyone. Welcome to another podcast from WSO2. There are many problems we see in the industry and digital transformation has become the main objective of every organization.

And we see the complexity these projects are creating as well as the organizations are looking at how they can make their developers productive. So we are going to discuss this topic and I have two industry leads our own CTO, Eric Newcomer, and Jason Bloomberg, founder of Intellyx welcome to the show.

So we can start with the topic. And why don’t you kind of set the listeners about this problem. And then we can discuss possible solutions as well as, technology providers, what we can do for this problem. Probably I’ll go to Eric and then come back to you, Jason. Eric, your thoughts on this and how you see this problem as an industry leader for a long time.

Eric: I look at this as an evolution of technology and an evolution of technology terms, we can see certain marks in the evolution of technology. Mainframe—it’s all started with mainframe so of course—and that was a very complex centralized environment place you went to, to do your computing —on the other end of the spectrum is the cloud native computing where the data centers contained vast numbers of smaller computers, stitched together.

And characteristics, so these are sufficiently different that you have to engineer. To get the benefits of each one of your applications. And how to do it is not always clear. And that leads to the, some of the conversations around how much complexity do you need. For example, from the old mainframe environment, when you’re thinking about moving apps from there, or migrating monoliths to microservices, you might have a very complex environment in which you’ve been used to working, but then when you go to microservices and cloud native computing, you have some different paradigms and different patterns, some different simplifications.

Based on, for example, you’re using HTTP protocol, which is a very different thing from MQ series or CORBA in the old days or RPC mechanisms. And how should you adjust your thinking and make sure you can use the right technologies to get the right benefits for your app.

Jason: It’s great you started with mainframes because I think it’s important to understand sort of the history of the complexity of enterprise IT environments.

Because really they’ve been complex all along. It’s not like we had some sort of wondrous old days where things were simple and it’s not like we’re achieving any greater simplicity. Right. We have enormous complexity. We’ve always had enormous complexity. But the complexity has shifted over time.

Right? What, the challenges that a host-based mainframe environment, even though that was very complex in its own way, are different challenges than we had in, the web days of the nineties or the, you know, the rise of cloud native computing, or cloud computing, and then cloud native computing today. So each of these waves of technology innovation brings with it, its own complexity challenges as well as new business priorities.

And I think that’s also an important thing to keep in mind is that it’s not just. We have complexity and we’re trying to deal with it and things are getting worse, not better. And that’s the story it’s really about saying that, you know, we, the enterprise organizations, the businesses, the customers really driving new requirements for enterprise IT and those new requirements for application capabilities, as well as development velocity, are driving changes in the way we deal with complexity. And this balance between velocity and complexity is now becoming one of the primary challenges of modern computing.

Eric: Absolutely. Where do you draw the line on the complexity scale?

Asanka: Yeah. I think an enterprise is in a way, a complex adaptive system.

And when it comes to this experience, that digital experience—internal consumers, as well as external consumers are looking at it, tasks go into the next level, as well as, as Eric started with the technology complexity and Jason, you added about the business complexity—industry is in a totally different level when we come one or two tickets back. With that, I think we know the problem space very well now.

So another area that I would like to touch base on this about now we have this complexity within the organizations and then we see, according to a recent McKinsey report around 87% of organizations are suffering from this developer shortage. So how we connect these two, now we get the complexity from one side, and we had to build the systems. And then there’s a shortage of the developers in the market as well.

So how do you see and how you correlate these two issues at the same time? Probably I will be with Jason first and then come back to Eric.

Jason: Well, looking at the big picture, how did we get here? How do we get in a situation where there’s a shortage of developers? It does tie to how we’ve been dealing with complexity over the decades.

And the way we deal with complexity, sort of from a developer perspective is building layers of abstraction. I mean, we’re still dealing with zeros and ones. We were dealing with zeros and ones when ENIAC first hit the scenes after World War II. And we’re still dealing with zeros and ones. So how do we take this world of zeros and ones —bits on a wire— and build all the enormous complexity and capabilities that enterprise IT exhibits. And it’s through layers of abstraction that are essentially hand coded, whether it’s your underlying assembler or your compiler, or a web abstraction, or cloud native abstractions, there’s many different layers of abstraction.

Each one of which is intended to simplify the usage of the technology for those people that are dealing with it while masking the underlying complexity. The challenge though, is that somebody has to build those layers of abstraction. Somebody has to manage them. Somebody has to ensure that they’re working properly.

And somebody has to think about innovation in the context of those layers of abstraction is how are we going to move to the next generation or whatever of capabilities. And this is really the responsibility now of the developer. So ever since the Dawn of computing creating software, it has been a manual task, right?

It’s something that human beings have to do essentially line by line. And there’s only so fast a human can create lines of code and that’s the fundamental bottleneck. So with the challenge now is, well, how do we accelerate that? How do we either give developers tools to accelerate their work, so it’s not about one line of code after another.

There’s a different way of approaching the software creation process. And then there’s also the question of automation. How much can we automate the software creation process? And does it even make sense to leverage automation in today’s world for creating software? Where is that still something that’s in the future?

Eric: Well, we also have this constant change, which relates to the constant challenge of getting new skills and new developers who understand those new technologies. Touch screen caused a huge wave of re-engineering when that came out the web is causing a huge wave of engineering. When you have web-scale companies that deal with hundreds of millions of users across the globe, what kind of infrastructure do they need, versus those systems that were serving a single company only?

So there’s the constant change in the technology, then constant change in the requirements and a constant change in the environment. That also are part of this challenge because it’s hard for developers and humans generally, to keep up with all these changes that are occurring at a perhaps even faster and faster pace.

The software industry has been for a long time looking for the same kind of sane and scientific methods that are used in manufacturing and other industries to speed development, to speed deployment, to automate things we may be finally starting to get there. There’s a tension between this, you know, as Jason, you talked about it’s an individual activity, is it a craft?

Is it somebody creating a basket, somebody creating a pot as an artistic individualistic activity? Or is it somebody creating an interface that plugs into another interface? That uses a container that gets deployed in Kubernetes, that’s all automated and speeds the process. We are getting to that point where we can also introduce more automation to deal with some of the complexity, but the automation at same time brings its own level of complexity.

How do you deal with that? So it’s one of these cycles that hasn’t ended for sure. And one of the things we’re trying to look at is how do you abstract —provide a syntax for people to express some of these concepts in a language, and abstract that language up into simplified visual and templatized interfaces to provide developer experiences that are more modern that go along with the more modern technology, try to keep up with it.

So it’s a constant battle and constant evolution, but as you know, companies such as ours, continue to try to keep up with this. We continue to try to raise the level of abstraction and provide a developer experience that sort of follows the trends of the industry and helps simplify the problem the best that we can.

Asanka: Yeah, I think we came to interesting points. What I understood by listening to both of you, it’s one, one issue is the lack of developers. And the second issue is a lack of developers with the correct skill set who can work with the current technology as well as deliver the products and the services the market is looking for.

So I think since we came to that level these days, every press and every organization is looking at low-code as a possible solution. So but then again, low-code was there for a long time and it was kind of useful with some organizations in a very productive way. But I think in general there are different feedback about low-code as well. And as an organization, WSO2, we are developing a local platform as well by looking at this problem. So how do you think Low-Code will be a solution? Actually, my question is more to that question. One is how Low-Code will solve the problem as well as is the industry ready with the current low-code platforms available to provide a proper solution for this problem?

Jason: Well low-code has been quite successful over the last few years. It’s getting quite a bit of attention and uptake across various enterprises and the tools and products in the market have been maturing as well. But there’s still a bit of a complexity shell game going on here. Right? I mean, you can give a developer, a Low-Code tool and that gives them a visual environment where they can drag and drop business logic and they can, just put widgets on an interface and build a mobile app really quick or whatever it is they’re trying to do.

But somebody still has to create the algorithms, right? The underlying business logic, somebody still has to figure out the integration logic and low-code tools may try to automate or simplify that. But often they push that complexity off, into hand coding. This is why Low-Code is called low-code and it is differentiated from no code —is that Low-Code allows for the fact that hand coding is still useful for certain capabilities and may actually be required. And the typical situations where you would hand code in the Low-Code environment might be custom integrations or custom user interface, widgets, or a business logic that is overly complex or too complex for the low-code environment, which typically has business logic capabilities, similar to Excel formula. If you have to create business logic more complicated than Excel formulas, then often you have to drop into hand coding.

So what we have now is this challenge where Low-Code is an important part of the story. It can accelerate professional developers work. It can facilitate collaboration among the developers as well. Business stakeholders and designers and other people who are important for the software effort. But we still have this need for hand coding. And so the challenge now is how do we support the hand coding environment or what I like to call pro code as different or differentiated from low-code and no-code in a way that centered on resolving some of these complexity issues.

So we have this balance between no code, which is really more for business users that can build simple applications. Those tools are becoming more aligned with Low-Code, which is for professional developers, simplify as much of the work they do and fosters collaboration, but still requires some hand coding.

And we have pro code tools, which are essentially an evolution of the IDEs of earlier generations that are now really designed for professional developers who need to leverage or want to leverage hand coding skills and capabilities, but want to leverage many of the abstractions that low-code brings to the table.

There’s no reason to hand-code a workflow if you have a workflow design tool at your disposal. It’s still a question of fostering a simplified environment that empowers professional developers to do the work they’re doing. So here we have this interesting balance. And when we get to talking about cloud native computing, where we’re thinking about a different architecture for applications, it becomes especially important because it’s difficult to create microservices in a containerized environment solely with the Low-Code tool, right?

Because there’s more considerations there than a Pro-Code tool that is a cloud native and designed for that purpose would be better suited for it, for that kind of situation.

Eric: Well, it’s quite correct. This has happened before we’ve seen this over time in the industry, 4GLs and visual programming languages, and we’re seeing a new wave of Low-Code.

Now, part of it is because of the new class of problems that are facing us with the new technologies, the cloud native, the software as a service, APIs that need to be put together. Right. Years ago, it was all about creating individual applications, automating manual processes. And now it’s about tools that work more and more with existing programs, existing APIs and integrations. A big driver for this has been, you know, I worked at Citi for a long time before coming to WSO2 about a year ago.

And I saw the dynamic there where the IT department was really unable to keep up with the demands of the business team. Toward new and better functionality, especially in the digitization area where the teams were really struggling to keep the older systems afloat while at the same time, trying to learn how to do newer applications that do applications that were consumer facing as opposed to business facing.

There were a lot of challenges, businesses were getting impatient and they were looking for tools to use themselves to address some of these problems. They were hiring people to do some of these coding tasks. To bridge the gap between their goals for digitization and the capabilities of the IT department to deliver that.

So I think that’s a really good reason for the wave. And I think, you know, Jason, as you, I think you alluded to, it’s starting to specialize, so we’re seeing a class of tools that are oriented toward business applications, Excel on steroids and so forth. We’re seeing a class of tools that are trying to automate repetitive tasks for robotic process automation.

And we’re seeing a class of tools evolve around cloud native development. That is so critical for the digital applications to succeed. Customers expect that digital experience you get from connecting to the cloud, always- on, scalable, agile and these kinds of tools are becoming more important as companies need enablers for those digital applications that will provide the integrations, provide the API consolidations yeah.

The microservices. And as you said, within that context of enablers or somethings that need a code level, and we’re, you know, we’re really focused on providing code that meets the programming model of the web for microservices. So it makes it very easy to do it in code and also easy to abstract it into Low-Code and still provide the capability to have no code templates to simplify the developments.

But here it’s also a specialization. Really enablement being the things that people need on the backend to make those digital applications successful on the front end. So I think we’re seeing a specialization and our part of that specialization is really on the backend enablers

Asanka: I think now there are two things we discuss.

One is about the increasing the developer productivity —pro developer productivity by using Low-Code. And then have a combination of low-code and pro-code. That is one side. And then Eric mentioned about people like citizen developers and then semi- technical users getting into development.

But I think the main problem with that what we see in the industry, those two areas, the pro code development and low-code development happening in two silos and those are not connected. And that is creating a lot of issues like low-code creating shadow applications or shadow IT applications.

And then pro code is happening at the separate level. I think the solution required the connectivity in between these two. I think that’s where the democratization of development is coming into the picture.

So how do you see this as the reality, because there can be resistance coming from the Pro-Code teams while the citizen developers coming into the application development.

That is one side of the story. So other side of the story, the business users might be not that confident and comfortable to getting into these projects that are providing digital experiences. So how do you see this as well as what we can do as developers, as well as IT leaders to connect these two silos and then have a productive environment inside these organizations?

Jason: Yeah, I think there’s well, there’s two there. I mean, there’s two sides to that question. I mean, one is the collaboration, you know, we have talked about, do you know, a Pro-Code team and a Low-Code team and that collaboration, I mean, that’s, that’s one of the core benefits of today’s Low-Code platforms is that they foster collaboration, right.

It gives you a route to the hand coding part of the Low-Code equation. It also provides a more visual collaborative environment that helps bring in stakeholders and other people, the graphic designers to user interface developers that are every bit as important as the software engineers for the success of the software project.

But I think there’s another story here. And this ties back to what Eric was saying about how WSO2 is focusing on backend enablement as well as the front end. A lot of the conversation in the market today around Low-Code is on the front end, right? Is it building applications that have user interfaces you want to put in front of customers and employees?

And of course, that’s very important. But there’s all this backend stuff that is also important. And in the context of cloud native computing it’s undergoing a real paradigm shift in how we’re architecting and deploying the backend part of the software. Right. We’re going to create microservices, which follow modularity patterns.

We’ve been following for over 20 years, but now leverage modularity, in a finer grained way. We are putting those in containers are they’re inherently ephemeral. So we have to deal with state information in a different way. And we have to focus on how we’re connecting microservices to each other, as well as to things outside of the container environment.

And so we’re focusing on integration in new and different ways, whether it’s leveraging service meshes, or other technologies for connecting a microservices together, or a greater focus on. API enablement and management in order to support integration of microservices with other things.

This is a whole different way of thinking about software, where yes, in the web days we were thinking about modular software. We were doing object-oriented programming, but it was a different context for modularity, for integration and for supporting rapid development and ephemeral software at scale.

So this is a very important part and most low-code tools don’t do a good job of dealing with this underlying architectural challenges because they are focused on front end issues primarily. Where a WSO2, you know I don’t mean here to plug you guys, but you’re focusing not only on the front end, you have the Low-Code tools and the API tools, but you also have the backend technologies as well.

And I think it’s really important to have this front to back, end to end context because organizations aren’t going to be able to implement modern cloud native applications, unless they take that approach.

Eric: We were talking with one of our customers earlier this week. And they talked about their frustration with some of the low-code tools they had brought on board.

It’s being unable to deal with some of these complexities. So there is often this sort of a cliff where you, you reach an end of the capabilities of a low-code tool, and then you have to go into a different environment to complete the application work. And we’re trying to avoid that by providing this all-in-one box, both the Low-Code and the pro-code capabilities, and perhaps more importantly for us and for the vision of where we see the industry going and Low-Code tools going.

Is that the abstractions that we have for dealing with how you program need to align with how you think about things. And Jason, you touched on a lot of important issues about how people have to think about things very differently now, when doing microservices in the cloud than we did, when we were doing service-based architectures in monolithic, you know, sort of enterprise -only environments.

You know, everything is out on the web now, the web programming model is coming into the enterprise with all the new web pioneers, what they’re doing and how they’re doing microservices and microservices designs and architectures.

Just as we’re seeing HTTP as an internal integration mechanism now from a lot of these companies is a big shift from the types of technologies that integration was done within the past.

Restful APIs have become the norm, which wasn’t the case, you know, in the enterprise at all. So we need to have —our feeling is we need to have a different set of language abstractions that recognize and allow people to think about this difference in a way in which it’s easier for them to translate the mental picture of what they want to do into the software picture and the ones and zeros of how it gets done.

So that’s where we start. And then we build on that instead of abstractions that are in the Ballerina language, which is oriented toward cloud native computing. And from there, I start building it up to having a low-code tool with graphical representation, no code tools with templates for plug-and-play, a plugin template to drag and drop activities, while still preserving the capability to work at that syntax level whenever you need to.

And I think a lot of it has to do with this ability to create a new foundation that aligns with the new technology. So new ways of doing things and new concepts that allow people to reason. In a more natural way through that language and the ability of that language to present itself in abstract ways that fit the low-code and no-code paradigms to really provide the help everybody needs.

Asanka: Yeah. Great. So I think this has to be an ongoing conversation. Unfortunately, we are running out of time. So thanks Jason and Eric for joining this conversation. And I hope it is very important for our listeners as well. Thanks everybody for joining. And we will come back with another topic in the future and keep subscribing to our channel.

Thank you very much.

Eric: Thanks a lot Asanka. Thanks everyone. Thanks, Jason.

Jason: Thank you.

Eric: Bye for now.

 

©2021 Intellyx LLC. At the time of publishing, WSO2 is an Intellyx subscriber. All dialogue in this program represents the expressed opinions of the hosts and guests, and are not necessarily the official position of Intellyx, or any company mentioned or included in this podcast audio or video.