The need for enterprise architecture remains as strong as ever, it’s just that the ground has shifted, and the traditional architecture practice has not kept pace.
For example, the amount of TOGAF and Zachman anyone really needs is a very tiny part of those voluminous methodologies. I have even recently seen people posting on LinkedIn about their TOGAF certification. Please don’t spend your time on that.
The TOGAF layers are important – business, application, and technology – but that’s about it. You just need to know what to decompose from the business to the technology layer, and it’s helpful to take more than one step to do so.
For modern applications I add another layer – the service or API layer, in between the business and application layers. And I would identify a business function or service, not leave the business layer generic.
This is because a business function or service maps well to an API (or interface). And you can design an API better at this level and then map it to the application and technology levels. Focusing TOGAF on API design is an example of adapting it to what you need, rather than simply trying to follow the methodology as is.
So if someone tries to talk you into learning all of TOGAF, just say no, thanks. Just pick and choose the parts you need, and adapt it to what you are doing. For example if you are developing services and APIs you certainly don’t need all of it. Same for Zachman.
One thing I will say, the TOGAF derived ArchiMate diagramming tools can be a big help in visualizing service and API definitions and mapping them to the application and technology layers.
At Citi, for example, when we were designing services with APIs, we just used the basic layered decomposition to identify whether or not a given business function existed in the application layer, or whether there was a gap to be filled by developing a new service, and drew it all using ArchiMate.
Service Oriented Architecture (SOA)
And while we’re on the subject of something that’s being criticized in blog posts and social media, I often am seeing SOA being called out as irrelevant, old fashioned, or just not helpful.
Nothing could be further from the truth. SOA is significantly superior to Object Oriented Design, for example (if that’s your alternative), because it models functions, not things. I know it’s confusing to say so, but an object is not a service, although it can be used to implement one.
SOA maps well and easily to business functions and services. Take the very simple example of a banking application. The bank offers basic services to its customers, such as deposit, withdrawal, transfer, paying a loan, etc. It’s straightforward to map each of these banking services to a software service, design an API for it, and decompose it to the application and technology layers.
An API design starts with the service name – deposit for example – and next identifies the data sent to the service by the service requester (or client) required to execute the service (bank account number, deposit amount, customer number, etc) and identifies any data needed for the reply message, if any (confirmation in this case). Once you have these basics you can iterate from there to complete the API and service design.
In contrast, if you are using object-oriented design, you have to first identify the “thing” or object – in this example the bank account – and then identify the methods on the object (deposit, withdraw, transfer, etc), and then develop the data models for the messages that invoke the methods.
This may not seem like a big difference but it’s an extra step and an indirect step at that. For a simple example it doesn’t create a lot of extra effort, but for a complex system it adds up and can introduce confusion over defining the object versus the service.
The impact of web standards
The web and web standards (HTTP, URL, HTML/XML) have basically replaced complex middleware systems for integrating applications. This results in more of the integration work going into the application space, and less being done by the middleware, since the web standards don’t have many of the features of traditional middleware systems.
This is important to understand when designing modern applications. HTTP by design does not support persistent sessions, the way RMI does for a Java RPC call for example. RMI ties the client and server together in a conversation, which has an advantage when there are multiple consecutive messages sent between them. But then when you are updating the client or the server, both communicating parties have to be updated at the same time.
Not only that but the design of distributed “ilities” (so called) in traditional middleware systems such as transactionality, recoverability, reliability and security also depend on this shared persistent connection. Because HTTP does not have persistent sessions, all communication has to be stateless – i.e. all information needed to progress the conversation between client and server has to be contained within the messages themselves, including any information required for the “ilities.” This results in more of this work being done in the application space, but the web standards are more abstract, making it easier to integrate with any implementation technology.
It’s also important to understand that the cloud was originally developed for web applications, resulting in different architectural patterns than those used in traditional data center applications (e.g. the scale up vs scale out difference). It’s easier to scale out with a stateless communication protocol. But it’s also more difficult to manage database consistency across multiple replicated services.
The point is that you have to take these things into consideration for modern systems based on web standards rather than on proprietary middleware. There are significant differences and they impact the architecture of applications and systems.
Preventing a “Big Ball of Mud”
If you take the opposite view, i.e. removing architecture entirely from a project, you risk ending up with a “big ball of mud” architecture. In other words, with no architecture at all and potentially a big, ungoverned mess.
Architecture is often compared to city planning. Cities need standards for water pipes, street widths, stop signs, traffic lights, train tracks, electricity voltage, and so on. Buildings need standards for lumber width, window sizes, door sizes, electric circuit breakers and plugs, brick dimensions, screws and nails, etc.
If you let everyone just decide on their own way to do anything you’d end up with shanty towns and construction chaos, like cities that don’t have building codes or inspectors.
Of the two banks I worked for, Credit Suisse had by far the more advanced architecture practice. Among other interesting things, they mapped the entire organization to a domain model and used it for governance to ensure new services and APIs followed consistent standards and were fit for purpose.
The enterprise architects at Credit Suisse mapped out a domain and subdomain model for the entire organization, following Domain Driven Design (DDD) principles. As Martin Fowler wrote in that book’s introduction, complexity in software is a direct result of complexity in the organization.
(Diagram is from a public presentation I gave at Carnegie Mellon University, April 2012.)
Credit Suisse was widely recognized at the time as having one of the largest and most advanced SOA implementations in the world. They had about 5,000 services running on the mainframe and distributed systems (Unix and Windows), a sophisticated governance process, and an organization-wide domain model (as shown). They conducted studies proving the multiple benefits of an SOA based approach compared to an EAI or ESB based approach.
An InfoQ article about the Credit Suisse SOA summarizes the first fifteen years, and mentions the volume of services implemented using CORBA initially and then Web services subsequently. This is the environment I worked in.
A major aspect of the Credit Suisse approach is that services are independent of their implementation. This requires approaching the interface from an architectural point of view: in other words, defining the service interface (or API) independently from its implementation.
Today, REST is the lingua franca of service oriented computer-computer interactions, just as the original web standards became for human-computer interaction, and the standard for services APIs (including microservices of course, since microservices essentially represent an evolution of SOA).
Among the most important concepts in DDD is the concept of a bounded context. You can see this in the diagram in the interfaces that cross domain boundaries — interfaces to and from the information bus, in other words. (The message bus itself, by the way, is also a logical construct, implemented using a combination of message queuing, CORBA, SOAP, and REST.)
The bounded context concept is key to microservices as well, since it clearly separates development in one microservice from development in another, and creates contracts that allow applications to be composed of multiple, independently-developed and deployed microservices.
In the Credit Suisse model, bounded contexts separated applications and databases by domain, in other words by department. HR, finance, risk, payments, equities, fixed income – each is assigned to its own domain (the model included subdomains, which are not shown).
Architects and developers working within a domain were free to update and change applications as long as the bounded context interface to other domains didn’t change, or was changed with the agreement of the architects and developers in the other domain.
Bounded contexts for microservice APIs
Microservices are typically packages of one or more related APIs. In the simple bank account example, Account Management is the microservice, and APIs hosted by that microservice service include deposit, withdrawal, funds transfer, loan payment, etc.
Such a grouping of functional APIs deployed as microservices typically depend on other microservices, either for technical aspects such as authentication, logging, or data access, or for additional related services such as those required for initiating an international payment, including sanctions and anti-money laundering checks.
The Account Management microservice communicates with other microservices using the bounded context concept, which means the Account Management team is free to update and change applications and data within the scope of the Account Management microservice as long as they don’t change the bounded context contract through which the Account Management microservice communicates with other microservices. At least not without the agreement of the other microservice teams.
This concept is essential to achieving agility and rapid deployment. One well known example is the “Buy Now” button on an Amazon web page (you can use this to buy my book ;-). As long as the Buy Now microservice team doesn’t change the bounded context interface through which their microservice communicates with that microservices on the web page, they can make any changes they want and deploy any time they want. This is the same story for all the other individual functions on the page, and this is what allows Amazon to push hundreds of production changes to its website daily.
Most people have heard about the famous Jeff Bezos memo requiring all Amazon developers to use services to build applications and systems to improving productivity, reuse, and enable rapid production change. But I’m not sure how many understand the architectural challenges in successfully governing such a strategy. Someone has to make sure that changes to bounded context interfaces don’t cause outages or incidents, and this requires significant discipline.
Deriving microservices from monoliths
When designing microservices, you may often find that the functionality a microservice requires is available in an existing application. While it’s possible to wrap that application with an API and leave it in place, you can achieve additional value by breaking out that functionality into a standalone microservice and using its API as a bounded context that makes the functionality available to other microservices, as shown in the diagram.
This is a good way to build up a library of microservices that you can use in multiple applications, such as a customer validation or credit check microservice.
Because an architectural governance process is essential to ensuring good design and consistent standards, before you start breaking out microservices from monoliths you will want to set the appropriate review standards and design guidelines.
Why target architecture is important
Target architecture embodies your vision and sets your strategic goals into a measurable framework.
As they say, if you don’t know where you are going, you can end up anywhere. A target architecture helps you choose investments that reach your strategic business goals. Not all investments are strategic, of course. Many are tactical necessities: conforming to a regulation, patching bug, or upgrading a software product for example.
But if you have developed a 3-5 year strategic target architecture, you can make a conscious decision about whether or not a given investment helps achieve it. You can also build a roadmap that helps you get from where you are (i.e your current state) to the target state over time. Such a roadmap is helpful in organizational communication and annual planning.
Target architectures were an essential part of the architecture program at Credit Suisse. At Citi we created about twenty of them for different product lines, helping them formalize the definition of the services those product lines wanted to offer their customers over time to increase revenue, and to create a plan for implementing them.
Even once you set a target, however, it’s important to understand you generally cannot go straight toward it.
At Credit Suisse, this was part of what we called “Managed Evolution” (the diagram is taken from the book referenced in the link) — in other words, the opposite of the natural evolution that results in mud balls.
The Managed Evolution process starts with the target architecture. Next are the interface abstractions that encapsulate existing and legacy applications. In other words, APIs that decouple the client or user interface from the API implementation. That way, implementation changes don’t impact the client API as you move from monolithic mainframes to microservices, for example. Again, applying the bounded context principle.
The Managed Evolution process takes into account the fact that not all changes to the IT system directly align to the target state. But like a sailboat tacking against the wind, even if a change takes you part of the way toward the target, you are making measurable progress to the modernization goal.
The Intellyx Take
Enterprise architecture is still important, if not more important than ever in some ways, as the industry moves toward modern systems, microservices, and APIs.
The approach to reducing complexity through Domain Driven Design is one example: DDD is tremendously helpful in dealing with the complexity of microservices. And its “bounded context” concept is more important than ever, since it is key to agile, automated deployment. Strong architectural governance over interface changes is essential to this, especially at scale.
The argument over “enterprise” architecture often misses the point by drawing a contrast between the present and the past, and trying to apply outdated methodologies such as TOGAF and Zachman without avoiding their limitations with respect to modern systems. To be relevant, architecture has to adjust and adapt to the constantly shifting IT landscape. Use TOGAF and Zachman but choose what you need from them, don’t just follow a generic methodology.
Target architectures and Managed Evolution concepts remain very relevant for setting and measuring strategic business goals and modernizing existing systems. As with anything else, of course, you always need to clearly align your work to strategic business goals to demonstrate value, and these architectural practices help you achieve that.
As with everything concerning complex enterprise applications and systems, your mileage may vary. But don’t throw the architecture baby out with the legacy bathwater – adopt and adapt what makes sense for your current environment, modern technologies, and business priorities.
Copyright © Intellyx B.V. Intellyx is an industry analysis and advisory firm focused on enterprise digital transformation. Covering every angle of enterprise IT from mainframes to artificial intelligence, our broad focus across technologies allows business executives and IT professionals to connect the dots among disruptive trends. As of the time of writing, none of the organizations mentioned in this article is an Intellyx customer. No AI was used to write this article. First image credit, Tom Fisk. Second image credit Pixabay. Both from Pexels.
