Agile Software Architecture: Clean Architecture and Evolutionary Architecture

What is an Agile software architecture? How to design a good architecture and implement the clean architecture? How to go beyond, embrace emergent design and implement an intentional architecture with Evolutionary Architecture?

What is Software Architecture?

In 2000, IEEE 1471-2000 defined the first standard about Software Architecture in a document that they called “Recommended Practice for Architecture Description of Software-Intensive Systems”.

Moreover, Martin Fowler explains Software Architecture more pragmatically as “things that people perceive as hard to change.” As a consequence, as Ralph Johnson turns it:

Furthermore, Martin Fowler elaborates that more than the document that Architecture generates, what is important is the common understanding in the teams on how all the pieces of the software fit together. At last, he stresses that it is quite useful to have good examples in the code, on how to properly implement the target architecture, as developers common refer to state of art code, to understand Architecture guidelines.

Illustration of Software Architecture: the Clean Architecture

There are already good technics to define Architecture, especially in the concern to make it modular, but developer do not systematically use them. For instance, Robert C. Martin, aka Uncle Bob, the creator of the approach Clean Code, developed the approach of the Clean Architecture by making a synthesis of the Architecture guidelines like:

• The Hexagonal Architecture (aka Ports and Adapters) by Alistair Cockburn.
• The Onion Architecture by Jeffrey Palermo.
• The DCI (Data, context and interaction) from James Coplien and Trygve Reenskaug.
• At last, the BCE (Boundary, Control and Entity) by Ivar Jacobson.

Principles of the Clean Architecture

The first pillar is to support independence between components whatever their nature:

• Frameworks: the architecture does not depend on frameworks, software or libraries.
• User Interface (UI): you can connect different types of UI or switch technology of UI without effort. For instance, plug a Web UI and a console UI on the same software.
• Database: there is no business intelligence in the DB management and you can switch DB easily depending on the needs, the size of the data and the number of transactions.
• External Agency: the software is disconnected from the outside world.

The second pillar is test: the Clean Architecture is testable and does not require all the components at the same time to run tests.

Layers of the Clean Architecture

Robert C. Martin defines the Clean Architecture as onion layers staring from the core:

• Firstly at the center, enterprise business rules: the rules that play for the whole enterprise. Some people advocate that it is relevant to add a sub-layer with domains as used in Domain Driven Design. Indeed, Architecture naturally splits by domain of knowledge where there is a consistency in business data and rules.
• Secondly, application business rules: the rules that play at the level of the application.
• Thirdly, the interface adapters such as controllers, gateways and presenters.
• At last, the frameworks and adapters/drivers like Web, UI, External interfaces, Database and devices.

Rules of the Clean Architecture

There are rules that manage the relationships between the layers of the Clean Architecture:

• The Dependency Rule: there can be dependencies only from one outside layer to the inside layer. In other words, the inside layers do not know the context of the outside layers.
• The communication is direct between two layers. To put it differently, the communication cannot jump a layer.
• At last, communication between layers is through data structures that are simple and isolated.

Benefits of the Clean Architecture

The benefits are the same as Continuous Delivery practices coming with Craftsmanship, DevOps:

• Easier and faster development of new features.
• Cheaper overhead cost for software development.
• Less maintenance cost of code with less need of refactoring as the components are independent.
• Better quality of code with the tests made easier to conduct.

The metaphor of Building Architecture

The Software Architecture notion roots in the metaphor of Building Architecture. But it is not a good metaphor for software development. Surely, most decisions about buildings are hard to change. Nevertheless, software does not have the same physical limits like buildings. Software has limitations but they are around imagination or organization. In other words, software is limited by developers’ limits, not physical limits.

So a better image for Software Architecture than building, is city planning: provide and support a vision, a big picture, to enforce a consistency across and between all the pieces of software.

What is a good software architecture?

Martin Fowler stresses 3 characteristics of a good software architecture.

Minimal

Keep your Software Architecture as simple as possible but reducing things that are irreversible because they are hard to change.

Couple questions may help:

• Where are the things that are hard to change and how do we make them easier to change?
• How do we prevent things from becoming part of the Software Architecture? Is there a way to make them easier to change then reversible?

Evolutive

Software Architecture should by be open to evolution with the mechanisms to do so.

Connected to development

As mentioned above, Software Architecture is a mean to limit effort and enforce consistency. In other words, the purpose of Software Architecture is to support Software Development. To do so Software Architecture has to be closely connected with development. Clearly it is not a separated activity from development, on the contrary. This can be achieve with:

• All the developers contribute to Architecture, even if guided by a senior profile in the team playing the role of Architect. They should contribute to the vision, then as the result they will feel responsible for it. Of course then should also have time to invest in architecture matters.
• The person playing the role of Architect should maximize team involvement, supporting and coaching them, but limiting his/her direct contribution. For instance, by reviewing code and pairing with developers to make them learn.

Emergent Design and Intentional Architecture

What Martin Fowler calls a good software architecture is close to emergent design and intentional architecture:

• Big Picture for all: the Intentional Architecture provides the teams with the big picture where to go. As a result, architecture acts as a glue and guides and aligns their parallel and independent actions. Surely, this does not prevent coordination but architecture acts as a primary mechanisms of alignment.
• Evolution through incremental change, for minimal architecture, driven by the development: Emergent Design analyzes the needs from development and extends the architecture just enough to implement and validate the next increment of code.

Evolutionary Architecture

Applying properly architecture is already a good start. But there is a need for more with the changes in the business environment:

• Pace for feature delivery keeps increasing.
• The business life time of products continue decreasing.
• And it is harder and harder to predict the future.

This is why it is valuable to go beyond good software architecture with Evolutionary Architecture.

Rebecca Parsons, Neal Ford and Patrick Kua wrote a book named Evolutionary Architecture that describes this approach and they delivered several conferences on this topic.

Introduction to Evolutionary Architecture

Evolutionary Architecture structures over 3 elements.

Incremental

Architecture evolves through incremental releases. Furthermore, each release is the sum of small refactoring managed independently. Those changes apply repeatably and reliably whatever the cycle time, as they are totally automated. In other words, the system remains secure and safe as it evolves. Really, all this is Continuous Delivery that is a pre-requisite to Evolutionary Architecture.

Guided

As we have seen above, architecture plays as a big picture. In Evolutionary Architecture, this vision is guided by what the authors call fitness functions. So, a fitness function is an explicit mechanisms that measures how close a given solution matches to a particular goal, like performance, security or scalability. To illustrate, you can implement a fitness function through metrics, tests or processes.

Fitness functions fall in 3 categories:

• Atomic, when they inspect a single attribute like that components have no dependencies.
• Holistic, when they cover a whole scope for a given dimension. For instance, when they check performance for a total transaction, make sure that it is below a certain time, and monitor the systemic behavior.
• Continuous, when they continuously and proactively enforce the architecture. For example, Netflix continuously test the resilience of their system with their simian army, also called the chaos monkey, where they randomly trigger failures to watch if the system properly faces them.

Multiple dimensions

There are several dimensions that play for the Architecture.

Capabilities

Often, you need to trade off between capabilities. To illustrate, when security increases, performance decreases as a result.

Here are couple capabilities to give you an idea of the trade-offs:

• Standard
• Consistent
• Extendable
• Customizable
• Flexible
• Performant
• Scalable
• Robust
• Resilient
• Secure
• Cost effective

Technics

Technics gathers libraries, tools, frameworks and platforms. And the concern is about all their lifecycle: introduction, configuration, upgrade, replacement, up to removal.

Domains

Domains (of knowledge) are also dimensions and typically come with the use of micro services focused per domain.

Evolutionary Architecture principles

Last responsible moment

Delay your decision as long as you possible without getting late. This makes it possible to maximize information to take the best decision. At the same time, this minimizes the architecture debt, as the longer you wait, the more chance you have your decision will last.

Design and build Architecture and Code for evolvability

Architecture

• Split your architecture in a way that makes sense to the business, for instance, by business area or domain. This also supports data ownership.
• Decide coupling taking into account all parameters. For example, coupling may be decided to increase performance.
• Have light documentation and tooling.

Code

• Find your weak spots and focus your efforts there. In short, those are the code areas that make people nervous when they are changed.
• Monitor your progress with the right metrics: measuring what is at stake but easy to understand.

Postel’s law

Jon Postel expressed what became this law when he defined an early specification of TCP.

Indeed, as soon as you start sending something, people expect that you are going to continue. So be conservative and persistent with what you send.
Yet, the approach with what you receive is different, as you want to be open to what you receive to the limit of the security. In addition, make controls only when you need for the transactions using the data, not more.

Conway’s law

This means that an organization designs a system that reflects its communication structures. In other words, a poor communication between two teams will result in a poor, complex and inefficient integration between the two pieces of software built respectively by those teams. To illustrate, silos typically result in poor integration.

The inverse Conway maneuver stresses that you can design the software architecture by adapting the organization to the expected Architecture. Therefore, if you do not want your software to look like your organization, change your organization.

Architect for testability

Testing is essential to go for an Evolutionary Architecture. Surely, without the safety net of testing automation, you cannot safely implement Evolutionary Architecture.

In addition, the experience demonstrates that testability results in system with a good architecture, in other words, an architecture simple to understand and easy to evolve.

More tips:

• Make sure that tests make sense to the business with components that the business understand.
• Ensure your testing testing coverage.

Evolutionary Architecture technics

Continuous Delivery

Automation is part of Continuous Delivery, even if there is much more in it. In this blog, Continuous Delivery covers both Craftsmanship, the state of art of coding and DevOps, that focuses on the relationship between Dev and Ops, and the automation of the underlying process.

Automation is key to Evolutionary Architecture:

• Craftsmanship covers:
  • Firstly, the automation of builds and deployments to enable Continuous Integration.
  • Secondly, the automation of testing.
• DevOps takes care of the automation of environments and configurations.

As a result, deployment should become a non-event. But it is not because you can release at any time that you need to. Surely, it depends on the needs of the business.

DB refactoring

The approach is to split big changes into a series of small changes. This reduces the complexity of a singe change. Clearly, with big changes where there is a cocktail effect, the complexity of the unitary changes combine exponentially. In this technic, developers integrate continuously database changes.

Each change is combine:

• The refactoring of the database structure.
• The migration of the data.
• The change of access code to the database.

All changes are managed with code following the approach of Continuous Delivery. Therefore, these pieces of code are managed like every other pieces of code in CD: with versioning.

Specifically for database refactoring and migration, tools like Liquibase and Active Record Migrations can support you.

At last, developers have all their own database and apply their changes in the various environments, from development to production. Once again, they perform all this through automation.

Contract testing

A contract structures what one system expects from another. This is also referred to as assumptions.

First step, is to capture the assumptions by making explicit the contract. Then conduct acceptance tests between systems, on their interfaces, to make sure that they meet their contracts. Truly, good enforcement of contract maximizes independency between systems, so they can focus on their core activity without wasting time to manage other systems.

Of course, this approach comes in pair with the Postel’s law, as you want to keep the exchanges to the minimum needed.

Choreography

Rebecca Parsons introduces the image of choreography to support the idea of empowerment of teams guided by a shared vision, as opposed to orchestration.

In orchestration, like in an orchestra, you have a conductor and everybody is paying attention to him or her. If the conductor stops, then everybody stop, as nobody knows what to do. Truly, this is a single point of failure.

On the contrary, in choreography, like in a dance performance, the choreographer sets the vision, in our case the big picture of the target architecture. Furthermore, outcomes are clearly defined, they are scripted. Clearly, each individual knows what they are supposed to do. As a result, individuals are able to perform to the vision without a conductor. This is a typical illustration of a distribution of authority. But this approach is not perfect either, as with more possible interactions, come more potential failures. Yet, in case of a problem or an unexpected event, contributors can improvise as the have the vision.

Other practices that support the Evolutionary Architecture

Technical practices

• Adopt the appropriate coupling based on the context.
• Use ports and adapters.
• Leverage toggle features for flexibility of activation. This can also be used when deploying shared services across applications. Each application can activate the new feature when they want. Then, the old feature is disactivated when not used any longer by any application. In short, this makes it possible to all domains evolve at their rhythm.
• Understand and monitor the various forms of technical debt.
• At last, find out and monitor reuses.

Domain

• Match the architecture to the business structure.
• Decentralize the architecture governance.

Agile

Agile is essential to move to Evolutionary Architecture, especially thanks to:

• Cross functional teams with all skills for build and run in the team.
• Experiment enabled to support the incremental and discovery approach.
• Switch from projects to products to enable ownership, including for architecture.

Where to start with Evolutionary Architecture?

When you have a legacy system, you make ask yourself where to start. Considering value and speed of change is of a help.

Focus on where there is more value and where the speed of change of the code is higher. This is where you will get more from Evolutionary Architecture. Indeed, it requires expertise and discipline and does not come without effort.

What’s next? Learn more about in Continuous Delivery, Agile and Agile at Scale

• Craftsmanship
• DevOps
• There are also plenty of other materials around Agile and Agile at Scale. In addition, here is my post to have an introduction to Agile at Scale.

Do you want to learn more about Agile Software Architecture, Clean Architecture and Evolutionary Architecture? Here are some valuable references

Software Architecture

• Overview and definition about Software Architecture.
• Capabilities.

Clean Architecture

• Post from Robert C. Martin.
• The Django Clean Architecture.

Introduction to good Software Architecture by Martin Fowler

• Video
• Post
• Document

Emergent Design and Intentional Architecture

• Post from Ron Jeffries.
• Post from DZone.

Evolutionary Architecture

• Post from Martin Fowler.
• First Video from Neal Ford.
• Second Video from Rebecca Parsons.
• Third Video from Patrick Kua.