Reviewing software architecture

The word architecture is old. Its origin traces back to times when people used to build things with rudimentary tools, often with their own hands. Yet, each generation repeatedly overcame the limitations of its era and constructed magnificent buildings that stand to this day. Take a look at the Florence Cathedral and its dome designed by Filippo Brunelleschi – what an excellent example of architecture!

Architects are more than just ordinary builders who build things without much thinking. Quite the opposite; they are the ones who care the most about aesthetics, underlying structures, and design principles. Sometimes, they play a fundamental role by pushing the limits of what is possible to do with the resources at hand. The Florence Cathedral, as has already been mentioned, proves that point.

I’ll not take this analogy too far because software is not like a physical building. And although there are some similarities between building and software architects, the latter differs considerably because of the living and evolving nature of their software craft. But we can agree that both share the same goal: to build things right.

This goal helps us understand what software architecture is. If we’re aiming to build not just working but also easily maintainable and well-structured software, it can even be considered to a certain degree as a piece of art because of the care and attention to detail we employ to build it. So, we can take this activity of building easily maintainable and well-structured software as a noble definition of software architecture.

It’s also important to state that a software architect’s role should not only be constrained to deciding how things should be made. As in the Florence Cathedral example, where Filippo Brunelleschi himself helped to afix bricks to the building to prove his ideas were sound, a software architect, in the same vein, should soil their hands to prove their architecture is good.

Software architecture should not be the fruit of one person’s mind. Although there are a few who urge others to pursue a path of technical excellence by providing guidance and establishing the foundations, for an architecture to evolve and mature, it’s necessary to utilize the collaboration and experience of everyone involved in the effort to improve the software quality.

What follows is a discussion around the technical and organizational challenges we may encounter in our journey to create and evolve a software architecture to help us tackle the threat of chaos and indomitable complexity.

Making decisions

All this discussion around software architecture concerns is relevant because we may undermine our capability to maintain and evolve software in the long run if we ignore those concerns. Of course, there are situations where we’re not so ambitious about how sophisticated, maintainable, and feature-rich our software will be. It may not be worth all the time and effort to build things in the right way for such situations because what’s needed is working software delivered as fast as possible. In the end, it’s a matter of priorities. But we should be cautious not to fall into the trap that we can fix things later. Sometimes we may have the money to do so but sometimes we may not. Bad decisions at the beginning of a project can cost us a high price in the future.

The decisions we take regarding code structure and software architecture lead us to what is called internal quality. The degree to which software code is well organized and maintainable corresponds to the internal quality. On the other hand, the value perception about how valuable and good software can be from a user’s perspective corresponds to the external quality. Internal and external quality are not directly connected. It’s not difficult to find useful software with a messy code base.

The effort spent on internal quality should be seen as an investment where the return is not immediate and visible to the user. The investment return comes as the software evolves. The value is perceived by constantly adding changes to the software without increasing the time and money required to add such changes, as the following pseudo-graph depicts:

Figure 1.1 – Pseudo-graph showing the impact of changes

But how can we make the right decisions? That’s a trick question because we often don’t have enough information to assist in a decision-making process that will lead to software architecture that best meets the business needs. Sometimes, even the users don’t know completely what they want, leading to new or changed requirements as the project evolves. We tackle such unpredictability by using a software architecture that helps us add changes in a sustainable way to ensure the code base grows without increased complexity and decreased maintainability.

The ability to quickly introduce changes is a major concern in software design, but we should be cautious about how much time we spend thinking about it. If we spend too much time designing, we may end up with an overengineered and possibly overpriced solution. On the other hand, if we ignore or do not reflect enough on design concerns, we may end up with a complex and hard-to-maintain solution. As pointed out by Extreme Programming Explained: Embrace Change, the resources spent on design efforts should match a system’s need to handle changes at an acceptable pace and cost.

In the end, we want the flexibility of adding new stuff while keeping complexity under control. With that in mind, this book is concerned with software architecture ideas that allow us to handle software design decisions to meet changing business needs. The hexagonal architecture helps us to build change-tolerant systems to support those needs.

The invisible things

Software development is not a trivial activity. It demands considerable effort to become competent in any programming language, and even greater effort to use that skill to build software that generates profit. Surprisingly, sometimes it may not be enough to just make profitable software.

When we talk about profitable software, we’re talking about software that solves real-world problems. Or, in the context of large enterprises, to speak more precisely, we mean software that meets business needs. Anyone who has worked in a large enterprise will know that the client generally doesn’t want to know how the software is built. They are interested in what they can see: working software meeting business expectations. After all, that’s what pays the bills at the end of the day.

But the things that clients cannot see also have some importance. Such things are known as non-functional requirements. They are things related to security, maintainability, operability, scalability, reliability and other capabilities. If adequate care is not taken, those things unseen from the client’s perspective can compromise the whole purpose of the software. That compromise can occur subtly and gradually, giving origin to several problems, including technical debt.

I’ve mentioned previously that software architecture is about doing things right. So, it means that among its concerns, we should include both unseen and seen things. For things that are seen by the client, it’s essential to deeply understand the problem domain. That’s where techniques such as domain-driven design can help us approach the problem in a way that allows us to structure the software in a form that makes sense not only for programmers but also for everyone involved in the problem domain. Domain-driven design also plays a key role in shaping the unseen part by cohesively defining the underlying structures that will allow us to solve client needs and doing that in a well-structured and maintainable manner.

Technical debt

Coined by Ward Cunningham, technical debt is a term used to describe how much unnecessary complexity exists in software code. Such unnecessary complexity may also be referred to as cruft – that is, the difference between the current code and how it would ideally be. We’ll see next how technical debt can appear in a software project.

Developing software that just works is one thing. You assemble code in a way you think is adequate to meet business needs, and then package and deploy it to production. In production, your software meets clients’ expectations, so everything is fine and life goes on. Sometime later, another developer comes in to add new features to that software you created. Like you, this developer assembles code in a way they think is adequate to meet business needs, but there are things in your code this developer doesn’t clearly understand. Hence, they add elements to the software in a slightly different manner than you would. The software makes its way into production, and the customer is satisfied. So, the cycle repeats.

Software working as expected is what we can clearly see in the previous scenario. But what we cannot see so clearly is that the lack of common ground defining how features should be added or modified leaves a gap that every developer will try to fill whenever they do not know how to handle such changes. This very gap leaves space for the growth of things such as technical debt.

Reality very often pushes us to situations where we just cannot avoid technical debt. Tight schedules, poor planning, unskilled people, and, of course, a lack of software architecture are some of the factors that can contribute to the creation of technical debt. Needless to say, we should not believe that the enforcement of software architecture will magically solve all our technical debt problems. Far from that; here, we’re just tackling one facet of the problem. All other technical debt factors will remain and can actually undermine our efforts to build maintainable software.

Vicious cycle

Financial debts tend to continue to grow regardless if you don’t pay them. Also, the bank and authorities can come after you and your assets if you don’t pay those debts in time. Contrary to its financial counterpart, technical debt doesn’t necessarily grow if you don’t pay it. What determines its growth, though, is the rate and nature of software changes. Based on that, we can assume that frequent and complex changes have a higher potential to increase technical debt.

You always have the prerogative not to pay technical debt – sometimes that’s the best choice depending on the circumstance – but you diminish your capacity to change the software as you do so. With higher technical debt rates, the code becomes more and more unmanageable, causing developers to either avoid touching the code at all or find awkward workarounds to solve the issues.

I believe most of us will have had at least once the unpleasant experience of maintaining brittle, insanely complex systems. In such scenarios, instead of spending time working on things that are valuable to the software, we spend more time fighting technical debt to open space to introduce new features. If we don’t keep the technical debt under control, one day, it will not be worth adding new features to the technical debt-overloaded system. That’s when people decide to abandon applications, start a new one, and repeat the cycle. So, the effort required to tackle technical debt should be considered worth it to break that cycle.

It’s not for everyone

This zest for quality and correctness that emerges from any serious architectural undertaking is not always present, There are scenarios where the most profit-driving software in a company is an absolute big ball of mud. It’s software that has grown without any sense of order and is complicated to understand and maintain. Developers who dare to tackle the complexity posed by this kind of system are like warriors fighting a hydra. The refactoring effort required to impose any order in such complexity is sometimes not worth it.

The big ball of mud is not the only problem. There are also cultural and organizational factors that can undermine any software architecture effort. Very often, I’ve stumbled upon teammates who simply didn’t care about architectural principles. The least-effort path to deliver code into production is the norm to be followed in their minds. It’s not hard to find this kind of person in projects with a high turnaround of developers. Because there is no sense of ownership in terms of quality and high standards, there is no incentive to produce high-quality code.

Pushing the discipline to follow a software architecture is hard. Both the technical team and management should be aligned on the advantages and implications of following such discipline. It’s important to understand that spending more time upfront on dealing with technical aspects that don’t add much value in terms of customer features may pay dividends in the long term. All the effort is paid back with more maintainable software, relieving developers who no longer need to fight hydras and managers who are now better positioned to meet business deadlines.

Before trying to promote, let alone enforce, any software architecture principle, it is advisable to assess the circumstances to make sure there are neither cultural nor organizational factors playing against the attitude of a few trying to meet or raise the bar to better-developed systems.

Monolithic or distributed

There is a recurring discussion in the software community about the organization of a system’s components and responsibilities. In the past, where expensive computing resources and network bandwidth were the problems that influenced the software architecture, developers tended to group plenty of responsibilities in a single software unit to optimize resource usage and avoid the network overhead that would occur in a distributed environment. But there is a tenuous line separating a maintainable and cohesive monolith from an entangled and hard-to-maintain one.

The crossing of such a line is a red flag showing the system has accumulated so many responsibilities and has become so complex to maintain that any change poses a severe risk of breaking down the entire software. I’m not saying that every monolithic that grows becomes a mess. I’m trying to convey that the accumulation of responsibilities can cause serious problems in a monolithic system when such responsibility aggregation is not handled with care. Apart from this responsibility issue, it’s also equally important to make sure the software is easy to develop, test, and deploy. If the software is too large, developers may have difficulty trying to run and test it locally. It can also have a serious impact on continuous integration pipelines, impacting the compiling, testing, and deployment stages of such pipelines, ultimately compromising the feedback loop that is so crucial in a DevOps context.

On the other hand, if we know when a system accumulates sufficient responsibility, we can rethink the overall software architecture and break down the large monolithic into smaller and more manageable, sometimes autonomous, software components that are often isolated in their own runtime environments. This approach got strong adoption with Service-Oriented Architecture (SOA) and then with what can be considered its evolution: the microservice architecture. Both SOA and microservices can be considered different flavors of distributed systems. Microservice architecture, in particular, is made possible mainly because computing and network resources are not as expensive as they used to be, bringing lots of benefits related to strong decoupling and faster software delivery. However, this comes with costs because earlier we had to deal with complexity in just one place, whereas now the challenge is dealing with complexity scattered around multiple services in the network.

This book proposes hexagonal architecture ideas that you can apply to monolithic and distributed systems. With monolithic, you may have the application being consumed by a frontend and, at the same time, consuming data from a database or other data sources. The hexagonal approach can help us develop a more change-tolerant monolithic system, testable even without the frontend and the database. The following diagram illustrates a common monolithic system:

Figure 1.2 – Hexagonal architecture with a monolithic system

In distributed systems, we may be dealing with lots of different technologies. The hexagonal architecture shines in these scenarios because its ports and adapters allow the software to deal with constant technology changes. The following diagram shows a typical microservice architecture where we could apply hexagonal principles:

Figure 1.3 – Hexagonal architecture with a microservices system

One of the great advantages of microservice architecture is that we can use different technologies and programming languages to compose the whole system. We can develop a frontend application using JavaScript, some APIs with Java, and a data processing application with Python. Hexagonal architecture can help us in this kind of heterogeneous technological scenario.

Now that we’re aware of some of the problems related to software architecture, we’re in a better position to explore possible solutions to mitigate those issues. To help us in that effort, let’s start by looking into the fundamentals of hexagonal architecture.