Problem solving for the big picture

Suppose you're planning a trip from New York to Los Angeles. There are two major aspects that you need to keep in mind:

• What do you need to do before starting the trip?
• What do you need to do during the trip to ensure that you stay on the right track?

Generally, there are two extreme options in planning for such a trip:

• Take your car and start driving. Figure things out along the way.
• Make a very detailed plan—figure out the route, note down the directions at every junction, plan for contingencies such as a flat tire, plan where you're going to stop, and so on.

The first scenario allows you to execute fast. The problem is that, more likely than not, your trip will be very adventurous. Most likely, your route will not be optimal. If you want to update your friend in LA on when you'll be reaching the destination, your estimates will vary wildly based on your current situation. Without a long-term plan, planning for an outcome in the future is best effort.

But the other extreme is also fraught with pitfalls. Every objective has time constraints, and spending time over-analyzing something might mean that you miss the bus (or car). More frequently, if you give these directions to someone else and, at a juncture, reality turns out not to be what you predicted, then the driver is left with little room to improvise.

Extending the analogy, the architecture of a software product is the plan for the journey of building a product that meets the requirements of the customers and other stakeholders, including the developers themselves!

Writing code for computers to understand and use to solve a problem has become easy. Modern tools do all of the heavy lifting for us. They suggest syntax, they optimize instructions, and they even correct some of our mistakes. But writing code that can be understood by other developers, works within multiple constraints, and evolves with changing requirements is an extremely hard thing to do.

Architecture is the shape given to a system by those who build it. Shape essentially means the constituent components, the arrangement of those components, and the ways in which those components communicate with each other. The purpose of that shape is to facilitate the development, deployment, operation, and maintenance of the software system contained within it. In today's world of ever changing requirements, building a platform on which we can execute quickly and effectively is the key to success.

The role of the architect

An architect is not a title or a rank; it's a role. The primary responsibility of the architect is to define a blueprint for what needs to be built and ensure that the rest of the team has sufficient details to get the job done. The architect guides the rest of the team toward this design during execution, while managing constant dialogues with all of the stakeholders.

The architect is also a sounding board for both developers and non-technical stakeholders, in terms of what is possible, what is not, and the cost implications (in terms of effort, trade-offs, technical debt, and so on) of various options.

It's possible to do the architect's job without coding. But in my personal opinion, this leads to stunted design. It's not possible to come up with a great design unless we understand the low-level details, constraints, and complexity. Many organizations dismiss the architect's role because of their negative experiences of architects that dictate from ivory towers and aren't engaged with the actual task of building working software. But, on the other hand, not having a blueprint can lead to a wild west code base, with small changes causing non-intuitive effects, in terms of effort and the quality of the product.

This book is not a theoretical study in software engineering. This book is meant for architects who want to build awesome, reliable, and high-performance products, while being in the kitchen as the product is getting built!

So what are the guidance systems or guard rails that the architect is expected to deliver on? Essentially, the team needs the following things from an architect.

Requirements clarification

Clarifying and distilling the top-level functional and nonfunctional requirements of the software is a key prerequisite for success. If you don't know what to build, your chances of building something that customers want are pretty slim. Product managers often get caught up on features, but rarely ask what non-functional requirements (or system qualities) the customers need. Sometimes, stakeholders will tell us that the system must be fast, but that's far too subjective. Non-functional requirements need to be specific, measurable, achievable, and testable if we are going to satisfy them. The architect needs to work with all stakeholders and ensure that functional and nonfunctional requirements are well crystallized and consumable for development.

In today's agile world, requirement analysis is an almost ongoing activity. But the architect helps the team navigate the requirements and take decisions on what to do (which may not always be so obvious).

True North

Besides the requirements, we need to define key engineering principles for the system. These principles include the following:

• High-level design: This is the decomposition of the system into high-level components. This serves as the blueprint that the product and code need to follow at every stage of the product development life cycle. For example, once we have a layered architecture (see the following section), then we can easily identify for any new requirement to which layer each new component should go to.
• Quality attributes: We want high quality code, and this means no code checking would be allowed without unit tests and 90% code coverage.
• Product velocity: The product has a bounded value in time and, to ensure that there is high developer productivity, the team should build Continuous Integration / Continuous Deployment (CICD) pipelines from the start.
• A/B testing: Every feature should have a flag, so that it can be shown only to an x percentage of users.

These generic guidelines or principles, along with the high-level design, help the team to make decisions at every stage.

Technology selection

Once we have an architecture, we need to define things, such as the programming languages and frameworks, and make source-versus-build choices for individual constructs. This can include database selection, vendor selection, technology strategy, deployment environment, upgrade policies, and so on. The sum of these factors can often make a straightforward task of choosing something simple into a complete nightmare. And then, finally, all of these technologies have to actually work well together.

Leadership in the kitchen

Once the team starts executing, the architect needs to provide technical leadership to the team. This does not mean taking every technical decision, but implies having ownership and ensuring that the individual components being built add up to the blueprint made. The architect sells the vision to the team at every design review meeting. Sometimes, steering needs to happen, in the form of tough questions asked to the developers in the design review (rather than prescribing solutions).

Coaching and mentoring

The developers working on such a product often need, and seek out, coaching and mentoring outside of their immediate deliverables. One of their core objectives is to learn, discuss tough problems, and improve their skills. Not having an environment where such interactions are facilitated leads to frustrations and developer churn.

While managing the technical stewardship of the product, many times, the architect needs to play the coach and mentor role for the developers. This could involve things ranging from technical feedback sessions to career counseling.

Target state versus current state

When architects and developers are given requirements, they often come up with beautiful and elegant designs. But generally, once the project kicks off, there is pressure on the team to deliver quickly. The business stakeholders want something out fast (a Minimum Viable Product), rather than wait for the Grand Final Product to be released. This makes sense in terms of de-risking the product and provides key feedback to the team, in terms of whether the product is fulfilling business requirements or not.

But this mode of operation also has a significant cost. Developers cut corners while building the project in order to meet the deadlines. Hence, even though we have a clean, beautiful target state in terms of architecture, the reality will not match this.

Having this mismatch is not wrong; rather, it's natural. But it is important for the team to have the target state in mind and define the next set of bite-sized chunks to take the product to the target state during each sprint. This means the architect needs to get involved in sprint planning for the team, along with the product and engineering managers.