Introducing agile EA

To start with, being agile in practice does not necessarily mean that we will follow any of the agile software development methodologies, such as Scrum or eXtreme Programming (XP). Being agile simply means being adaptive to the continuous changes to the requirements within your enterprise and being responsive with the right amount of information, at the right time, to the right people.

Another important point to keep in mind is we are not introducing a new methodology that is better or worse than TOGAF®, and we are not following the Open Agile Architecture™ either (https://pubs.opengroup.org/architecture/o-aa-standard/index.html). We are still following TOGAF®; it will still guide our progress and we will be reusing more of the framework's components, such as the taxonomy, metamodels, TOGAF®'s techniques and guidelines, and most of the ADM content. The main differences will be in the order of the ADM phases and steps.

This section will explain how EA can provide better value if implemented with agile principles in mind by focusing on delivering tangible and valuable artifacts that can grow as the work progresses. Before we go deeper into the approach, let's start by understanding what agile EA means.

Understanding agile EA?

Part of our proposed solution is to be agile in the EA practice and to focus on delivering products where they are needed the most. These products are called architectural artifacts within the context of EA, and they include all the diagrams, catalogs (dictionaries), and matrices (cross tables) that make the documents that are produced during an EA practice. I highly recommend that you read the 12 agile principles that are defined in the Agile Manifesto for your knowledge if you are not already familiar with them (https://agilemanifesto.org/) but replace every software word you see with artifact to see EA through agile eyes. The agile principles can be applied to any project in life, including personal ones, and not just on software development.

As an enterprise architect, people will expect you to provide them with advice and solutions everywhere in the enterprise. If they feel that you will slow their progress down or enforce things that they do not require, you will end up detaching yourself from the rest of the organization and will end up sitting alone in your ivory tower.

One main reason for EA failure in many organizations is that product owners overlook involving the EA office in their projects to avoid possible delays. Product owners want their products to be up and running as soon as possible. They will not feel comfortable adding tasks to their scope of work that they do not believe are needed just because the framework says so. They will not be happy if you ask them to hold their progress and wait until the EA office finishes tailoring the framework, defining value streams, and defining the governance model.

Defining – or even just refining – all these components can keep you and the rest of the team busy for months, if not years, and the product owner will soon perceive you as an obstacle and a risk to the project. It is a fact that you need to respect if you want to have a successful cooperative relationship with the rest of the enterprise.

Following a methodology is a way to do things right, no doubt about it. However, the product owners will not wait for you to finish tasks that they do not require and are not part of their project's scope of work. This is where you need to balance doing things right and doing the right things.

If the objective is to develop a new web application, for example, as an enterprise architect you must complete the full picture by connecting all the dots and making sure that the application realizes actual business services, and the application has – or will have – the required technology infrastructure that will support it. The key to being more practical than theoretical is setting up the scope and prioritizing your tasks to deliver useful artifacts within the available time.

Continuing with the web application example, it is more useful and more appreciated by the product owners if you start by conceptualizing the application services that they have in mind and help them to plan and design them. Once you have the desired artifacts, you can add more content to them and/or develop other artifacts that are based on them. You will keep using TOGAF® and its material as a reference for the big picture, but you do not have to start from the top, and you do not have to complete each phase before you start the next one. That is the waterfall approach, which was deprecated from software development years ago and must be deprecated from EA development as well.

Comparing agile EA with EA

The idea of agile EA is not new, but in fact, it is as old as EA itself. John Zachman, the father of EA, introduced his framework in the mid-1980s and it is still known today as the Zachman Framework. Every EA practitioner or learner will have heard of it even though the number of Zachman Framework practitioners is far fewer than those of TOGAF®. The Zachman Framework is incredibly famous for its 6x6 matrix, which has What, How, Where, Who, When, and Why as columns, and has enterprise layers such as Scope Contexts, Business Concepts, System Logic, Technology Physics, Tool Components, and Operational Classes as rows (as defined in https://www.zachman.com/about-the-zachman-framework).

Unlike TOGAF®, the Zachman Framework has no specific process to follow, and architects are free to start anywhere they want on the matrix. Once they define the artifacts they need, they can simply go to any other cell on the matrix and build more artifacts. This is an agile approach that was introduced way before agile software development was even thought of. Architects can start with the process definition, for example, followed by responsibility configuration and then the inventory instantiation artifact without constraint. They can decide which artifacts to develop based on stakeholders' demands and requirements and the availability of information, rather than deciding based on a sequential flow of steps.

The big picture will be formed as you build more artifacts here and there and as more details and relationships are revealed, exactly like finding and fitting the pieces of a puzzle. Sometimes you will find that you have made some wrong assumptions based on the information that you had in hand at that time, and the artifact is incomplete or unclear, but this is a very acceptable side effect of agile development, and you need to keep in mind that nothing is written in stone and every artifact is subject to changes.

Sharing your artifacts continuously with the rest of the team and with the product owners as you gradually build them will help in communicating your thoughts in the early stages and will help you make the required corrections and adjustments as needed. Making continuous small changes and updates to the artifacts is better than going through a lengthy waterfall process and getting caught in Effort Blackholes.

Embedding agile EA into TOGAF®

As mentioned earlier, this book is based on TOGAF®, and the reason we talked briefly about Zachman in the previous section was to show that the agile way of EA development is not a strange idea or an extreme thought to be afraid of. In this section, we will show you how you can embed the agile mindset into TOGAF® without compromising the principles of the framework. These are the main elements of the agile EA approach:

• The big picture will remain guided by TOGAF®.
• Start anywhere where EA contribution is required.
• Focus on delivering artifacts.
• Use smaller metamodels.
• Build the architecture governance as the architecture evolves.
• Use an EA tool for the architecture repository.

The following subsections will explain each of the preceding elements in more detail.

The big picture will remain guided by TOGAF®

Being agile and focusing on artifacts does not mean losing sight of the big picture. Everything that is in TOGAF® will still be used as guidance and this is how:

• TOGAF®'s content metamodel will still be the big picture that shows the different elements within the enterprise and the possible relations between them. We will use ArchiMate®'s metamodel because its metamodels provide more details than the ones provided by TOGAF®, but both framework models have the same background.
• We will still use the taxonomy of TOGAF® and ArchiMate® as the formal definition of the enterprise elements. We may elaborate on some definitions if needed, but we will stick to the standard definitions all the time.
• We will still use the list of artifacts and the list of proposed inputs and outputs by TOGAF® as guidelines.
• We will still use TOGAF® for guiding the definition of our governance model but without forcing it. This means that not every role, process, or responsibility must be built upfront, but we take what we need and change as we progress.
• We will adapt and reuse any of the tools and techniques that are provided by TOGAF® and adjust them to our needs as we progress.

Start anywhere where EA contribution is required

As an enterprise architect, you need to contribute to every development progress for any business unit and at any layer of the enterprise. Whether it is a strategy-crafting initiative, a marketing campaign, a new software procurement, or a plan to retire some legacy mainframe applications, enterprise architects need to be involved in all these initiatives as they progress. They do not have to be the subject matter experts in all these different domains, but they need to add value to all these areas by encouraging the use of standards and connecting and aligning the different components in the different layers. Involving the enterprise architects in multiple projects at different locations of the enterprise can uncover some hidden relationships between these projects that only someone with EA eyes can see.

To demonstrate how enterprise architects can contribute to any layer with great value, let's take the mainframe application retirement as an example:

• Retiring an application will affect the other applications that exchange data by taking the forms of inputs, outputs, or both. If not taken into consideration, this integration dependency may get broken and will result in unplanned errors or missing information.
• It will affect the infrastructure (the technology) that the application uses so either it becomes available for other applications, or it becomes useless and ready to be retired.
• It will affect the people operating, administrating, and using the application so either they will require training to be able to use the replacement application or they need to be reallocated and play different roles for the best use of resources.
• It will affect the business services that depend on the application in automating some or all of the processes. If multiple business services depend on the application, there will be a possibility of some of them going down or being interrupted due to the broken dependency.
• Having a service or services down may affect the organization's ability to achieve its strategic goals, which may by itself have effects on other elements in other layers because everything within the enterprise is connected.

Only enterprise architects can provide these dependency views, and this is one example of where they can add value to any project at any layer. We will talk more about this in detail in Chapter 8, Business Architecture Models.

Focus on delivering artifacts

One benefit of following an EA framework is to make deliverables and artifacts standardized, organized, consistent, and of higher quality. If the delivered artifacts are of no use to the stakeholders, then your efforts and their time will be wasted, something that EA has struggled with for years. Remember that any effort that does not have a deliverable is a wasted effort. There is the concept of delivering the Most Viable Product (MVP) in agile software development, having the minimum essential features delivered first, then the product keeps growing by adding more features to it. That concept must be followed during EA development as well and the product within the EA context is an artifact.

One of the recommendations in agile software development is to break down the tasks into smaller pieces or parts, and we will follow this in agile EA. Having a deliverable such as the Architecture Definition Document (ADD) that contains multiple smaller artifacts, such as the business processes catalog, business services catalog, application components catalog, business processes to application components matrix, application components to technology services matrix, and many other sections, it can take an exceedingly long time with all the different sections that it contains. So, instead of having a single task for developing the ADD, it is better to break it down into smaller artifacts that each can be completed within a few days or 2 weeks at most, or else they must be broken down into smaller parts that fit within the 2-week window.

If you have multiple engagements to participate in, then you need to distribute your time spent on all these engagements without affecting or slowing down the others. You can keep growing your artifacts simultaneously as you go, and you may discover some dependencies between the two that no one considered before.

Use smaller metamodels

Metamodels are the templates that architects use as references when creating EA artifacts to maintain the consistency and the quality of the artifacts across the EA repository. TOGAF® and ArchiMate® have generic metamodels that you can start with, but they do not cover every possible relationship and sometimes they are difficult to read and understand. Customizing the standard metamodels is recommended to fit the exact requirements of your organization; however, trying to tailor the entire TOGAF® and ArchiMate® metamodels can take a very long time. Following the same concept of breaking large artifacts down into smaller ones, you must break the effort of tailoring the metamodel down into smaller parts.

The next section of this chapter will introduce the concept of focused metamodels, which we will be following throughout the rest of the book, but for now, remember that creating metamodels is as important as creating the artifacts themselves. Even if you are the only architect who is building artifacts, having metamodels will help you maintain the required consistency within your own set of deliverables.

Build the architecture governance as the architecture evolves

Defining the architecture governance model is essential to ensure the proper development and maintenance of EA artifacts. With proper governance, every person involved in the architecture development and maintenance knows what to do to keep the artifacts updated. Being agile does not mean being chaotic and without processes or strategy. Building an architecture repository without a governance model around it will result in having most of the artifacts outdated or with broken relationships within a year at most. Outdated or incorrect information will break the trust between stakeholders and the information that the EA repository provides, so it must be avoided. Therefore, we know the importance of having an established architecture governance model but as with everything else, it must be broken down into smaller parts and we start with only the ones that we need for the MVP, then we enhance it by adding more to it.

Chapter 37 of TOGAF® 9.2 (https://pubs.opengroup.org/architecture/togaf9-doc/arch/chap37.html) provides the concept of the EA repository, and Chapter 44 (https://pubs.opengroup.org/architecture/togaf9-doc/arch/chap44.html) provides the concept of architecture governance. Both chapters represent the visionary big pictures that you can aim for. However, that concept can be too big and difficult to implement if you are just starting, so you need to build the governance MVP that can govern only what is being developed and avoid boiling the ocean. Attempting to boil the ocean will result in Effort Blackholes with no doubt. It is important to look at the big picture and know what is behind it, but it is more important to use your resources wisely. Chapter 10, Operating the EA Repository, of this book will provide you with a simple governance framework that can evolve as per the maturity of your organization.

Use an EA tool for the architecture repository

It is very possible to implement TOGAF® or any EA framework using Microsoft Office tools and hosting everything on a shared network drive. However, having an EA tool such as Sparx makes it easier and more practical to do so, especially if you are doing it in an agile way. Since agile is all about accepting changes even at later stages of development, doing that without a tool can be extremely difficult to maintain. The tool will let you know if you are changing an element that is connected to other elements, for example, which may break the established relationship. We will talk more about the EA tool that we will use to build the EA repository in the last section of this chapter. For now, you need to keep in mind that having an EA tool makes modifying the artifacts easier, which will help you do EA in an agile way.

Now you know the agile EA approach and the importance of building artifacts, it is time to introduce the modeling language that will be used for this purpose.