Moving from Waterfall to Agile projects

As we move into the cloud, other new terms around working practices come to the fore. DevOps, DevSecOps, and Agile are becoming ingrained in those responsible for building software and infrastructure.

If you come from a software or a DevOps background, there is a good chance you already understand these concepts, but if not, it helps to understand them.

Waterfall

Traditional waterfall project delivery has distinct phases to manage and control the build. In the past, it has often been considered crucial that much effort goes into planning and designing a solution before any engineering or building work commences.

A typical example is that of building a house. Before a single brick is laid, a complete architectural blueprint is produced. Next, foundations must be put in place, followed by the walls, roof, and interiors. The idea is that should you change your mind halfway through, it would be challenging to change anything. If you decide a house needs to be larger after the roof is built, you would need to tear everything down and start again.

With a waterfall approach, every step must be well planned and agreed at the outset. The software industry developed a bad reputation for delivering projects late and over budget. Businesses soon realized that this was not necessarily because of mismanagement but because it is difficult to articulate a vision for something that does not yet exist and, in many cases, has never existed before.

If we take the building metaphor, houses can be built as they are because, in many cases, they are merely copying elements of another house. Each house has a lot in common—walls, floors, and a roof, and there are set ways of building each of these.

The following diagram shows a typical setup of a waterfall project, with well-defined steps completed in turn:

Figure 1.7 – Typical waterfall process

With software, this is not always the case. We often build new applications to address a need that has never been considered or addressed before. Trying to follow a waterfall approach has led to many failed projects, mainly because it's impossible to design or even articulate the requirements upfront fully.

Agile

Thus, Agile was born. The concept is to break down a big project into lots of smaller projects that each deliver a particular facet of the entire solution. Each mini-project is called a sprint, and each sprint runs through a complete project life cycle—design, plan, build, test, and review.

The following diagram shows that in many ways, Agile is lots of mini-waterfall projects:

Figure 1.8 – Agile process

Sprints are also short-lived, usually 1 or 2 weeks, but at the end of each one, something is delivered. A waterfall project may last months or years before anything is provided to a customer—thus, there is a high margin for error. A small misunderstanding of a single element along the way results in an end state that does not meet requirements.

A particular tenet of Agile is "fail fast"—it is better to know something is wrong and correct it as soon as possible than have that problem exacerbate over time.

This sprint-led delivery mechanism can only be achieved if solutions are built in a particular manner. The application must be modular, and those modules designed in such a way that they can be easily swapped out or modified in response to changing requirements. An application architect must consider this when designing systems.

At this point, you may be wondering how this relates to the cloud. Agile suits software delivery because solutions can be built in small increments, creating lots of small modules combined into an entire solution. To support this, DevOps tooling provides automated mechanisms that deploy code in a repeatable, consistent manner.

As infrastructure in the cloud is virtualized, deployments can now be scripted and therefore automated—this is known as Infrastructure as Code (IaC).

IaC

In Azure, components can be created either in the portal using the graphical user interface (GUI) with PowerShell or by using JSON templates. In other words, you can deploy infrastructure in Azure purely with code.

IaC provides many benefits, such as the ability to define VMs, storage, databases, or any Azure component in a way that promotes reusability, automation, and testing.

Tools such as Azure DevOps provide a central repository for all your code that can be controlled and built upon in an iterative process by multiple engineers. DevOps also builds, tests, and releases your infrastructure using automated pipelines—again, in the same way that modern software is deployed.

The DevOps name embodies the fact that operational infrastructure can now be built using development methodologies and can follow the same Agile principles.

DevSecOps takes this even further and includes codifying security components into release pipelines. Security must be designed and built hand in hand with infrastructure and security at every level, as opposed to merely being a perimeter or gateway device.

Cloud architects must, therefore, be fully conversant with the taxonomy, principles, and benefits of Agile, DevOps, and DevSecOps, incorporating them into working practices and designs.

Microsoft provides a range of tools and components to support your role and provides best-in-class solutions that are reliable, resilient, scalable, and—of course—secure.

As we have seen, architecture in the cloud involves many more areas than you might traditionally have gotten involved in. Hopefully, you will appreciate the reasons why these changes have occurred.

From changes in technology to new ways of working, your role has changed in different ways—although challenging, this can also be very exciting as you become involved across various disciplines and work closely with business users.