Embracing Agile frameworks for collaboration

In this section, we will revisit the event that brought Agile into the mainstream - “The Manifesto for Agile Software Development” - and the leading lightweight software development methodology that followed: the Scrum framework. We’ll start by revisiting the Agile Manifesto to understand its role in shaping modern agile thinking.

Rediscovering the Agile Manifesto

In Utah’s snowy peaks of the Wasatch Mountains, a landmark event transpired between February 11 to 13, 2001, at The Lodge at Snowbird ski resort. Seventeen renowned thought leaders from various corners of the software development industry convened. Their goal? To bridge their methodologies and discover a unified approach to software development. From this summit, the Manifesto for Agile Software Development was born. And it has become the defining description of what it means to be an Agile software development organization.

The manifesto (source: https://agilemanifesto.org/) reads the following. We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

"While we acknowledge the value of the components on the right, we believe those on the left hold greater significance."

The manifesto also outlines the Principles behind the Agile Manifesto. But since we provided a modern list of Agile principles earlier in this chapter, we won’t list the 12 principles here. If you want to learn more, you can find them here: https://agilemanifesto.org/principles.html.

Important note

This manifesto was more than a statement. It provided the collective wisdom of pioneers who sought to break free from the shackles of conventional, documentation-heavy software development processes. It marked the genesis of an approach rooted in collaboration, adaptability, and delivering tangible value.

So, time moves on, and as it does, people inevitably learn from their experiences and question what has become dogma. That statement is undoubtedly true for the Agile Manifesto. While still quoted often, some detractors feel we need to refine the original thinking. In the next section, you’ll discover some critiques and new thinking about what agility means today.

Moving beyond the Agile Manifesto

The Agile Manifesto has been the foundation of the Agile movement, driving change in software development and project management. Over two decades later, the world of technology has evolved drastically, and so have the contexts in which Agile practices are applied. This has led to debates and discussions about the current relevance of the values and principles laid out in the manifesto. So, let’s explore some of the current thinking and the evolution of the Agile movement.

Expanding Agile’s horizons

Initially designed for software development, Agile methodologies have expanded into other business domains, and even hardware development. With this expansion, there’s been a realization that some principles of the Agile Manifesto, while universal in spirit, may require rethinking to fit contexts outside of software development. For instance, the focus on working software as the primary measure of success needs to be translated differently in non-software domains.

This issue is a primary driver for our developing the BASE conceptual model and pattern language, which we’ll introduce in Chapter 9, Defining a Business Agility System for the Enterprise (BASE).

Harmonizing continuous delivery with Agile

The Agile Manifesto has always emphasized customer collaboration and the ability to adapt to change. These are human-driven endeavors. However, with the exponential growth in technology and user expectations, there’s been an undeniable push for continuous delivery and streamlined operations through advanced tooling, integrations, and automation.

This modern demand for seamless software delivery has drastically reshaped our understanding of agility. Continuous integration and continuous delivery (CI/CD) pipelines, augmented by DevOps practices and tooling, introduce frequent integration of work, streamlined information flows, and a heavy reliance on automation.

This modern approach speeds up software development work and requires a shift in mindset. Being Agile is no longer just about iterative cycles and feedback loops; it now entails a deep integration of these cycles into continuous delivery processes, ensuring that agility permeates every stage of the software development lifecycle.

The manifestation of this shift is evident in the rise of DevOps and VSM. These modern paradigms champion holistic value delivery and challenge the traditional boundaries of siloed iterative development.

Merging the cadence of Agile iterations with the consistency of continuous delivery presents a dynamic blend, redefining how businesses approach product development.

We’ll discuss methods to integrate Lean and Agile cadences later in this book, starting with Chapter 8, Implementing Basic Lean-Agile Solutions Teams (BLAST). For now, it’s important to understand that we can have more than one team working in collaboration across a product life cycle. This emphasis on product development has profoundly influenced the essence of operating as a value-centric Agile business, distinct from managing constraint-based projects.

Expanding digital horizons

In the section on transitioning to product-oriented thinking, we discussed the importance of emphasizing products over projects. However, the topic extends beyond just managing software delivery products.

As we transition from a project-centric to a product-centric mindset, we must recognize that products include all types of products and services, many of which are integrated as solutions. The Internet of Things (IoT) is a good example. Additionally, we need to be aware that software products can support value stream improvements for the organization, whether internally and externally facing. Today, everything from tangible products to processes and services can be enhanced through digital tools.

In Chapter 4, Driving Improvements with Value Stream Management (VSM), we’ll underscore the importance of a comprehensive organizational improvement strategy, one that goes beyond just software. You’ll also learn how to make product-oriented, value-based improvement assessments.

Given the constraints on resources, it’s imperative to strategize where we can maximize returns on our investments. It may not always be linked to software. Yet, software and digital solutions can often be catalysts for transforming a process or enhancing a tangible product or service.

Given that all organizations have limited resources, we must evaluate where we get the biggest bang for our investments, which might not be software-related. Still, in many cases, software and digital services can facilitate a change in a process and a physical product or service. The generic VSM methods and tools we’ll introduce in Chapter 4 and the business intelligence strategy you’ll learn about in Chapter 10, Enhancing Decision-Making in the Lean-Agile Enterprise, will help you make those types of business assessments and investment decisions.

Embracing the rise of business agility

Agile transformation has moved beyond IT departments. Organizations now see agility as a business-wide competency. This has led to discussions on whether the Agile Manifesto’s values sufficiently address broader organizational concerns, such as strategy, governance, and culture. Business agility encompasses a more holistic view, considering market adaptability, organizational flexibility, and cultural agility. While the Agile Manifesto provides a strong foundation, some argue it may not be comprehensive for this broader scope.

Revisiting the role of individuals

The Agile Manifesto values “individuals and interactions over processes and tools.” However, with the rise of digital tools and platforms that facilitate Agile practices (such as Jira, Trello, or Slack), there’s an ongoing debate about the balance between human interactions and the role of tools in enhancing collaboration. While tools help us visualize and streamline processes, the essence of Agile remains in human cooperation. Finding the right balance is an ongoing challenge.

Adapting to globalization with a distributed team

The manifesto emphasizes face-to-face communication. However, the global nature of today’s businesses often means distributed teams working across time zones. The COVID-19 pandemic further amplified remote working. This has led to questions about how the manifesto’s principles apply in such distributed, remote settings. Effective agility in this new context requires a mix of digital tools, revised communication strategies, and trust-building.

In summary, while the Agile Manifesto remains a foundational document, the evolution of the Agile movement requires continuous reflection and adaptation. The principles laid out in the manifesto provide a compass, but the journey of agility is dynamic, shaped by the changing landscapes of technology, business, and global contexts.

At this point, you should have a relatively comprehensive understanding of what it means to achieve business agility, both past and present. Now, let’s discuss the leading Agile framework, Scrum, and how it supports business agility.

Important note

This chapter uses Scrum as its reference Agile framework for individual team environments. We will also introduce team-of-teams (ToT) concepts for more extensive development and problem-solving requirements that require multiple teams to collaborate and integrate their work.

Charting Scrum’s ascendancy in Agile practices

The Scrum framework stands out among the various lightweight software development methodologies, emerging as the predominant approach for small teams. For example, the 16th Annual State Of Agile Report (2022) by Digital.ai underscores its popularity. The report notes that “Scrum continues to lead, increasing from 58% in the 14th survey to 87% in the current survey.” The Scaled Agile Framework® (SAFe®) remains the most popular choice for enterprises, yet only 26% of respondents reported using it, which is a decline of over 50% compared to the previous year. However, the survey’s scaling agility question changed which might account for the majority of this decline.

The official Scrum Guide (https://scrumguides.org/) defines Scrum as “a lightweight framework that assists individuals, teams, and organizations in deriving value through adaptive solutions for intricate challenges.”

Scrum retains its leadership role for small teams because of its simplicity, flexibility, and proven success in promoting collaboration and adaptability in diverse project environments. Its iterative approach and emphasis on continuous feedback make it highly effective in delivering valuable products or solutions to complex problems quickly and responsively.

As we progress through this chapter, we’ll review how Scrum works. For now, it will be helpful to know that Scrum implements an iterative and incremental model of Agile via Sprints. Every Sprint, or development iteration, commences with a working hypothesis that can be tested. As the Sprint progresses, teams adjust incrementally, drawing from their observations and experiments.

We’ll take a quick look at the Scrum events that guide the flow of work across a Sprint, as shown in Figure 2.2, in the following subsection.

Understanding the Scrum Sprint cycle

The following diagram details the Scrum events that guide its workflow:

Figure 2.2 – Scrum framework

Work starts with creating the Product Backlog, which is managed by the Product Owner, and incorporating input from various stakeholders, including the Scrum Team and customers. During Product Backlog Refinement, the Scrum Team clarifies the backlog items and their priorities. The Sprint begins with Sprint Planning, where the team selects the highest-priority work items from the Product Backlog, creating the Sprint Backlog. Sprints are time-boxed development cycles, which means the durations are fixed, regardless of whether the Scrum Team meets its Sprint Goal objectives.

Developers pull items from the Sprint Backlog based on their capacity. Ideally, any team member can handle any job, but specialization is sometimes required. The Daily Scrum is a daily, time-boxed 15-minute meeting to inspect progress toward the Sprint Goal and adapt the Sprint Backlog as necessary, adjusting the upcoming planned work as needed. The entire Scrum Team is responsible for creating a valuable, useful, and potentially releasable Increment every Sprint. Each iteration concludes with two events: the Sprint Review, where the increment is inspected, and the Sprint Retrospective, which allows the team to reflect on how to improve quality and effectiveness. The Scrum Master is accountable for the team’s effectiveness, helping it improve its practices and communication and removing barriers.

Now that we’ve introduced the basics of Scrum, let’s review why it works.

Applying Scrum’s foundational principles

These are the two primary concepts that underpin Scrum:

• Empiricism: Scrum operates on the principle that knowledge is derived from practical experiences and iterative experimentation, making decisions based on observations.
• Lean Thinking: The Scrum Guide acknowledges the influence of Lean Thinking on Scrum, although it does not explicitly specify fundamental Lean principles, such as eliminating waste or improving value flow. However, Scrum shares several Lean concepts, such as focusing on adding value from the customers’ perspectives, delivering in short cycles, providing immediate feedback, respecting people, and improving continuously.

To optimize value delivery, combining Scrum with Lean’s comprehensive concepts is essential. To accomplish this objective, the BASE and BLAST frameworks integrates the work of multiple Lean-Agile teams to support product and value stream improvements, streamlining work, material, and information flows, recognizing and mitigating constraints, centering work around specific products and services, and eliminating wasteful processes.

Adapting to Scrum’s deliberate flexibility

Scrum’s founders, Ken Schwaber and Jeff Sutherland, quickly point out that Scrum is an intentionally incomplete framework. Rather than prescribing a specific way of working or limiting the types of work it can be applied to, Scrum remains generically designed, offering broad applicability across various domains. In other words, Scrum provides a foundational framework to improve agility, whether the challenge lies in business, science, or technical problem-solving.

Scrum’s flexibility offers the following benefits:

• Widescale applicability: Scrum’s versatile nature ensures that it is suitable for diverse problems or business domains, granting teams the flexibility to address various challenges.
• Freedom of methodology: Development teams are granted autonomy within the Scrum framework. They have the skills and knowledge necessary for their tasks and can choose the best methods and tools that align with their unique work requirements.
• Framework adherence: While Scrum grants autonomy, it also emphasizes discipline. Every Scrum Team operates within the boundaries set by the framework. These guidelines serve as beacons, guiding Scrum team interactions and ensuring harmonious relationships.

In essence, Scrum provides a flexible yet structured environment, striking a balance between freedom of operation and the consistency of a framework. Note that a small team delivering the work is at the heart of every Sprint iteration. We’ll talk about that next.