In this article by Mitesh Soni, the author of the book DevOps Bootcamp we will discuss how to get quick understanding of DevOps from 10000 feet with real world examples on how to prepare for changing a culture. This will allow us to build the foundation of the DevOps concepts by discussing what our goals are, as well as getting buy-in from Organization Management. Basically, we will try to cover DevOps practices that can make application lifecycle management easy and effective.

It is very important to understand that DevOps is not a framework, tool or any technology. It is more about culture of any organization. It is also a way people work in an organization using defined processes and by utilizing automation tools to make daily work more effective and less manual.

To understand the basic importance of DevOps, we will cover following topics in this article:

• Need for DevOps
• How DevOps culture can evolve?
• Importance of PPT – People, Process, and Technology
• Why DevOps is not all about Tools
• DevOps Assessment Questions

(For more resources related to this topic, see here.)

Need for DevOps

There is a famous quote by Harriet Tubman which you can find on (http://harriettubmanbiography.com). It says :

Every great dream begins with a dreamer. Always remember, you have within you the strength, the patience, and the passion to reach for the stars to change the world

Change is the law of life and that is also applicable to organization as well. And if any organization or individuals look only at the past or present patterns, culture, or practices then they are certain to miss the future best practices. In the dynamic IT world, we need to keep pace with the technology evolution.

We can relate to George Bernard Shaw's saying:

Progress is impossible without change, and those who cannot change their minds cannot change anything.

Here we are focusing on changing the way we manage application lifecycle.

Important question is whether we really need this change? Do we really need to go through the pain of this change?

Answer is Yes.

One may ask that such kind of change in business or culture must not be forceful. Agree.

Let's understand the pain points faced by organizations in Application lifecycle management in modern world with the help of the following figure:

Considering the changing patterns and competitive environment is business, it is the need of an hour to improve application lifecycle management.

Are there any factors that can be helpful in this modern times that can help us to improve application lifecycle management?

Yes. Cloud Computing has changed the game. It has open doors for many path breaking solutions and innovations.

Let's understand what Cloud Computing is and then we will see overview of DevOps and how Cloud is useful in DevOps.

Overview of Cloud Computing

Cloud computing is a type of computing that provides multi-tenant or dedicated computing resources such as compute, storage, and network which are delivered to Cloud consumers on demand. It comes in different flavors that includes Cloud Deployment Models and Cloud Service Models. The most important thing in this is the way its pricing model works that is pay as you go.

Cloud Deployment Models describes the way Cloud resources are deployed such as behind the firewall and on the premise exclusively for a specific organization that is Private Cloud; or Cloud resources that are available to all organizations and individuals that is Public Cloud; or Cloud resources that are available to specific set of organizations that share similar types of interests or similar types of requirements that is Community Cloud; or Cloud resources that combines two or more deployment models that is known as Hybrid Cloud.

Cloud Service Models describes the way Cloud resources are made available to Cloud consumers.

It can be in form of pure Infrastructure where virtual machines are accessible and controlled by Cloud consumer or end user that is Infrastructure as a Service (IaaS); or Platform where runtime environments are provided so installation and configuration of all software needed to run application are already available and managed by Cloud Service Provider that is Platform as a Service; or Software as a Service where whole application is made available by Cloud Service Provider with responsibility of Infrastructure and Platform remains with Cloud Service Provider.

There are many Service Models that have emerged during last few years but IaaS, PaaS, and SaaS are based on the National Institute of Standards and Technology (NIST) definition.

Cloud computing has few characteristics which are significant such as Multi-Tenancy, Pay as you Use similar to electricity or Gas connection, On demand Self Service, Resource Pooling for better utilization of compute, storage and network resources, Rapid Elasticity for scaling up and scaling down resources based on needs in automated fashion and Measured Service for billing.

Over the years, usage of different Cloud Deployment Models has varied based on use cases. Initially Public Cloud was used for applications that were considered non-critical while Private Cloud was used for critical application where security was a major concern.

Hybrid and Public Cloud usage evolved over the time with experience and confidence in the services provided by Cloud Service Providers. Similarly, usage of different Cloud Service Models has varied based on the use cases and flexibility. IaaS was the most popular in early days but PaaS is catching up in its maturity and ease of use with enterprise capabilities.

Overview of DevOps

DevOps is all about a culture of an organization, processes, and technology to develop communication and collaboration between Development and IT Operations teams to manage application life-cycle more effectively than the existing ways of doing it. We often tend to work based on patterns to find reusable solutions from similar kind of problems or challenges.

Over the years, achievements and failed experiments, Best practices, automation scripts, configuration management tools, and methodologies becomes integral part of Culture.

It helps to define practices for a way of designing, a way of developing, a way of testing, a way of setting up resources, a way of managing environments, a way of configuration management, a way of deploying an application, a way of gathering feedback, a way of code improvements, and a way of doing innovations.

Following are some of the visible benefits that can be achieved by implementing DevOps practices.

DevOps culture is considered as innovative package to integrate Dev and Ops team in effective manner that includes components such as Continuous Build Integration, Continuous Testing, Cloud Resource Provisioning, Continuous Delivery, Continuous Deployment, Continuous Monitoring, Continuous Feedback, Continuous Improvement, and Continuous Innovation to make application delivery faster as per the demand of Agile methodology.

However, it is not only about development and operations team that are involved. Testing team, Business Analysts, Build Engineers, Automation team, Cloud Team, and many other stakeholders are involved in this exercise of evolving existing culture.

DevOps culture is not much different than the Organization culture which has shared values and behavioral aspect. It needs adjustment in mindsets and processes to align with new technology and tools.

Challenges for Development and Operations Team

There are some challenges why this scenario has occurred and that is why DevOps is going in upward direction and talk of the town in all Information Technology related discussions.

Challenges for the Development Team

Developers are enthusiastic and willing to adopt new technologies and approaches to solve problems. However they face many challenges including below:

• The competitive market creates pressure of on-time delivery
• They have to take care of production-ready code management and new feature implementation
• The release cycle is often long and hence the development team has to make assumptions before the application deployment finally takes place. In such a scenario, it takes more time to fix the issues that occurred during deployment in the staging or production environment

Challenges for the Operations Team

Operations team is always careful in changing resources or using any new technologies or new approaches as they want stability. However they face many challenges including below:

• Resource contention: It's difficult to handle increasing resource demands
• Redesigning or tweaking: This is needed to run the application in the production environment
• Diagnosing and rectifying: They are supposed to diagnose and rectify issues after application deployment in isolation

Considering all the challenges faced by development and operations team, how should we improve existing processes, make use of automation tools to make processes more effective, and change people's mindset? Let's see in the next section on how to evolve DevOps culture in the organization and improve efficiency and effectiveness.

How DevOps culture can evolve?

Inefficient estimation, long time to market, and other issues led to a change in the waterfall model, resulting in the agile model. Evolving a culture is not a time bound or overnight process. It can be a step by step and stage wise process that can be achieved without dependencies on the other stages.

We can achieve Continuous Integration without Cloud Provisioning. We can achieve Cloud Provisioning without Configuration Management. We can achieve Continuous Testing without any other DevOps practices. Following are different types of stages to achieve DevOps practices.

Agile Development

Agile development or the agile based methodology are useful for building an application by empowering individuals and encouraging interactions, giving importance to working software, customer collaboration—using feedback for improvement in subsequent steps—and responding to change in efficient manner.

One of the most attractive benefits of agile development is continuous delivery in short time frames or, in agile terms, sprints. Thus, the agile approach of application development, improvement in technology, and disruptive innovations and approaches have created a gap between development and operations teams.

DevOps

DevOps attempts to fill these gaps by developing a partnership between the development and operations teams. The DevOps movement emphasizes communication, collaboration, and integration between software developers and IT operations.

DevOps promotes collaboration, and collaboration is facilitated by automation and orchestration in order to improve processes. In other words, DevOps essentially extends the continuous development goals of the agile movement to continuous integration and release.

DevOps is a combination of agile practices and processes leveraging the benefits of cloud solutions. Agile development and testing methodologies help us meet the goals of continuously integrating, developing, building, deploying, testing, and releasing applications.

Build Automation

An automated build helps us create an application build using build automation tools such as Gradle, Apache Ant and Apache Maven.

An automated build process includes the activities such as Compiling source code into class files or binary files, Providing references to third-party library files, Providing the path of configuration files, Packaging class files or binary files into Package files, Executing automated test cases, Deploying package files on local or remote machines and Reducing manual effort in creating the package file.

Continuous Integration

In simple words, Continuous Integration or CI is a software engineering practice where each check-in made by a developer is verified by either of the following:

• Pull mechanism: Executing an automated build at a scheduled time and
• Push mechanism: Executing an automated build when changes are saved in the repository.

This step is followed by executing a unit test against the latest changes available in the source code repository. Continuous integration is a popular DevOps practice that requires developers to integrate code into a code repositories such as Git and SVN multiple times a day to verify integrity of the code.

Each check-in is then verified by an automated build, allowing teams to detect problems early.

Cloud Provisioning

Cloud provisioning has opened the door to treat Infrastructure as a Code and that makes the entire process extremely efficient and effective as we are automating process that involved manual intervention to a huge extent.

Pay as you go billing model has made required resources more affordable to not only large organizations but also to mid and small scale organizations as well as individuals.

It helps to go for improvements and innovations as earlier resource constraints were blocking organizations to go for extra mile because of cost and maintenance. Once we have agility in infrastructure resources then we can think of automating installation and configuration of packages that are required to run the application.

Configuration Management

Configuration management (CM) manages changes in the system or, to be more specific, the server run time environment. There are many tools available in the market with which we can achieve configuration management. Popular tools are Chef, Puppet, Ansible, Salt, and so on.

Let's consider an example where we need to manage multiple servers with same kind of configuration.

For example, we need to install Tomcat on each server. What if we need to change the port on all servers or update some packages or provide rights to some users? Any kind of modification in this scenario is a manual and, if so, error-prone process. As the same configuration is being used for all the servers, automation can be useful here.

Continuous Delivery

Continuous Delivery and Continuous Deployment are used interchangeably. However, there is a small difference between them.

Continuous delivery is a process of deploying an application in any environment in an automated fashion and providing continuous feedback to improve its quality.

Automated approach may not change in Continuous Delivery and Continuous Deployment. Approval process and some other minor things can change.

Continuous Testing and Deployment

Continuous Testing is a very important phase of end to end application lifecycle management process. It involves functional testing, performance testing, security testing and so on.

Selenium, Appium, Apache JMeter, and many other tools can be utilized for the same.

Continuous deployment, on the other hand, is all about deploying an application with the latest changes to the production environment.

Continuous Monitoring

Continuous monitoring is a backbone of end-to-end delivery pipeline, and open source monitoring tools are like toppings on an ice cream scoop.

It is desirable to have monitoring at almost every stage in order to have transparency about all the processes, as shown in the following diagram. It also helps us troubleshoot quickly. Monitoring should be a well thought-out implementation of a plan.

Let's try to depict entire process as continuous approach in the diagram below.

We need to understand here that it is a phased approach and it is not necessary to automate every phase of automation at once. It is more effective to take one DevOps practice at a time, implement it and realize its benefit before implementing another one.

This way we are safe enough to assess the improvements of changing culture in the organization and remove manual efforts from the application lifecycle management.

Importance of PPT – People, Process, and Technology

PPT is an important word in any organization. Wait! We are not talking about Powerpoint Presentation. Here, we are focusing on People, Processes, and Tools / Technology. Let's understand why and how they are important in changing culture of any organization.

People

As per the famous quote from Jack Canfield :

Successful people maintain a positive focus in life no matter what is going on around them. They stay focused on their past successes rather than their past failures, and on the next action steps they need to take to get them closer to the fulfillment of their goals rather than all the other distractions that life presents to them.

Curious question can be, why People matter? In one sentence, if we try to answer it then it would be: Because We are trying to change Culture.

So?

People are important part of any culture and only people can drive the change or change themselves to adapt to new processes or defining new processes and to learn new tools or technologies.

Let's understand how and why with “Formula for Change“.

David Gleicher created the “Formula for Change” in early 1960s as per references available

in Wikipedia. Kathie Dannemiller refined it in 1980. This formula provides a model to assess the relative strengths affecting the possible success of organisational change initiatives.

Gleicher (original) version: C = (ABD) > X, where: C = change, A = the status quo dissatisfaction, B = a desired clear state, D = is practical steps to the desired state, X = the cost of the change.

Dannemiller version: D x V x F > R; where D, V, and F must be present for organizational change to take place where: D = Dissatisfaction with how things are now; V = Vision of what is possible; F = First, concrete steps that can be taken towards the vision; If the product of these three factors is greater than R = Resistance then change is possible.

Essentially, it implies that there has to be strong Dissatisfaction with existing things or processes, Vision of what is possible with new trends, technologies, and innovations with respect to market scenario; concrete steps that can be taken towards achieving the vision.

For More Details on 'Formula for change' you can visit this wiki page : https://en.wikipedia.org/wiki/Formula_for_change#cite_note-myth-1

If it comes to sharing an experience, I would say it is very important to train people to adopt new culture. It is a game of patience. We can't change mindset of people overnight and we need to understand first before changing the culture.

Often I see Job Opening with a DevOps knowledge or DevOps Engineers and I feel that they should not be imported but people should be trained in the existing environment with

Changing things gradually to manage resistance. We don't need special DevOps team, we need more communication and collaboration between developers, test teams, automation enablers, and cloud or infrastructure team.

It is essential for all to understand pain points of each other. In number of organization I have worked, we used to have COE (Center of Excellence) in place to manage new technologies, innovations or culture. As an automation enabler and part of DevOps team, we should be working as facilitator only and not a part of silo.

Processes

Here is a famous quote from Tom Peters which says :

Almost all quality improvement comes via simplification of design, manufacturing… layout, processes, and procedures

Quality is extremely important when we are dealing with evolving a culture. We need processes and policies for doing things in proper way and standardized across the projects so sequence of operations, constraints, rules and so on are well defined to measure success.

We need to set processes for following things:

• Agile Planning
• Resource Planning & Provisioning
• Configuration Management
• Role based Access Control to Cloud resources and other tools used in Automation
• Static Code Analysis – Rules for Programming Languages
• Testing Methodology and Tools  Release Management

These processes are also important for measuring success in the process of evolving DevOps culture.

Technology

Here is a famous quote from Steve Jobs which says:

Technology is nothing. What's important is that you have a faith in people, that they're basically good and smart, and if you give them tools, they'll do wonderful things with them

Technology helps people and organizations to bring creativity and innovations while changing the culture. Without Technology, it is difficult to achieve speed and effectiveness in the daily and routine automation operations. Cloud Computing, Configuration Management tools, and Build Pipeline are among few that is useful in resource provisioning, installing runtime environment, and orchestration. Essentially, it helps to speed up different aspects of application lifecycle management.

Why DevOps is not all about Tools

Yes, tools are nothing. It is not that important factor in changing the culture of any organization. Reason is very simple. No matter what technology we use, we will perform Continuous Integration, Cloud Provisioning, Configuration Management, Continuous Delivery, Continuous Deployment, Continuous Monitoring and so on.

Category wise different tool sets can be used but all perform similar things. It is just the way that tool perform operation that differs else outcome is same. Following are some tools based on the categories:

 Let's see how different tools can be useful in different stages for different operations. This may change based on number of environments or the number of DevOps practices we follow in different organizations.

If we need to categorize tools based on different DevOps best practices then we can categorize them based on open source and commercial categories. Below are just sample examples.

In this book we will try to focus on the Open source category as well as Commercial tools. We will use Jenkins and Visual Studio Team Services for all the major automation and orchestration related activities.

DevOps Assessment Questions

DevOps is a culture and we are very much aware with that fact. However, before implementing automation, putting processes in place and evolving culture, we need to understand existing status of organizations' culture and whether we need to introduce new processes or automation tools.

We need to be very clear that we need to make the existing culture more efficient rather than importing culture. To accommodate assessment framework is difficult but we will try to provide some questions and hints based on which it will be easier to create an assessment framework.

Create categories for which we want to ask questions and get responses for specific application.

Few Sample Questions:

1. Do you follow Agile Principles / Scrum or Kanban?
2. Do you use any tool to keep track of Scrum or Kanban?
3. What is normal sprint duration (2 weeks or 3 weeks)
4. Is there a definitive and explicit definition of done for all phases of work?
5. Are you using any Source Code Repository?
6. Which Source Code Repository Do you use?
7. Are you using any build automation tool such as Ant or Maven or Gradle or not?
8. Are you using any custom script for build automation?
9. Do you have Android and iOS based applications?
10. Are you using any tools for Static Code Analysis?
11. Are you using multiple environment for application deployment for different teams such as Dev, Test, Stage, pre-prod, prod etc. ?
12. Are you using On Premise Infrastructure or Cloud based Infrastructure?
13. Are you using any Configuration management tool or script for installing application packages or runtime environment?
14. Are you using any automated scripts to deploy applications in prod and non-prod environments?
15. Are you using manual approval before application release in any specific environment?
16. Are you using any orchestration tool or script for Application Lifecycle Management?
17. Are you using automation tools for Functional Testing, Load Testing, Security Testing, and Mobile Testing?
18. Are you using any tools for Application and Infrastructure Monitoring?
19. How are defects logged, triaged, and prioritized for resolving them based on priority?
20. Are you using notification services to let stakeholders know about the status of application lifecycle management?

Once questions are ready, prepare responses and based on responses decide rating for each response that is given for the above questions.

Make a framework flexible so even if we change any question in any category then it will be managed automatically.

Once rating is given, capture responses and calculate overall ratings by introducing different conditions and intelligence into the framework.

Create category wise final ratings and create different kind of charts from the final rating to improve the reading value of it.

The important thing to note here is the significance of organizations' expertise in each area of Application lifecycle management. It will give assessment framework a new dimension to add intelligence and make it more effective.

Summary

In this article, we have set many goals to achieve throughout this book. We have covered Continuous Integration, Resource provisioning in the Cloud environment, Configuration Management, Continuous Delivery, Continuous Deployment, and Continuous Monitoring.

Setting goals is the first step in turning the invisible into the visible. Tony Robbins

We have seen how Cloud Computing has changed the way innovation was perceived earlier and how feasible it has become now. We have also covered need for DevOps and all different DevOps practices in brief.

People, Processes, and Technology is also important in this whole process of changing existing culture of an organization. We tried to touch upon the reasons why they are important.

Tools are important but not the show stopper; Any toolset can be utilized and changing a culture doesn't need specific set of tools.

We have discussed in brief about DevOps Assessment Framework as well. It will help to get going on the path of changing culture.

Resources for Article:

Further resources on this subject:

• Introduction to DevOps [article]
• DevOps Tools and Technologies [article]
• Command Line Tools for DevOps [article]