Software development challenges

With decades of software engineering and software development advances, we have learned techniques and methods to improve the software productivity and customer satisfaction. However, every day we push the bar to a new limit and software development organizations are faced with the challenge of catching up with the ever increasing expectations from customers.

The hardware costs are steadily declining and the software is growing in complexity day-by-day. Yet, customers are baffled by the fact that the software is slow even on faster machines. With the increasing restrictions on costs, resources, and time, software vendors identify the improvement areas. Thereby, software vendors want to increase the productivity by filling the gaps and adopting efficient processes, methods, and tools. Therefore, the next important question is "What are the crucial software development challenges"?

Today, several software development projects are faced with similar challenges, primarily focused on the distributed teams and collaboration. Major categories of the challenges include but are not limited to the following:

• Teams are globally spread out

• Cultural differences make it difficult to communicate effectively

• Difficulty in coordination with Global Teams

• Fast paced technology paradigms

• Customized development practices in teams

In the following sections, we will see the distributed teams, collaboration, and team management aspects of software development in detail.

Distributed teams

Increasing globalization has brought a few changes in the software development industry. One of them is "off shoring" either part of the system or whole modules. Off shoring is chosen for obvious reasons such as business cost effectiveness, availability of talent pool, and other strategic reasons. Off shoring has led to global software development teams with members from at least two geographical regions collaborating on the same project.

	Software Professionals must adapt to the new reality of offshore outsourcing.
	--Bertrand Meyer, ETH Zurich

A team based on the same culture often has many common things to share and communicate informally compared to the teams spanning multiple cultures. Time zone difference, geographical distance, language, and cultural barriers create more information and psychological gaps in the project.

The co-located teams have the advantage that they have a line of sight, which helps them have informal discussions and thereby understand each other well. In the case of the geographically distributed teams, there is a possibility of misunderstanding created by communication. The tone of the language, and the choice of the words used in written or verbal communication, largely depend on the culture that one belongs to.

In practice, team members get information necessary to perform their tasks largely from peers, as opposed to formal, documented sources. To be able to tap this social resource, team members need to build social capital, meaning that they need to establish trust relationships with their peers.

This process of building trust does not occur so readily in a globally distributed team, sometimes leading to problems in the collaboration process. Informal collaboration tools, coupled with appropriate management practices, are an important part of the solution.

Collaboration

The act of working jointly in a co-operative manner to achieve a piece of a task is collaboration and it can happen in a formal or informal way. In the formal way of collaboration, typically the project manager or team leader assigns a "contact person" for any kind of questions or issues to answer. In the informal way of collaboration, team members work together at lunch or over a cup of coffee.

	No matter what the problem is, it's always a people problem.
	--Gerald M. Weinberg

The proliferation of several technological advancements such as Web 2.0, Service Oriented Architecture, Mashups, Software as a Service, Cloud Computing, and many others made teams depend on each other for expertise and carry out the knowledge management in a collaborative manner.

Due to the nature of the cultures and backgrounds, formal or/and the informal way of communication is preferred one over the other. Convergence of the collaboration tools into the software development tool was needed to solve the collaboration problems. Collaboration tools offer much more than the basic communication channels provided by e-mail and telephones. These tools can make team members aware of each other. After team members are aware of each other, they are then able to communicate and work together to achieve common goals.

Collaboration is supported through tools that allow communication, knowledge sharing, and analysis. Tools such as blogs, Wikis, bulletin boards, and instant messaging clients are a few collaborative tools used now. An interesting side effect of informal knowledge sharing tools, such as blogs and Wikis, is that they increase awareness. When people read blogs, they can learn more about the authors and their interests. In this way, even if two people are in a totally different geographical area, they may know a lot of each other from the social collaboration and communication tools.

Project management

With the global teams in place, pressure to achieve higher team productivity makes team management even more challenging.

Team addition

When a member joins the team, he should be introduced to the project technology, development environments, given proper access rights to the tools, and so on. The team member is forced to learn several tools, processes, and techniques in a fast paced environment unless he comes from a similar environment.

Task assignment and scheduling

Once the project is broken in to smaller chunks, they should be scheduled to meet the release schedule. The team manager should be sensitive about the geographically distributed teams with different vacation times, holiday times, or anything that affects schedule. In such situation, the communication between the team manager and team member becomes the bottleneck for the success of the project.

The team manager should have efficient tools for task assignment and task scheduling. The challenge is multidimensional where a single module's team is spread across the geography. In such case, the team within should co-ordinate to achieve their tasks and must find an efficient way to communicate.

Task assignment and scheduling includes the following:

• Tasks dependency analysis

• Determining the amount of slack

• Dealing with uncertainties

• Tolerances in estimation

Project tracking

Defining the requirements, designing smaller chunks of tasks, and scheduling the tasks does not yield the project success without tracking the project. Project tracking should make it visible about who is doing what, tasks in progress, whether there's any slack, whether there are any problems, whether anyone needs help, and team-member efficiency. A burn-down chart will also help to get an idea of the work left to do versus time. In Agile development, a burn-down chart is a graphical representation of work left to do versus time.

Build, test, and release

Once requirements are coded for the functionality, they must be integrated to verify and validate. Any issues in the build should be immediately notified to the team and corrective action should be taken. In the modern software development, it is not uncommon to track the software stability with nightly builds.

Many projects follow the practice of writing unit tests and sometimes automated tests are embedded during the build process. Considering that developers covered all the functionality with the unit test cases, a build gives a considerable confidence about the product. Then, the Quality Assurance (QA) team can do the functional testing.

Depending on the project and team, one may have different release time frames. Some projects have relatively shorter release cycles (for example, 2-4 weeks) and some have longer release cycles (every six months or even longer than one year). In either case, it is important to communicate the importance of the releases and schedule to the entire team.

Transparency

In the global or distributed team setting, it is very easy to misunderstand and misrepresent the communication. Project updates not happening on an open platform leads to team members feeling insecure and mistrusted. Or even worse when you come to know about something after the fact.

Note

Transparency increases the team member's confidence and trust.

Choosing the tools in a way to make the requirements, task estimates, task schedule, work progress, project tracking, project health, build results, release schedule, and project announcements transparent is crucial to gain the trust and confidence of the team. Each one contributes to the complexity of the software being developed.

The following table summarizes the challenges in the software development with respect to each stakeholder:

Software development challenges
Team member	What tools should I use? How should I get the access rights? Who should I approach for the project plan? Who are my colleagues and what are they responsible for? What's happening in my team? Should I get the code reviewed by someone? If yes, who is it? The process documents are too large. I don't have time to read those documents. I need to back up my current changes as I have to work on a higher priority task. Context switching is complex. I want to know which team members are online and discuss things with them. I can plan better if I know the release schedule. I always have to ask my project manager for the release schedule as it's not published. If a build failure is due to my code changes, I would like to know first and fix it. I want to know my pending tasks and completed tasks. Something went wrong in the build. How can I know all the change sets from past one week? I have a customer bug. Can I trace between the code and issue and vice versa?
Project manager	Global or distributed team's availability should be known. Scheduling and assignment of tasks must consider the global work days. Work in progress for all the team members should be available. Not sure if we will be able to meet the schedule. An indicated risk is valuable. Depending on the team's load, the tasks should be assigned or reassigned. Therefore, I should know each member's work load. Unit tests and successful build gives confidence on the existing functionality. It is necessary to know the changes between last and current build in an integrated way with requirements and test cases. Project updates should be propagated to the whole team, even if they are in different time zone. Making updates to the project plan is tedious and cost me a lot of time. I need to do a high level presentation to my manager. How can I get the project overview? All the design discussions should be tracked and recorded. Is there a simple way to do it? I want to know whenever a high severity task is raised on this project categorized by the QA team or external customer. This process is very specialized and customized. How can I help the new team member to follow the process effortlessly? My team should concentrate on the productivity rather than spending time on issues tracking, documenting, communicating, and building the software. Can I give role-based access to the development and QA team? It is very crucial to know the time taken to resolve the external issues. I want to categorize the bugs and track them.
QA team	Is there a way to know the release schedule? Is there a way to know the build schedule? If there is a problem with a module, who should I contact? I want to know the stability of the code. Is there a way to check this? Even if I am not given all the access rights, I need to at least have the overview of the project progress.
External stakeholder	I need to get an overview of the project progress. I want to see special reports (such as burn-down chart) based on the current state.

Depending on the organization, geographical location, and technology area, the challenges could be multidimensional. Two projects may not have the same situations and constraints. However, they may have very similar challenges of collaboration and distributed teams in project management. All the mentioned problems in the table can be summarized as follows:

• A team member inception must be easy

• Team members do not want to deal with the process overheads

• Team members need process transparency

• Version control systems must know and understand issues

• Task switching must be easy

• Management needs a crisp overview of project available 24/7

• Tracking of requirements and tasks must be easy

• Replanning must be lightweight and easy

• Team communication must be easy—without overheads

• Development, QA, and management teams must have access to project status

• QA must have access to release schedules and any impediments

• Stakeholders must have a controlled access to the project status reports

  Note

  Tools must know the team, process, and life cycle to aid the project team.