There are projects that are fairly straight forward and relatively predictable, where rational tools of planning and decision making can be applied. Other projects that are more complex or unpredictable call for a different approach relying more on self-organization and innovation. Most software development projects are considered complex and unpredictable in nature due to the convergence of three factors: people, requirements and technology. The various approaches used for delivering and managing projects could be more easily understood in the context of process control models and project complexity.
Process control models
The Defined and Empirical process control models are the two primary methods used for controlling sets of interrelated tasks that, together, transform input into output. Every project is different and will need to be managed from a slightly different place on the spectrum between Defined and Empirical processes.
Waterfall and Agile methods are two of the most prominent approaches to software development.
The traditional Waterfall approach has been based on the Defined process control model, which has defined inputs that produce almost identical outputs every time. It assumes that almost everything can be decided upfront, and that the team just needs to follow and execute the plan. To this end, many Defined processes include such sequential and discrete phases as planning and discovery, design, development, testing, and implementation.
In contrast, teams using an Empirical approach test and retest their plans with evidence of progress so that corrective action can be taken as soon as possible. Agile methods have been based on the Empirical process that is nonlinear and is most suitable for projects where requirements are not known and/or are surrounded by a lot of uncertainty. The Empirical process requires frequent inspections and adjustments. It allows the project team to try new possibilities, test new theories about old assumptions, and ultimately, through innovation, create a better product.
Understanding project complexity
The Stacey Diagram, developed by Prof. Ralph Stacey in the 1990s, is the simplest and most powerful tool for understanding project complexity. It has been adopted throughout the world by many different industries.
The Stacey Diagram combines two axes, one for agreement and one for certainty.
The axis of agreement is determined by how much the team members agree on the topics that are being discussed. The level of agreement on what should be done is a crucial factor in determining project success criteria.
The axis of certainty goes from high certainty to low certainty. When there is high certainty, the team members know how to deal with the problem and can predict the outcome with a "high degree of certainty". At the other end of the certainty scale are decisions that are far from certain. These situations are often complex and new to the team, group or organization. Predicting outcomes in this area "far from certainty" is extremely difficult.
When there is significant disagreement among the team members on what is to be done, but little dispute on how, the team should expect challenges when defining the scope and goals of the project. When there is close agreement on the scope and goals of the project, but a great degree of uncertainty on how to proceed, the team should anticipate conflict when defining the technical approach and timeline.
Projects with high degree of both disagreement and uncertainty fall into the "Complex" segment of Stacey's Diagram. These types of projects and decisions are best suited for the Empirical process control approaches because it makes all three factors of the project (people, requirements and technology) visible, inspectable and adaptable.
Which project delivery approach will give you the best chance of accomplishing your strategic objectives?
Selecting an appropriate approach for a project is one of the most important things that a project team can do to increase the chances of success. Understanding that every project is different, it makes sense that every project will need to be managed from a slightly different place on the spectrum between the Defined and Empirical processes and the two axes of project complexity diagram.
Consider the following questions when deciding which approach is best suited for your project:
How defined are the project's goals and requirements?
On some projects, the client knows exactly what they want and what the goals are. On other projects, it can be difficult to know all of the requirements and objectives. In situations where requirements are difficult to define or likely to change, you need an approach which allows frequent inspections and adjustments.
How clear is the solution for achieving the project objectives?
There are many projects where the desired objectives of the project are well defined, but the steps needed to achieve those objectives are not. When the team is faced with difficult and uncertain decisions around technology and implementation, your approach needs to be flexible to manage the potential challenges.
Does incremental development approach add value to the client?
It's important to know if the stakeholders would benefit from the project being delivered in an incremental way. The team needs to know if the features need to be delivered incrementally in order to add instant real value or is it more beneficial to deliver when a Minimum Viable Product (MVP) is ready.
How large is the project?
You can probably use some aspects of an Empirical process on any project. However, you can determine which approach is most appropriate by looking at the project characteristics. It might not make sense to use the Empirical process to complete a very repetitive project work in a couple short sprints. Instead, you might just use a Defined approach.
Are you going to have a dedicated (full time) resources?
It’s important to know if you are going to have a small tight-knitted team dedicated to the project close to full time or if you’re going to have a large team only partially dedicated to the project.