Critical Assumption Planning – Case Study
Major Australian Telco benefits from using CAP
During late 1996, the Critical Assumption Planning process was introduced into the Product Development Group within a major Australian Telecommunications Company. Listed below is a whitepaper outlining this implementation. Comments from people using the process at this company can be found here.
Background
A Major Australian Telecommunication Company utilised the CAP process in late 1996 to improve the product development process.
Prior to the implementation of CAP, very few product developments were ever rejected, and few were presented to senior management with any level of risk management. Consequently some products failed after introduction into the market.
The Manager of the Product Development team learnt about the use of CAP at BellSouth Business Systems and was the champion for the introduction of CAP into this major company. Following the introduction of the process to the Product Development Managers, Senior Managers (including the COO) responsible for approving capital expenditure for new products were trained on the process, and then started to utilise CAP analysis tools in every new product development.
A culture of “NO CAP – NO MONEY” was developed so that every opportunity utilised the CAP process with no exceptions.
The result of the introduction of the CAP process meant that now some product developments were “killed” well before they reached the market – releasing valuable development resources and funds for other projects.
In the 6 months following the introduction of the process, it was estimated that more developments were reviewed and approved for development than in the previous 19 month period.
Two important lessons from the introduction of CAP were
1) Ensure Senior Management are trained on the process and actively utilise it in ALL investment decisions
2) Establish a “CAP review team” to support other developers and ensure CAP analysis is of a uniform quality.
Why CAP was introduced?
Prior to CAP, only one of some 100 products developed had ever been refused funding. There was no “kill culture”. It became evident that there were products that were being developed and launched to the market that carried an unacceptable level of risk. There was no uniform process in place to ensure each project was de-risked. As each product development could involve between $1M and $40M of funding, there was an urgent requirement for a universal tool to provide a thorough and rigorous analysis of all opportunities across the business. Importantly, all stakeholders in the development from Sales, Marketing, Business Review and Capital Planning had to understand the CAP process as active practitioners to ensure an accurate assessment of each CAP analysis was being conducted.
Product Development
At the time, Product Development was a part of the Marketing Division, with approximately 30 Developers. A deliberate decision was made to introduce CAP into the product development area, as the company get it working in one area before it was applied it to other areas of the company.
Initially, the Development staff were trained at two separate sessions over a 3 month period. New developments were utilised for the training so that real and relevant examples were used in the training.
Steve Hogben, Manager of the Product Development Group commented “We planned training of development staff as well as training of management and we looked at a period of 6-9 months to implement. It has allowed us to improve the business cases of the product developments we’ve taken on, and it has allowed us to do more development work. In the first 6 months of the CAP implementation, we probably delivered more products to market than we had in the 12 – 18 months before, and I’d put that down to CAP.”
The CAP training was extended to Senior Management following the 2 sessions of developer training. A CAP training manual and CAP template were produced to ensure a uniform quality across all developments.
Execution
As expected, the first few CAP analysis were not as complete as those that were developed over the coming months. Both the Senior Management Gatekeepers, as well as Product Developers had to adjust to a new process, and the implications of finding fault with their development assumptions by utilising the CAP process. Some developers, and their project sponsors (generally a sales team, eager to meet sales targets) tried to push forward without a complete CAP analysis. These developers were turned away from capital investment meetings.
After a short period of time, other areas of the company not required to develop a CAP analysis were learning that the only way a project would even be considered for capital funding was if it contained a complete CAP assessment. This helped propagate the “CAP culture” throughout the company, and many other areas such as Sales and Technical groups started to request CAP training materials so that they could also be involved in the CAP analysis.
One other side effect of the introduction of CAP was that the negative impacts of a “killed project” were removed. Developers that had become emotionally attached to a project and therefore were reluctant to terminate the development started to utilise CAP to rationally review their project. Developers were rewarded for “killing” projects that were unable to produce an appropriate CAP analysis, or where the CAP analysis indicated that the element of risk was unacceptable to the business.
Other areas of the business (such as sales) that had traditionally applied pressure on developers to launch products “no matter what” soon became aware of how CAP was being used at all levels of the company to identify and manage risk.
Senior Gatekeepers embraced CAP as it allowed them to better utilise the scarce allocation of capital for new developments on those more likely to be successful in the marketplace. Word soon filtered to other parts of the company about how only healthy projects would make it through the CAP review and gain funding approval.
CAP team
To ensure each CAP analysis was of a uniform quality, a group of developers were taken from the training with the best appreciation of the process and set up as a support team to support the other Development Managers to review CAP submissions. This was so that Product Development could be seen to be producing CAP work of uniform quality no matter who produced the work. This became a critical part of gaining acceptance of the process. Each CAP team member became an expert CAP practitioner, and a CAP mentor in the process. One of the CAP team members was subsequently invited by BellSouth Business Systems (BBS) to develop a training program, and deliver it to BBS developers in Atlanta over a 3 week period.
Senior Management
The product development process utilized was a stage-gate process consisting of 4 key stages
Concept
Justification
Planning
Implementation
At each stage, a presentation had to be made to Senior Management Gatekeepers to supply additional funding (or have further funding denied and the project stopped). Overlaying the CAP process at each stage ensured that the iterative approach offered by CAP could be fully utilised. Pre CAP, each stage offered incremental funding to get the product to market. Post CAP, money for validating CAP test cases was allocated to ensure that proper CAP process was followed.
Results
The introduction of CAP was seen as a success, as it allowed Product Developers, as well as Senior Management to have a more accurate view of the risk and viability of each development.
Comments from various people in the company confirm the benefits of CAP.
“The CAP process has been a tremendous benefit…We’ve used the process religiously for all investment decisions that we’ve made and it really has been tremendously beneficial – a real eye opener. One of the real benefits we see from it is that if you really sit down in a structured way and assess the risks associated with new products and projects, it really adds an element that you don’t really see if you run more on gut feel and just the financial implications”. – Chief Operating Officer
“Prior to CAP there was no formal process relating to risk…we assumed that all products had the same risk profile which was very unrealistic. With CAP we much more sophisticated in how we look at our risks. CAP gives me a lot more information than I would normally be provided, and have to assume when assessing a new project.” – Capital Planning Business Analyst and Key Gatekeeper Advisor.
“It [CAP] has allowed us to improve the business cases of the product developments we’ve taken on, and it has allowed us to do more development work. In the first 6 months of the CAP implementation, we probably delivered more products to market than we had in the 12 – 18 months before, and I’d put that down to CAP.” – Manager Product Development.
The real key to the successful implementation of CAP was the adoption by Senior Management from the COO down, and their strict adherence to the CAP process.
About the Author: The Author was a Product Development Manager with the company mentioned in this case study and was involved in 5 products that underwent a full CAP analysis. One project had a value of A$44M, and was one of the first to be approved with a full CAP analysis.
Read more about the CAP process here.
Critical Assumption PlanningSM is a service mark of D. Dunham & Company.