Pitfalls of Process Frameworks

Related Sites

Tough to get specific because people generally don't like having their dirty linen aired in public, so they tend to keep quiet while consultants, etc. are usually under nondisclosure. So all one gets is anecdotal evidence by word of mouth.

I will assume you are talking about process frameworks like CMM, ISO-9000, SPICE, and the like rather than design and implementation frameworks like RUP, because this issue is closer to my heart. B-) Things like RUP are really just a part of a larger process context that the more general process frameworks address. In my experience the most common problems with process framework implementations that go awry are:

Trying to do too much too soon. Process frameworks are usually self-corrective by institutionalizing incremental process improvement. Therefore there is no need to strive to get it right the first time. Since many of the things the framework says should be done are new to the organization, they aren't qualified to know how to do it right. So it is a waste of time for them to try. The golden rule is: do the absolute minimum additional work initially to become compliant.

No understanding of good process documentation. Process documentation comes in two parts. The first is a high level, abstract description of the steps. Preferably around 5 or 6 and certainly less than 10. If more detail is needed, steps can be blown out hierarchically as separate processes. The step description should be what everyone can agree upon as the process description without requiring that anyone has to significantly change the way they do things.

The second part of process documentation is supporting or background material. This describes viable alternatives for particular steps and additional detailed guidelines for executing the step. The idea is to provide guidance but individual projects can tailor things if there is a need to do so. Basically this provides greater detail without introducing controversy in the initial documentation effort -- as best practices are identified later, they can be incorporated into subprocesses.

Lack of a systematic way to define processes. Typically implementing a process framework requires defining and documenting processes. A lot of companies spend huge amounts of effort with committees developing process documentation. These committees wrangle for months over details. There are standard techniques for defining processes within a fixed time frame that should be used. This is also another area for a minimalist approach initially.

Having the wrong goals. Too often goals are tied to schedules and ratings (e.g., Level 3 in 3 years). The purpose of process frameworks is to improve the way software is developed. Any goals and metrics should be tied to monitoring expected improvements in the process that should result from implementing the framework. The corollary is that when initially implementing a framework, one should only implement those portions where there seems to be a clear benefit to be had.

Data and metrics. Process frameworks require the use of metrics and, consequently, the collection of data. Too often companies define metrics first and then collect data for them. This leads to lots of disparate data collection and redundant data collection that gets in everyone's way. A better approach initially is to develop a very basic data collection system that is carefully designed to be non-invasive. Then one defines the metrics based upon that data. Later one can enhance the system when there is a demonstrated, quantified need to do so.

Michael A. Beedle et al., Scrum: An pattern language for hyperproductive software development
(criticizes the principles underlying CMM-like process frameworks)