QualityDev.net

There are many tools available to provide developers with metrics to help determine the quality of the code they are writing. However, there are situations where using tools in isolation, or not understanding the subtleties of the data they are giving you can result in indications that your code quality is higher than it actually is. This is especially true when tools are run over large bodies of legacy code.

This can be best demonstrated using an example:

JDepend is a free tool that ‘traverses Java class file directories and generates design quality metrics for each Java package’ (there is also a plug-in for Eclipse users). It is a good tool that helps detect smells such as high coupling and circular dependencies. No matter how good the tool is though, its results can be misinterpreted.

One of the features of JDepend is to show a graph that represents each package’s ‘Distance from the main sequence’. This metric is an indicator of the package’s balance between abstractness and stability and can help you get a sense of the maintainability of the package in the future. The values range from 0-1, with 0 representing highly unstable packages and 1 representing stable packages. (For more explanation of these types of metrics, see Andy Glover’s article on coupling metrics here.)

The screenshot below shows a selection of packages from the Jedit project (used for example purposes only). The packages with green dots on the graph show a close balance between abstractness and stability, while the one black dot represents a more unstable package that I may want to investigate further.

Narrowing it down, I quickly find the offending package and to appease my boss (who wants all code to have green dots) I can game the results of JDepend by simply adding an empty Interface to the package, thereby increasing its abstractness value which results in the package moving closer to the Main Sequence:

Adding an empty interface to a package just to satisfy a metric adds redundant code, looks ugly and is, well, plain stupid. I don’t believe anyone would actually do this to satisfy a green dot on a graph.

The reason I use this example is this. Imagine running a tool like JDepend over some legacy code to determine whether the code needs refactoring or a redesign for quality purposes (e.g. near the end of the project). If all the packages show a green result on the graph, can you be satisfied that your code is of high quality? Not really, as you are probably only looking for code that fails to satisfy these thresholds, rather than looking more closely for reasons why the ones that pass do pass.

Can we trust these tools to give us reliable metrics on the quality of our code? Is the practice of using these tools a waste of time? The answer to both these questions is a resounding ‘No!’

When using a tool, any tool, you must truly understand the details of what the tool actually does, look behind the raw information it gives and put the results in context. Using JDepend, does a Coupling ratio of 3 mean your code is great quality or full of bugs? How do you know, unless you understand the context of the code and package? Also, this tool is not actually telling you whether what it is reporting is good or bad, but giving you the information to investigate and decide for yourself. This is an important point as sometimes these types of tools are misinterpreted (or marketed) as providing a high/low quality mark, when really they are not. Therein lies the danger of using colored icons such as black, red or green dots.

In this example, it would be better to run JDepend after a static code analyzer (again, properly understood and configured). That way, the analyzer can detect issues such as empty code blocks or interfaces and unused imports, and eliminate them before running JDepend.

One of the most difficult questions I am asked by customers is: “As a manager, what incentives can be used to get developers to pay attention to metrics and hit thresholds I’ve set?” This question was discussed at a round table by a few attendees here at Agile 2007 this week.

One company represented here had offered cash bonus incentives for individuals and teams that hit thresholds for desired metrics. Inevitably, people started to ‘game’ the system and, since there was only one prize offered per month, after the first month the developers found reaching these numbers mundane and de-motivating. The company ended up paying the bonus in a round robin format to try to keep up morale. – Not exactly the desired outcome!

The best answer I can offer to this problem is to appeal to developers’ sense of professionalism, education and pride, in that order.

1. Professionalism: As long as the development team understands why the metrics are being used, i.e. the business value they provide, or the quality process that they are a part of, then I believe developers will support the effort. As professionals, they should understand that their work needs to be measured against organizational or industry standards (especially in a regulated industry), but oftentimes managers do not spend enough time explaining why quality metrics are being put in place.
2. Education: Integrating metrics into a feedback loop provides a great educational tool. Visibility of common mistakes help developers self-improve, and steps can be taken to fill skill gaps. In the long term, that will lead to more job satisfaction. Too often, metrics are not shared widely enough or discussed with the team on a regular basis.
3. Pride: Software Development is something that people want to do! I have never met anyone who was forced into the job because they could not find anything else to do. Most developers want to produce the highest quality work possible, and they are proud of their output. If somebody/something can easily point out how they can improve their craft, the more mature developers will pay attention. As long as the metrics have some credibility.

If a metrics program is put in place, care should be taken to fully explain the reasons why they are being used and what, if any, feedback loop is going to accompany them. If developers are told how these metrics are going to help them in their job, and the metrics are based around proven quality measures, rather than being productivity related, then the metrics are more likely to be embraced. Of all these issues, the feedback loop is the most important. After all, if people feel that the metrics are not being used, why bother collecting them in the first place?

I’m at Agile 2007 this week having a grand old time. On Monday, Bob Martin discussed some benefits of pair programming in his ‘Introduction to XP’. Martin believes too much can happen between when code is written and when a code review occurs, and that it is more efficient to have the review done as the code is being written. Also, he suggests that a daily change of pairs is essential, so that more developers gain more knowledge of the system. This is doubly beneficial because (a) if one person is absent, no knowledge of the system is lost and (b) individuals gain more experience of the different technologies involved, which is motivating and would not occur if a traditional ‘specialist’ methodology was adopted.

In my experience, pair programming is the least practiced tenet of Agile methodologies. I have heard several reasons for not adopting this technique, including:

• Management not willing to pay two people for what they perceive as one person’s job
• Developers do not want to be “criticized” by their colleagues
• Pairs spend too much time arguing about trivial issues regarding coding techniques.

It put me in mind of Alistair Cockburn and Laurie Williams’ paper (PDF) on the subject, in which they concluded that:

• The additional overhead of pair programming was only 15%
• Pair programming resulted in a 15% reduction in defects
• Although that may sound like break even, the time that would have been required to find and fix the defects was 15 times more than the time represented by the loss in initial productivity.

In other words, in their study, they observed a 1,500% ROI.

My take? As professionals, developers will want the team to produce the best quality work possible, and pair programming is a step toward this. Unfortunately, despite the potential ROI benefits, pair programming represents such a radical change to current development practices that I believe that it will remain a minority practice for at least several years.