Measuring Programmer Productivity

In my role as CIO at CareGroup and Harvard Medical school, I oversee nearly 100 software developers. Many organizations purchase software or outsource development to avoid managing developers, but we Build AND Buy and will continue to do so. I was recently asked to share the metrics we use to evaluate our developers. Here is our framework:

1. How may applications does the developer maintain? Applications can vary in size and complexity, so that must also be documented. This helps us understand how many staff are needed as our application portfolio expands.

2. How many new applications can a developer create annually? Some programmers develop version 1.0 of new applications, while others maintain existing applications or modules. This helps us understand our development skill set and our ability to take on new development projects.

3. What is the lifecycle stage of each application assigned to a developer? (New, mature, need to replace within 2 years). This is a reasonable way to measure work completed and forecast upcoming work.

4. How much bug free code is developed per year? This is an imperfect measure, since it measures quantity, not quality, but it is useful to understand as a proxy for coding productivity.

5. Are the application stakeholders satisfied with the quantity and quality of application features developed?

We piloted these measures at Harvard Medical School by

1. Inventorying all the major web applications by developer, including the technologies used.

2. Categorizing this inventory into applications which are new and those which are in maintenance. We also rated each application for intensity of support (High, Medium, Low) based on the frequency of code change that has been historically required to maintain it

3. Rating each application's lifecycle stage.

4. Documenting the number of application releases by each developer over the past year.

5. Creating a survey to measure user satisfaction with each application.

In the end, we published a detailed scorecard for each developer and a summary for all developers. This data alone doesn't tell the whole story, but it does help us plan and better manage our development staff.