A: “Almost.”
Q: “Why Almost?”
I’m glad you asked. There’s an awesome, albeit somewhat dated, article titled They Write the Right Stuff. The article is about implementing a process to manage software development. In the article, the following metrics are mentioned:
- 3 Previous Builds:
- 420,000 lines of code
- 1 error each
- 11 Previous Builds:
- 17 total errors
Let’s break down those numbers.
Analyzing the 3 Previous Builds
Each of the three previous builds experienced 1 error and contained 420,000 lines of code. The math says the errors per lines of code is roughly 0.00000238 or an error rate of 0.000238%. In engineering, we call that close.
Analyzing the Last 11 Builds
If we apply the same math to the previous 11 builds, the math [17 / (11*420,000)] says errors per lines of code is roughly 0.00000368 or an error rate of
0.000368%. Higher, yes. But still close.
OBM (One Big Metric)
I’m a fan of identifying OBM, or One Big Metric, for comparisons. My favorite OBM story is about whether one should use premium or regular gasoline in a vehicle. Some measure miles-per-gallon, or mpg. I care more about cost-per-mile, or cpm. By measuring, I found premium gasoline had a lower cpm than regular gasoline for my old Jeep pickup (note: there’s more to the calculation than I mention here. If you’d like to correct me, feel free to do so in the Comments section. Warning: You are arguing with an engineer…).
Picking OBM is fraught with danger. In fact, danger surrounds the entire process of statistical analysis. You can find a cool summary of data mistakes here.
As in my Jeep pickup example, I usually choose an economic OBM.
OBM for Almost-Perfect Software Development
In the article, near the bottom, they mention the budget: $35,000,000 per year.
$35,000,000 / 420,000 lines of code == $83.33 per line of code
Math Homework
According to my OBM, how much does your software cost? To find out, divide your annual software development budget by the lines of code currently supported by your team.
If your number is close to $83 per line of code, you can expect an error rate that’s a small fraction of 1%.
If your number is lower than $83 per line of code (and I’m going to guess your number is way lower than $83 per line of code), then you should expect a higher error rate.
Conclusion
Q: Is it possible to reduce the error rate of software development?
A: Yes. But it’s expensive.
Comments