Dumb metrics

I am generally a fan of metrics, but I know metrics are very limiting and only tell part of a story. My general rule of thumb for metrics is that if you measure and report it, then you need to be prepared for the number to be gamed in some way or form.

There’s also a lot of what I call “dumb” metrics for lack of a better term. These are metrics that are reported to measure one thing, but end up have negative effects that outweigh the benefit of measuring that one thing to begin with.

One example of this was the infamous metric, “utilization” often used in large consulting and accounting firms. I spent 5 years at one of these firms and while my experience was largely positive, one of the silliest things that the firm did was put heavy weight on utilization.

In short, utilization is a measure of billable client hours over the total amount of hours you are working. It’s intended to measure how much one contributes to client service (aka revenue) of the firm. Now the metric is innocent enough as on the surface you’d want people who are highly utilized, i.e. generating revenue for the firm.

In practice, this metric ended up being gamed, caused burn out and/or ended up being the luck of the draw.

One metric at review time was always utilization. Given that your promotions and raises were made based on utilization and given that you self-reported time, what ended up happening was people would exaggerate the amount of hours or bill the client more than needed. Yes, it was unethical, but it was such common practice in these firms that nearly everyone did it.

Those who were chasing high utilization ended up slaving away and destroying their work-life balance in chase of this magic utilization number. It was never a good outcome to watch someone be 200% utilized (~80 hours a week) consistently.

Lastly, you had the bunch that just had difficult clients or were on terrible projects that crashed your utilization rates. Sometimes we would have our best people on these projects but they would never get promoted because their utilization was too low.

This “dumb” metric has led to thousands of people quitting the firm, but yet to my knowledge, the firm was still using it to track development. At startups, you need to make sure you’re not tracking dumb metrics. There’s no time for these metrics where the downside outweighs the benefits for your staff and team.