In 1979, Domino’s advertised their famous, “30 minutes or free” guarantee. In 1993, Dominoes owner, Thomas Monaghan ended the guarantee citing perceptions that the policy encouraged the pizza delivery drivers to speed and drive recklessly. While restaurants offering food delivery learned their lesson from Domino’s, the idea that a customer can expect their order in about thirty minutes persists. I worked with a global chain of pizza restaurants years ago that felt pressured by what they referred to as the customer’s “30-minute mindset.” They wanted to speed up their operations, but they did not want to hold the drivers accountable to a standard that might encourage unsafe driving. Instead, they developed a standard for the time it took from an order being placed until the pizza was sliced and in a box waiting for the driver; the company referred to the span of time from order to box as the “in-store time.” They reasoned that the overall wait time for a customer could be reduced through a focused effort to reduce the in-store time.
The company created a bonus for restaurant managers who kept their overall in-store time below a certain number for a full month. After the incentive program had been in place for several months, a few district managers began noticing some unusual patterns in the various reports they received each month on restaurant performance in their respective districts. Some of the restaurants that had the lowest overall in-store time also showed a decline in sales. At the same time, the call center that logged complaint calls from customers had noticed that the complaints about late orders had declined, but the complaints about poor service when placing the order had increased.
When the district managers visited the restaurants with the best in-store times they noticed something disturbing. At the busiest times, the employees responsible for taking phone orders seemed to be ignoring customers who were on hold waiting to place their order. The district managers could see the blinking lights on the phone indicating that someone was waiting to place an order. It turned out that some of their restaurant managers had directed their staff to stop taking orders if the production line got backed up. A pizza oven can only handle so many pizzas at one time and you can’t speed up the cook time for a pizza. If the oven can’t accept any more pizzas, it creates a bottleneck and pizzas ready to go in the oven have to wait while the in-store time clock ticks away. If you want to ensure that the oven does not get backed up, just slow down the number of orders coming in.
The story of the pizza company illustrates a common organizational trap that I would summarize as the confusion between measuring for improvement and measuring for control. When a person purchases a fitness tracker and begins monitoring the steps they take each day, they want to measure something for improvement. If your car insurance company offers you a discount if you install a sensor that monitors your driving habits, they might advertise it as a way to improve safety, but they also want to control their costs. When you measure for improvement, you put the data in the hands of the people who can make use of it to change for the better. When you measure for control, you use the data to make comparisons that have consequences imposed on people by others with authority.
When I am a student and you grade my test, I learn how well I have mastered the subject. If the test score is only for me and not used for any other purpose, then it is a measure for improvement. If an administrator’s bonus is linked to my performance on the test and my score becomes a data point in the school district’s rating and the school district’s rating becomes a data point in determining property taxes, suddenly there’s a lot more than improving my mastery of the subject at stake.
The distinction I am making between measuring for improvement and measuring for control is elegantly explained by a principle known as Goodhart’s Law. A popularized statement of the law developed by the economist Charles Goodhart goes like this: When a measure becomes a target, it ceases to be a good measure. Goodhart’s original formulation from his 1981 paper on monetary policy in the United Kingdom is less pithy, but more consistent with the distinction I am trying to make. Goodhart wrote, “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” Consider the law in the context of public school test scores. One might argue that putting pressure on teachers and students to meet test score performance targets will moderate the degree to which the test scores can be used to reach valid conclusions about what students are learning.
Measurement is the tool of choice for managing performance, yet we rarely stop to question whether or not we are paying attention to the right measures, or whether the very act of measurement creates dysfunctional behaviors. Through critical inquiry, we may discover that we have been equating success with what is easiest to measure rather than equating success with what we want to accomplish. Because of our addiction to managing through measurement, we find the closest quantifiable substitute for what we really want and then focus on measuring the substitute.
Eventually, the focus on the measure replaces a focus on the goal that the measure stands for. For example, we don’t want a society comprised of people with high grade point averages; we want a society comprised of well-educated citizens.
What’s worse, when authority figures incentivize achievement of a metric that substitutes for what we really want, Goodhart’s law kicks in and we end up with data tainted by people gaming the system.