A New Metrics System for IT

How Cisco is approaching the Holy Grail of truly measuring IT value.

The old effort to figure out IT's impact on organizational productivity has new life.

At Cisco Systems, a cross-departmental council has been working to hash out metrics for improving business process operations, according to CIO Brad Boston. In 2003, the council developed metrics to evaluate the efficiency of its online order processing. Less than 30 percent of orders were getting automatically routed to manufacturing, since high error rates necessitated manual input, according to Boston. "We mapped the process of orders coming in and developed metrics. We optimized the process, put new tools in and doubled the percentage of orders that went directly to manufacturing within six to nine months," he says.

A new twist in productivity metrics is the brave new world of business process design and analysis. CIOs are using (and in some cases creating) performance metrics to help their organizations get a better handle on partnerships and service agreements, and to claim (or keep) a place at the strategic planning table. CIOs say that these emerging "value" metrics

Value metrics occupy the intersection of hard and soft performance measures, combining observation and intuition.

1. What value are we contributing to revenue growth,
2. to being able to roll out new products and services,
3. to cut the time to do those things,
4. add to customer satisfaction and retention and cut costs?"

Metrics as Tool used by CIO:
While acknowledging the limitations of productivity metrics, CIOs use them to strategize, budget, weigh outsourcing decisions, run (and terminate) projects and communicate with other senior executives.

Pitfalls of Productivity
Ironically, the preoccupation with productivity can be a drain on productivity. "We don't have a good way to [calculate IT productivity], either to forecast or to measure," Intel's Busch says.

"As a result, we end up justifying and rejustifying, and analyzing and reanalyzing stuff, and coming at it from different angles and trying to persuade people with different measures."

"I don't think personnel appraisals are as dependent on productivity metrics," Busch says. "We measure people on results rather than on unit output per period of time.

"It may sound a little philosophical, but I'm not sure whether you can talk about productivity in a world where you're not actually talking about repeatable tasks," BT's Stockman says.

Productivity metrics are morphing to fit the environment. The key is finding a way to establish a productivity baseline, according to Busch. "I don't think we're very close to it," he says. "It's hard to tie [economic measurement of companies' results] back to individual project activity in a lot of cases. We need something that's much more trackable."