How are your KPIs? Revisiting how you measure performance

Make KPIs closely relate to maintenance actions that are possible. PHOTO: THINKSTOCK

Key performance indicators (KPIs) are supposed to measure equipment and process performance, as well as costs. But do they? It depends on what, how and when you measure, and how you interpret the results.

Christer Idhammar, the founder of IDCON Inc., reliability and maintenance management consultants based in Raleigh, NC, says many organizations are blinded by the latest dashboards, graphics and information posted with too many KPIs. In most cases these don’t mean much to the people who can impact those numbers. Driving or leading, KPIs need to be closely related to actions people can influence and to maintenance actions that are possible.

J.P. Pascoli, manager of engineering and maintenance at uranium provider Cameco Corp. in Port Hope, Ont., adds that maintenance performance management through KPIs is often not effective because indicators are not visible or understandable at the shop floor level. Many companies may not have the financial resources to invest in high-tech display monitors, and if they do, the content may not be sufficient for specific crew performance measurement and engagement.

Ben Stevens, founder of DataTrak Systems Inc., an asset management solutions provider based in Godfrey, Ont., offers a simple method of sorting out the ones that work from the ones that should be junked: first define the major objectives, then ask some penetrating questions about whether KPIs match these objectives:

• Do they focus attention on a critical status or trend; measure it effectively and promptly; quickly identify a key change in the trend or status to prompt a change in behaviour; and are they easily measured and easily understood?
• Does the measurement, analysis and reporting process properly reflect the performance of the equipment?
• Are the results quickly available?
• Is the measurement and analysis process easy and precise?
• Is it well understood by the maintenance team?
• Is the output message clearly understood by the maintenance team and by management? Do people read and use it?

Predicting failure
If you use mean time between failure (MTBF), know the mean of the failure times and the distribution of the failures around the mean. A sharply peaked distribution will give a more useful prediction because the probability of failure varying significantly away from the mean will be low. But this relies on a number of critical factors: data related specifically to the equipment being examined; consistent definitions of failure; consistent and accurate data collection and analysis; and understanding failure modes, degradation rates, changes in operations and changes in maintenance.

Using MTBF to develop an equipment replacement policy can be very misleading. For example, if your MTBF is 1,000 hours, by definition, half of the equipment will have already failed by the time you schedule the replacement. If you aren’t comfortable with a 50% failure rate, avoid using it.

Is MTBF a useful KPI? Stevens says the answer lies in a follow-up question: what will you do with the results of the KPI? Suppose you measure equipment MTBF = 1,000, and this compares with last year’s number of 950 or 1,050. What action will follow? And if you take no action, why are you measuring it?

Which KPI should be used to measure the success of an EAMS and/or CMMS? Stevens says there are three types of KPIs for a CMMS that should be considered, depending on which stage has been reached in the completion of an EAMS implementation. First are KPIs that show the progress made toward the implementation of an EAMS/CMMS. Next are the KPIs that show how well your EAMS/CMMS is performing. Review the objectives established in the planning and justification of your project. In general, the relevant KPIs will be those that measure the maintenance and materials processes. Third are the KPIs that show the improvement in your maintenance and materials business effectiveness – for example the results of applying a CMMS to your operation.

Biggest improvements
Stevens recommends aiming for value-based KPIs – key performance indicators that measure the value of the maintenance function in terms of the overall value of the organization.

Each company’s KPIs will be different; your selection of the best should be based on company objectives. Focus on where you want to make the biggest improvements. How many KPIs do you want to track? Too many is a waste of time and resources; too few does not give you enough insight. The value of the KPI lies in the actions that it prompts. If the KPI causes no call for action, question whether it’s doing the job.

A chapter titled “Pros and cons of the role of KPIs” in the 2001 book Maintenance Excellence, edited by the late John Campbell and Dr. Andrew Jardine, states management uses performance measurement primarily for monitoring purposes and that many performance indicators have been developed to support operational decisions. They are, at best, descriptive signals that some maintenance action needs to be taken. The book classifies KPIs into three categories: measures of equipment performance; measures of cost performance; and measures of process performance. However, the underlying assumptions of these measures are often not considered when results are interpreted; thus, their value can be questionable.

You need a balanced results presentation to measure performance. The so-called balanced scorecard (BSC) proposed by R.S. Kaplan and D.P. Norton (Harvard Business School Press, 1996) offers a template for this. It translates a business unit’s mission and strategy into objectives and quantifiable measures. BSC has been implemented in many companies. Experience indicates that the BCS’s greatest impact is to drive the change process.

KPIs are a useful tool for improving maintenance – if they are chosen well, measure what you want tracked and measurements are interpreted correctly.

When KPIs are done right, the results speak for themselves.

Steve Gahbauer is an engineer, a Toronto-based business writer and a regular contributing editor.

This article appears in the March 2015 issue of PLANT.