Benchmarking

Benchmarking - why do we do it? It should help change the conversation from IT cost, to IT value. It should also help the organisation decide what service it wants and needs, and to have meaningful discussions between service levels and price. Low cost and high service levels tend not to match!

So, how best to do it? Starting point is usually, we're unique, can't compare us to others. But, have to start somewhere. Select a peer group. Choose a reference group close to your volume and complexity. Gartner has a model for doing this, taking into account various drivers and using statistical techniques to remove outliers etc.

Can use just key metrics, eg how much do I spend compared to others? But, get more value if you go deeper, and look at specific areas and analyse data more.

But, this is still concentrating on cost, and there is more to benchmarking than cost. I'm interested in things like satisfaction, innovation, quality etc.
Should we start with cost, and then look at quality? But how do we measure quality? Picking the right KPIs can be difficult. I remember talking to the CEO of JANET about this, and he made the point that he could measure up time of the JANET network and get it to 99.99%. Very good. It can fail for 1 hour, to 1 institution and be well within those figures. But, if that hour was at the beginning of Clearing, it could seriously affect that institution's satisfaction.

If you just look at spend, you have to take into account all of the differentiating factors. Eg spend per student would depend on what services you offer, whether you're a research University, what your mix of courses was.

So, benchmarking is a big challenge, the trick is to get meaningful data. And meaningful data to compare it to.


- Posted using BlogPress from my iPad