Hello!

Is anyone else a client using the Interactive Benchmarking product?  I'd love to connect and get your thoughts on reporting in CT and see if the issues we're experiencing are similar (unable to download more than one industry peer group, not fitting neatly within the peer groups, using the Archetypes).

Thanks!

Hello @Dan Donalson. We have tried benchmarking on and off over the years, with no clear conclusion.

The Apptio product has not evolved in recent times, at least that I know of. As you hint, it all depends on more or less obscure parameters. We also found that it was difficult to compare anyway.

We even tried Gartner, but we stopped after a few iterations: it was expensive and the results, despite heavy massaging of the data, unconclusive.

By now, I think that benchmarking is an interesting exercise, but only from an intellectual perspective. We couldn't find any practical applications.

Maybe other members out there have a different experience.

P.S.
I still remember the day when I presented some results about Management & Administration: our numbers were much higher (in $$$), than the peer group. Our CIO asked whether he should fire a bunch of his management team... Just an anecdote.

Hello!

Is anyone else a client using the Interactive Benchmarking product?  I'd love to connect and get your thoughts on reporting in CT and see if the issues we're experiencing are similar (unable to download more than one industry peer group, not fitting neatly within the peer groups, using the Archetypes).

Thanks!

Hi, Guillermo,

Some time back there was actually a significant uplift to the Benchmarking product. Although an Apptian, I'm a tough critic, and I was impressed. The statistical analyses are solid and clear. As a benchmarking product, it's strong, plus it's dynamic. It's not a point in time exercise by a consultant firm. If you have an ATUM model with cost pools and towers (and you can add units), you can benchmark whenever you want. Updates are instantaneous if automated or just a minute if manual. It's a great tool.

Is benchmarking useful? Not by itself. It needs to be part of a story. I'll share two I just experienced in the past few months with customers. Both of these customers have standard ATUM models. Both are FSI.

1. FSI invested in a new customer tool which turned out to have issues. Addressing those took significant effort. Software dev was expected to be high, but the customer wanted to see the scale of the problem. Was it within range of other FSIs' software spend? They configured benchmarking in a few days, validated over a few weeks, and were able to see how far out of range the problem put them and where. The details of what they were able to do as a result are not public, but it involved large changes to their outside services and software development.
2. FSI in early stages of TBM. As above, benchmarking was quickly enabled. A demo was given to the CIO at the next business review. They were surprised to see how different their spend model was from FSI in other regions. Like most FSIs, application investment was high. Money is digital today, so no surprise there, but everything else was opposite. Our discussion that followed had the CIO rethinking their IT model and what needed to change to be globally competitive.
3. Bonus third use case from a few years ago, another FSI. Customer was trying to justify a major infra re-investment. With benchmarking, they were able to show how poorly their infra unit costs scaled compared to other FSIs and how uncompetitive that made their services. Requested budget was approved as a result.

Hope those help.

@Dan Donalson, I've always found the key is to find the story benchmarking helps tell.

Hello @Dan Donalson. We have tried benchmarking on and off over the years, with no clear conclusion.

The Apptio product has not evolved in recent times, at least that I know of. As you hint, it all depends on more or less obscure parameters. We also found that it was difficult to compare anyway.

We even tried Gartner, but we stopped after a few iterations: it was expensive and the results, despite heavy massaging of the data, unconclusive.

By now, I think that benchmarking is an interesting exercise, but only from an intellectual perspective. We couldn't find any practical applications.

Maybe other members out there have a different experience.

P.S.
I still remember the day when I presented some results about Management & Administration: our numbers were much higher (in $$$), than the peer group. Our CIO asked whether he should fire a bunch of his management team... Just an anecdote.

Hello!

Is anyone else a client using the Interactive Benchmarking product?  I'd love to connect and get your thoughts on reporting in CT and see if the issues we're experiencing are similar (unable to download more than one industry peer group, not fitting neatly within the peer groups, using the Archetypes).

Thanks!

Hi, Guillermo,

Some time back there was actually a significant uplift to the Benchmarking product. Although an Apptian, I'm a tough critic, and I was impressed. The statistical analyses are solid and clear. As a benchmarking product, it's strong, plus it's dynamic. It's not a point in time exercise by a consultant firm. If you have an ATUM model with cost pools and towers (and you can add units), you can benchmark whenever you want. Updates are instantaneous if automated or just a minute if manual. It's a great tool.

Is benchmarking useful? Not by itself. It needs to be part of a story. I'll share two I just experienced in the past few months with customers. Both of these customers have standard ATUM models. Both are FSI.

1. FSI invested in a new customer tool which turned out to have issues. Addressing those took significant effort. Software dev was expected to be high, but the customer wanted to see the scale of the problem. Was it within range of other FSIs' software spend? They configured benchmarking in a few days, validated over a few weeks, and were able to see how far out of range the problem put them and where. The details of what they were able to do as a result are not public, but it involved large changes to their outside services and software development.
2. FSI in early stages of TBM. As above, benchmarking was quickly enabled. A demo was given to the CIO at the next business review. They were surprised to see how different their spend model was from FSI in other regions. Like most FSIs, application investment was high. Money is digital today, so no surprise there, but everything else was opposite. Our discussion that followed had the CIO rethinking their IT model and what needed to change to be globally competitive.
3. Bonus third use case from a few years ago, another FSI. Customer was trying to justify a major infra re-investment. With benchmarking, they were able to show how poorly their infra unit costs scaled compared to other FSIs and how uncompetitive that made their services. Requested budget was approved as a result.

Hope those help.

@Dan Donalson, I've always found the key is to find the story benchmarking helps tell.

Hello @Dan Donalson. We have tried benchmarking on and off over the years, with no clear conclusion.

The Apptio product has not evolved in recent times, at least that I know of. As you hint, it all depends on more or less obscure parameters. We also found that it was difficult to compare anyway.

We even tried Gartner, but we stopped after a few iterations: it was expensive and the results, despite heavy massaging of the data, unconclusive.

By now, I think that benchmarking is an interesting exercise, but only from an intellectual perspective. We couldn't find any practical applications.

Maybe other members out there have a different experience.

P.S.
I still remember the day when I presented some results about Management & Administration: our numbers were much higher (in $$$), than the peer group. Our CIO asked whether he should fire a bunch of his management team... Just an anecdote.

Hello!

Is anyone else a client using the Interactive Benchmarking product?  I'd love to connect and get your thoughts on reporting in CT and see if the issues we're experiencing are similar (unable to download more than one industry peer group, not fitting neatly within the peer groups, using the Archetypes).

Thanks!