IT metrics: a snore? Not when cloud computing is involved

Amazon’s recent announcement about the availability of spot pricing for EC2 instances got me thinking a bit about the role measurement is going to play in cloud computing. In fact, I sat in on a few sessions at the Gartner Data Center Conference earlier this month focused exclusively on metrics.

A snore? Actually not.

The increasingly dynamic world of cloud computing is going to require (in fact, completely rely upon) a robust and trustworthy set of metrics for how things are going. So much so, in fact, that I’d expect a bit of a renaissance in this space.


And while Amazon’s market-leading moves point to all sorts of requirements to measure costs, availability, and other requirements relating to their EC2 offering, many IT shops are thinking much more locally: how to get going on their own private compute clouds. IT metrics & measurement are going to get more and more important for those IT shops, too. Getting the right metrics working in a way that helps you make IT decisions is a really important step. Sometimes you’ll be impressed with all sorts of useful data about your infrastructure. Just as often you aren’t going to like what those metrics report back.

Here are a few additional thoughts about using metrics to help manage IT, using the Gartner metrics sessions from Vegas as a jumping off point:

An interesting conundrum for metrics: will they help align IT & business – or make things worse?

Gartner analyst Milind Govekar spent an entire session at Gartner’s conference (“Improving Your IT Business through Metrics”) talking about ways to manage IT, and ways to use metrics to that end. Not that this is new. In fact, one of the sessions that Gartner seems to run every year is something about how misaligned IT and the business are…or how best to get them aligned…or how they are moving closer. But I heard a few interesting wrinkles this time around on how measurement should/could be used:

· Without a certain level of IT operational maturity, investment in infrastructure – and measurement of that infrastructure -- is not going to yield great results. Govekar pointed out that majority of the customers they talk to are about halfway through a journey toward more dynamic, adaptable IT. And that’s about the point that the organizations can see a measureable pay-off from the investments. “That’s where you [the IT department] start to make a difference as a trusted service provider,” Govekar said. So, that also means you should expect the journey up to that point to have some bumps. Several analysts noted a need to improve process maturity to make true IT operations progress. And, of course, Gartner conveniently gives you a way to measure where your particular shop is on that journey.

· Metrics can and should be important, but be careful. Choose wisely. Govekar put it very succinctly in his session when he said,“Be careful with metrics: they drive people to do good things; they drive people to do bad things.” As we (hopefully) learned from Wall Street last year, people optimize around what they are measured and compensated for. Make sure it’s the right thing.

· Organizational patience is key. Govekar listed senior management commitment, persistence, and patience as very important to efforts to use metrics to drive IT change. An organization without an interest in continual improvement is not likely to be able to grapple with what the metrics will reveal – and the cultural changes that are necessary to get to that more dynamic IT they think they want.

· You have to be working at a level that business people will care about in order to have any chance of getting the ear of the CEO. The Gartner conference keynote from Donna Scott and Ronnie Colville focused on this practical truth: if you can’t talk to your business folks about ways that IT can help them improve what they’re trying to do, you’re going to have a difficult path to getting your job done. And, your CIO is not likely to be seen as a full partner in the business. While “technology may be used to help innovation,” said Scott, “people and processes are the backbone of innovation, not technology.” Good point. All those virtual machines may sound cool, but not necessarily to everyone. And whatever stats, facts, and figures you do end up with, don’t expect them to be as meaningful to the business set as they might be to the IT folks.

In one of the more interesting sessions about metrics, however, Gartner began to paint a picture of some of the innovative things you can do with them

Gartner analyst Joseph Baylock had some intriguing thoughts in his keynote about ways to use patterns from your infrastructure & operations data to improve IT’s sensitivity to the business. And, if the fancy brochures I found in the lobby are any indication, it sounds like Gartner is going to be spending much more time and effort thinking this through.

Baylock talked about creating an IT operations model that seeks out patterns, applies them to models, and then adapts accordingly. What could this mean for IT? Probably nothing quite yet. However, for companies with performance-driven cultures, Gartner saw this as a way to move from using lagging indicators for managing IT (and, hence, your business) to using leading indicators.

So, not only will you want to be able to measure what you're using at any given time (from Amazon or whomever else externally – and from your own internal infrastructure), but perhaps you’ll want to start to get smart about sifting through the mounds of data to make good choices.

Sounds to me like it’s worth keeping an eye (or two) open -- and on this topic -- after all.