A cloudy look back at 2010

Today seemed like a good day to take stock of the year in cloud computing, at least according to the view from this Data Center Dialog blog – and from what you as readers thought was interesting over the past 12 months.

Setting the tone for the year: cloud computing M&A

It probably isn’t any big surprise that 3 of the 4 most popular articles here in 2010 had to do with one of the big trends of the year in cloud computing – acquisitions. (Especially since my employer, CA Technologies, had a big role in driving that trend.) CA Technologies made quite a bit of impact with our successive acquisitions of Oblicore, 3Tera, and Nimsoft at the beginning of the year. We followed up by bringing onboard others like 4Base, Arcot, and Hyperformix.

But those first three set the tone for the year: the cloud was the next IT battleground and the big players (like CA) were taking it very seriously. CRN noted our moves as one of the 10 Biggest Cloud Stories of 2010. Derrick Harris of GigaOm called us out as one of the 9 companies that drove cloud in 2010. And Krishnan Subramanian included CA's pick-up of 3Tera and Nimsoft in his list of key cloud acquisitions for the year at CloudAve.

As you’d expect, folks came to Data Center Dialog to get more details on these deals. We had subsequent announcements around each company (like the release of CA 3Tera AppLogic 2.9), but the Nimsoft one got far and away the most interest. I thought one of the more interesting moments was how Gary Read reacted to a bunch of accusations of being a “sell-out” and going to the dark side by joining one of the Big 4 management vendors they had been aggressively selling against. Sure, some of the respondents were competitors trying to spread FUD, but he handled it all clearly and directly -- Gary's signature style, I’ve come to learn.

What mattered a lot? How cloud is changing IT roles

Aside from those acquisitions, one topic was by far the most popular: how cloud computing was going to change the role of IT as a whole – and for individual IT jobs as well. I turned my November Cloud Expo presentation into a couple posts on the topic. Judging by readership and comments, my “endangered species” list for IT jobs was the most popular. It included some speculation that jobs like capacity planning, network and server administration, and even CIO were going the way of the dodo. Or were at least in need of some evolution.

Part 2 conjured up some new titles that might be appearing on IT business cards very soon, thanks to the cloud. But that wasn’t nearly as interesting for some reason. Maybe fear really is the great motivator. Concern about the changes that cloud computing is causing to peoples’ jobs certainly figured as a strong negative in the survey we published just a few weeks back. Despite a move toward “cloud thinking” in IT, fear of job loss drove a lot of the negative vibes about the topic. Of course, at the same time, IT folks are seeing cloud as a great thing to have on their resumes.

All in all, this is one of the major issues for cloud computing, not just for 2010, but in general. The important issue around cloud computing is not so much about figuring out technology, it’s about figuring out how to run and organize IT in a way that makes the best use of technology, creates processes that are most useful for the business, and that people learn to live and work with on a daily basis. I don’t think I’m going out on a limb here to say that this topic will be key in 2011, too.

Learning from past discussions on internal clouds

James Urquhart noted in his “cloud computing gifts of 2010” post at CNET that the internal/private cloud debate wound its way down during the year, ending in a truce. “The argument died down…when both sides realized nobody was listening, and various elements of the IT community were pursuing one or the other – or both – options whether or not it was ‘right.’” I tend to agree.

These discussions (arguments?), however, made one of my oldest posts, “Are internal clouds bogus?” from January 2009, the 5th most popular one – *this* year. I stand by my conclusion (and it seems to match where the market has ended up): regardless of what name you give the move to deliver a more dynamic IT infrastructure inside your 4 walls, it’s compelling. And customers are pursuing it.

Cloud computing 101 remained important

2010 was a year in which the basics remained important. The definitions really came into focus, and a big chunk of the IT world joined the conversation about cloud computing. That meant that things like my Cloud Computing 101 post, expanding on my presentation on the same topic at CA World in May, garnered a lot of attention.

Folks were making sure they had the basics down, especially since a lot of the previously mentioned arguments were settling down a bit. My post outlined a bunch of the things I learned from giving my Cloud 101 talk, namely don’t get too far ahead of your headlights. If you start being too theoretical, customers will quickly snap you right back to reality. And that’s how it should be.

Beginning to think about the bigger implications of cloud computing

However, several forward-looking topics ended up at the top of the list at Data Center Dialog this year as well. Readers showed interest in some of the things that cloud computing was enabling, and what it might mean in the long run. Consider these posts as starting points for lots more conversations going forward:

Despite new capabilities, are we just treating cloud servers like physical ones? Some data I saw from RightScale about how people are actually using cloud servers got me thinking that despite the promise of virtualization and cloud, people perhaps aren’t making the most of these new-fangled options. In fact, it sounded like we were just doing the same thing with these cloud servers as we’ve always done with physical ones. It seemed to me that missed the whole point.

Can we start thinking of IT differently – maybe as a supply chain? As we started to talk about the CA Technologies view of where we think IT is headed, we talked a lot about a shift away from “IT as a factory” in which everything was created internally, to one where IT is the orchestrator of service coming from many internal and external sources. It implies a lot of changes, including expanded management requirements. And, it caught a lot of analyst, press, customer, -- and reader – attention, including this post from May.

Is cloud a bad thing for IT vendors? Specifically, is cloud going to cut deeply into the revenues that existing hardware and software vendors are getting today from IT infrastructure? This certainly hasn’t been completely resolved yet. 2010 was definitely a year where vendors made their intentions known, however, that they aren’t going to be standing still. Oracle, HP, IBM, BMC, VMware, CA, and a cast of thousands (OK, dozens at least) of start-ups all made significant moves, often at their own user conferences, or events like Cloud Expo or Cloud Connect.

What new measurement capabilities will we need in a cloud-connected world? If we are going to be living in a world that enables you to source IT services from a huge variety of providers, there is definitely a need to help make those choices. And even to just have a common, simple, business-level measuring stick for IT services in the first place. CA Technologies took a market-leading stab at that by contributing to the Service Measurement Index that Carnegie Mellon is developing, and by launching the Cloud Commons community. This post explained both.

So what’s ahead for 2011 in cloud computing?

That sounds like a good topic for a blog post in the new year. Until then, best wishes as you say farewell to 2010. And rest up. If 2011 is anything like 2010, we’ll need it.

Thinking about IT as a supply chain creates new management challenges

CA World has wrapped up, finally giving me time to post a few comments about the major product news of the week from the CA Cloud Products & Solutions Business Line: the announcement of the CA Cloud-Connected Management Suite. (You can catch up on the other supporting announcements, the launch of Cloud Commons and the formation of a consortium to create and drive the Service Measurement Index, in my previous post.)

I heard a lot of very positive feedback about CA Technologies’ plans, many from customers, partners, and industry watchers & experts that we’ve been talking to quite a lot for advice and feedback.

However, I think it’s always a good idea to start from the beginning, especially for those who didn’t make it to Vegas to hear the CA execs (or comedian Jake Johannsen, Maroon 5, or James Cameron, for that matter) in person. I think the premise behind this forthcoming product suite warrants a bit of color and background, especially since the implications for IT are pretty significant, if all goes as described.

Cloud computing means IT loses its monopoly

The premise is this: cloud computing enables you to run IT differently than before. IT has had a type of monopoly. It has, for the most part, been a monolithic provider of IT service to support an organization.

Enter cloud computing. As cloud computing matures, it breaks IT’s monopoly. Business users now have a choice: they can certainly go to their IT teams for support with new initiatives as they have been doing. But, they can also investigate SaaS or many different types of cloud services directly.

As in, without IT’s involvement.

I’ve talked about business users “going rogue” in a couple previous posts. Business users are investigating cloud services on their own more and more frequently given the increasing ubiquity of cloud capabilities, their pay-as-you-go models, and the speed you can get something up and running.

As Chris O’Malley, CA’s EVP for Cloud Products & Solutions, said in his CA World cloud track opening session last week (video replay here), “IT faces a rather stark choice: either add value to the acquisition of cloud services by business users, or risk becoming increasingly less relevant.”

Ouch.

Instead, IT can be the manager of a supply chain of services

So what things would IT need to be able to do in order to help business users make the best IT sourcing choices, regardless of what the final answer is? They’d need to do less of what they’ve typically done – manually making sure the low-level components are working the way that are supposed to – and become more of a trusted adviser to the business.

The big shift here is how the business is getting its services: those services are now potentially coming from many disparate sources. And the source of a particular IT service can (and should) change over time.

The result? IT needs to manage – and constantly improve – a changing IT service supply chain. Gartner analysts Ben Pring and Allie Young published a paper last May about these "service value chains," saying they "will be at the heart of cloud services." Their research updated Gartner's model for "understanding the ways in which multiple components sourced from multiple suppliers will come together in the new service delivery models loosely defined as 'cloud computing.'"

"Cloud computing isn't your future," said Forrester analyst James Staten in his more recent post, "it's a new part of your overall IT portfolio." IT will use internal capabilities, external ones, or a smart mix of both. And that is something it can’t manage with existing tools.

To be clear, things like security, service assurance, virtualization management, automation, and other similar capabilities that are extended to take into account cloud-connected environments are still critical (CA Technologies provides a portfolio of solutions for this, as do other big & small vendors). However, they don’t help you understand, control, and constantly improve this new IT supply chain.

Delivering on the new IT supply chain requirements

We decided to do something about this. We started by figuring out what pieces would be needed to make this possible. In other words, what are the new requirements if IT is to take on this role?

O’Malley rattled them off in his CA World presentation. We think the list includes these things:

· Gain insight to compare. First, you need insight into your IT services – both internal and external. You need to discover who is using what, and you need a standard way to describe and evaluate those services, based on important business-related metrics – something that could be used for apples-to-apples comparisons.

· Connect to expert knowledge. Second, you need access to relevant data about cloud services, feedback about those services, and best practices from a community of both peers and experts. This helps you decide if, when, and how you should use cloud computing for a particular application or capability. Customers told us that they need and want a place to share experiences with likeminded people on many, many cloud topics to overcome the huge information gap -- a gap that comes from the rapid changes driven by innovation itself.

· Cloud-enable applications. Third, you need an environment where you can make changes and act on this information. You need to enable both existing and new applications and IT infrastructure to be more flexible by abstracting them from current physical dependencies. This will make it easy to move workloads to an internal or external cloud environment and back again, if you decide that is the right thing to do.

· Challenge your choices. You also need to optimize the choices you’ve made while selecting and implementing available options. You need to constantly challenge and reassess those choices to take into account both new information and changing business goals.

· Deploy, manage, & secure those decisions. Finally, you need to be able to deploy, manage and secure services that leverage the choices you’ve made to meet enterprise-grade, industrial-strength requirements.

Once you have these core capabilities, you need to use them over and over again, making sure you are constantly improving over time. Because in this scenario, the right answer is going to change as time goes on.

As you might guess, we took this list as a good starting point for the capabilities of our new CA Cloud-Connected Management Suite. The suite will have 4 components: CA Cloud Insight, CA Cloud Compose, CA Cloud Optimize, and CA Cloud Orchestrate. For a summary of what each of these will be, you can look at this short paper or the press release. But you’ll notice they follow this list pretty closely.

How do you turn ideas for cloud management into reality?

CA Technologies is now in the midst of making the concept I just described real. We’ve been focused on organic development efforts, and have also found a number of innovative companies in the market with some core technologies to help deliver on this.

For those of you who have been following the string of acquisitions that CA has made in the cloud computing space in the past 11 months, you can probably see how this suite weaves together technology from the acquired companies. You’ll recognize core capabilities of Oblicore Guarantee as part of CA Cloud Insight, 3Tera AppLogic at the heart of CA Cloud Compose, and Cassatt in CA Cloud Optimize.

But, these technologies alone don’t fully flesh out the requirements I listed. Nothing in the industry does yet. That’s why, even after the acquisitions, there’s a bunch of development that’s on-going to deliver what customers need.

Understanding the IT supply chain: using Cloud Commons and the Service Measurement Index

To help get a handle on the components and management of this IT supply chain idea, we believe that IT needs not only a new set of product capabilities, but also a place to do research on the many cloud services and capabilities – and a way to measure them. Thinking it through, in order for customers to get what they need out of those functions, it seemed much smarter to have them exist separate from CA.

So, we set up Cloud Commons as an independent community and website. And we collaborated with Carnegie Mellon University to get the Service Measurement Index (SMI) started, and then placed SMI in the stewardship of their consortium, so it wouldn’t be tied to CA alone. These moves hopefully get the ball rolling, helping make both useful to the industry as a whole, regardless of whether someone is a CA Technologies customer or not.

But, put all these pieces together – a community of expert information, a standard way to evaluate cloud services, and this new suite of products – and you have the beginnings of a new set of tools and capabilities to help IT manage this new supply chain. And, in fact, a new, expanded role for IT.

Where are we now? Both potential and pitfalls

CA World was our first stake in the ground. The industry has been wondering how we were piecing all of our acquisitions and other work together. We intended the announcements and various sessions at CA World to at least provide the strategic framework for what we’re doing.

Now comes the hard part: delivery. We’re back in our offices, hard at work on that, only a little sad not to still be in Vegas (a week there is quite enough, though, I have to tell you).

Meanwhile, customers can get started with things like the Oblicore and 3Tera products now, giving them a taste of what’s to come, with a convenient upgrade path to the new suite later.

The first deliverables (CA Cloud Insight & CA Cloud Compose) are due by the final quarter of the calendar year. You can be sure we (and I) will keep you posted on our progress.

Until then, we want, expect, and welcome lots of feedback on every piece of this. It’s probably not what people were expecting from CA, but that’s just fine with me. Please feel free to chime in.

A new way to compare your cloud computing options

If cloud computing does what everyone is saying it’s going to do, organizations are going to end up with many, many ways to get the IT service they need to support their business. And choice is good.

But having thousands of choices with no clear way to decide (or even prioritize) is not a recipe for success. Three of the announcements I’m helping with at CA World in Las Vegas this week (#caworld on Twitter) are aimed at addressing that problem: finding smart ways for companies to make those choices – and ways to constantly challenge those choices. Read on for a bit of detail and commentary on the first two; I’ll post another blog after Chris O’Malley’s Cloud & SaaS Focus Area Opening to cover the third.

Answering the bigger questions

These announcements are the result of quite a bit of soul searching at CA (now called CA Technologies, a hint that significant change is afoot) about how the company could and should help fill in some key missing pieces for really getting utility out of cloud computing.

What we saw and heard from customer after customer was that there are a number of very important issues around the cloud, issues that everyone’s been talking about: security, assuring performance and availability, managing and automating underlying technologies like physical and virtual servers. And, to be sure, we (and others) have answers to a lot of those that are getting better and better all the time (including some announcements from us coming this week).

However, there was a bigger problem not getting answered.

Business users are realizing that the IT department is no longer the single source for delivering the IT service they need. They can go around IT. Or at least use cloud services, SaaS offerings, and the like as pretty strong bargaining chips in the negotiations with IT.

And IT has had a pretty difficult time putting what they can deliver side by side with what can be sourced from the cloud and doing a fact-based comparison so the business can make the right decision.

We also realized that there are things that CA Technologies could and should deliver to help answer some of these questions, and things that are better driven by others with relevant expertise and experience. So we are doing a mix of both.

First, a standard way to measure IT service in business terms

Sunday afternoon, Jeff Perdue, a senior researcher at Carnegie Mellon University, and CA Technologies’ David Hodgson announced an effort to provide the industry a standard way to do these kinds of IT service comparisons. The result is the beginnings of the Service Measurement Index and a consortium to support it.

The Service Measurement Index (SMI) is the first of its kind – a relative index focused on describing IT services in business terms. Instead of looking just at cost, SMI provides a way to balance between six important characteristics: quality, agility, risk, cost, capability, and security.

CMU and CA Technologies got this started, but CMU will take the lead from here on out, pulling in educational institutions, global end user organizations (both commercial and public sector), and technology vendors. There’s no way this is something a single vendor could pull off, and CMU has the chops to help make this consortium -- and its work -- real.

And, even if you’re not a big user of cloud computing services now, SMI can be used to rate your existing internal services. Given that ways to do apples-to-apples comparisons are hard to come by, that should be good news.

There is a panel this week at CA World with lots more information about SMI and the consortium; I’ll forward (and tweet) more details as the week progresses.

Second, a community and website for qualitative & quantitative feedback on cloud services

Hodgson also announced Cloud Commons on Sunday (I need to have a word with whoever’s doing this wacky announcement scheduling). Cloud Commons (http://www.cloudcommons.com/) is an independent community that CA Technologies is launching and supporting, again with the help of a lot of partners, that is now open and available for use. It’s intended as a place for cloud fans and skeptics alike. It’s for end users, technology providers, industry experts, and others to share experiences, best practices, and qualitative & quantitative information about the many types of cloud services available.

Cloud Commons will use the Service Measurement Index as one way for people to describe their experiences with cloud services and compare them to others. There will also be ways to add commentary and to interact with others in the community. Think of it as a Consumer Reports for cloud services, except you and your peers are the experts providing input.

Since this is going to be a living, breathing community, much of the content that will make Cloud Commons useful will come directly from the specific input of many, many IT folks. Knowing that we have to start somewhere, we got the ball rolling with a broad range of current and historical data gathered for major cloud sites and starting-point SMI data from an extensive research project with a leading analyst firm on the characteristics of particular cloud services (including e-mail, e-commerce, and a few other common examples).

But frankly, what you’ll see up there today are the bare bones. I’m expecting that the most interesting content started trickling in tonight when customers on the CA World show floor started registering and filling in survey data about what they are doing with cloud services – and their opinions on those services, good or bad.

In the press conference today, Hodgson was asked how Cloud Commons could possibly provide all the right information. “This isn’t CA doing this, this is the community,” said Hodgson. The site isn’t going to be perfect right out of the gate, but it will morph and grow with feedback, he believed. And that’s actually a strength. “Some of the answers will come from the community – what do people want to see? Just the fact that it’s going to exist as a concept will start the conversation that the industry needs.”

Next up: new CA Technologies cloud management solutions

The third cloud-related announcement is about a set of products that we have up our sleeves that Chris O’Malley will announce from stage on Monday. I’ll post on that after it happens.

Until then, take a look at Cloud Commons, dig into SMI, and give feedback both on cloud services and on the concepts I’ve been talking about here in general. That’s what will make all of this useful, after all.

Also of note: if you are interested in being a regular, more official contributor to Cloud Commons or would like to make sure your cloud services appear in the Cloud Commons marketplace, drop me a line. And to reiterate, we’re very serious about everyone being welcome: partners, customers, competitors, industry analysts, pundits of all types. Definitely take us up on the offer.