Why cloud computing hype isn't bad for IT after all

A week or two back, ReadWriteWeb ran and published the results of a reader poll of the “most over-hyped cloud technologies.” Amusingly, the results (aside from a NoSQL mention) read like the basic NIST definition of the key components of cloud computing. Software as a Service, private clouds, Infrastructure as a Service, and Platform as a Service all made the top 5.

Wow, I thought. That barely scratches the surface. Plenty more cloud computing terms were enjoying their moment of irrational exuberance, but were being left out in the cold by this particular survey. A few Twitter conversations unearthed some very deserving nominations. Not to be forgotten:


• Hybrid clouds. Apparently hybrid clouds didn’t have quite enough hype-y-ness to make the list. Weird, considering that term tends to be the punch line to nearly every cloud strategy and direction conversation that I hear. Better luck at next year’s awards ceremony, I guess.


• Cloud bursting (nominated by @reillyusa and @AaronMotsinger). Some folks have been arguing back and forth about whether it is really a legitimate (or even possible) use case. @pdowning1077 noted he much preferred the term “capacity on demand,” but that doesn’t help settle the argument.


• Cloud brokers. Forrester has been posting some interesting research for its subscribers on this new role (also defined by NIST in the July 5 version of its standards roadmap, if you want a standards org to weigh in for legitimacy). I’d say this conversation is still very early. The hype wagon train for this term has just set off down the road.


But probably the most impactful comment was another by @pdowning1077. “How about just the term ‘cloud’ [in general]?” he asked. How could they forget to include the mother of all hype-worthy terms in their polling?


So much hype that “cloud computing” becomes meaningless?


The same week of all this discussion, David Linthicum reported that cloud computing (the term) had now become essentially “meaningless.” That comment came on the heels of Gartner’s annual publication of their hype cycles. A quick scan notes that cloud computing is still close to where it was last year, just nosing over the (hype-laden) peak of inflated expectations. Private cloud computing is rapidly moving to join it, perched perilously over the trough of disillusionment, ready to take the leap.


OK, no one would argue with the extreme levels of marketing attention from everyone from start-ups to 30-year-old software companies (who, us?) to service providers. But just because a bunch of marketing people are in a frenzy doesn’t mean we should write off the trend they are talking about as a bunch of meaningless fluff.


The hype has caused IT to pay attention to cloud computing


In fact, if I’m reading the market right, I’d say that there are actually some really good things that have come out of the hype around cloud (and continue to do so).


• We suddenly had something to call this good idea. There were a bunch of technologies and entrepreneurs out there struggling for several years to put a palatable name to what they were working on. Some started off calling this grid computing, some utility computing, and others more obscure terms than those. But, the early hype around cloud computing a few years back gave a name to the idea. We pulled several of these companies into CA (Cassatt and 3Tera, to name two), but many others were struggling with this same issue. One of my early posts on this blog was about how the term private cloud may not have been precise or perfect, but it enabled us to have the right conversation. I think the same thing goes for the overall cloud computing concept.

• It created a way to catch the attention and imagination of enterprise IT. By talking about a Big Vision of IT infrastructure that matched compute supply with compute demands at any given time (and matched costs accordingly), ears perked up. It was the next logical topic to discuss with the IT guys who were fresh from thinking about how virtualization could free them up from particular pieces of hardware. In a world in which IT is fighting for every budget dollar, mostly just to keep treading water, an idea about how to get off this downhill hamster wheel is at least appealing to consider. That’s step one. (Ken Oestreich, by the way, has a great blog from a few months back on the brief history of the vision of cloud computing.)

• The hype extended the discussion past the technologists to the business people. All the hubbub over cloud computing got the business users excited at a time when the economy was giving them little to be excited about. “So, you mean I might have a way to turn some of these business ideas into reality, despite the drubbing that the sour economy has given us and the measly budget that my IT partners say we have at our disposal?” This has been important – the business guys are the ones, in the end, pushing when IT starts to get nervous and pulls back from the visionary edge that cloud puts them on.


• The hype has pressured big vendors into some self-reflection that will be beneficial for their customers. Many of the larger vendors jumped on the cloud bandwagon through new offerings, blatant rebranding of old offerings (shame on you), acquisitions, and the like. To make any of these moves, vendors have had to take stock and rethink what they can and should be providing given what their customers want. In some cases (like here at CA with Nimsoft), it causes the vendors to broaden the set of customers they are actually serving.


• The intense amount of discussion has started an intense amount of scrutiny, revealing how useful cloud can actually be. One thing that happens when the hype levels reach fever pitch is that people start pushing back. The recent demand for real-world examples and exasperation over cloud outages has been the natural backlash from being force-fed lots and lots of best-case scenarios, rainbows, and unicorns. Journalists and analysts have often helped push for these kind of reality checks, though they also tend to pile on as technologies or ideas drop into the “trough of disillusionment” that Gartner is so fond of describing. Enterprise IT, business users, and the vendors themselves all eventually do a fair bit of policing, sometimes too late for their own good, but we seem to be headed in this (positive) direction right now.

So while a lot of the hype can seem like so much wasted energy from all parties, when the trend or shift being hyped actually has merit, something useful comes out the other end. Now, would

most of us (advertising agencies and ad reps excluded) prefer some way to skip the aggravation of this process and jump right to the end? I’d bet so. However, consider this all a bit of a trial by fire. The only way for something to be proven strong enough to pass through the fire is, well, to actually do it.

So, hold your nose and smile. Hype is good – with a few important caveats. Be critical. Be well-armed with the right questions to ask in order to discern the valuable from the merely fancifully over-marketed. Be ready to see the value in approaching something a new way, even if it’s something you’ve done the same way for decades. Be pragmatic enough to know it won’t happen overnight or with the wave of a magic wand.

If it makes you feel any better, cloud computing isn’t the only term getting the Gartner Hype Curve treatment this year. Added to the list, according to this ReadWriteWeb article, were big data, gamification, Internet of Things, and consumerization. Misery loves company, I guess.


And, in the meantime, it may be time to come up with your own term to start campaigning for next year’s Cloud Hype Awards. I think the hype is here to stay for a while longer.

Why it pays to be early -- especially with this much cloud choice

It pays to be early.

Take my flight today, for example. I’ve done this flying-to-New York thing a few times. I’ve learned the hard way that it’s a good idea to reserve your seat early. I know when to head toward the line at the gate to minimize time spent standing around and maximize the chance that there’s still overhead bin space onboard. And, if it looks like this particular flight is headed for delay or cancellation, I already have a pretty good view of what my options are likely to be. I might even already be dialing/browsing customer service.

I think the same applies for cloud computing. To really have a good view of what you need to know, the folks who have been through this a couple times certainly have a head start. Being early to the party lets you assess what’s happening from a position of experience – peppered with a humbling but healthy dose of reality along the way.

I think today’s big cloud announcements from CA Technologies help drive home this point. (As you might have guessed, they are part of what I’ve been working on recently: 10 new/enhanced offerings for enterprises, 4 for service providers, plus a market accelerator program.) The announcements represent quite a bit of early market experience wrapped up for the benefit of very specific customer sets.

What we (and our customers) learned in the past 18 months

Last year, IT was asking some very basic questions about cloud computing, the core of which boiled down to “So, what is cloud computing, anyway?” CA Technologies kicked off 2010 with an aggressive cloud acquisition spree that surprised more than a few folks. We brought aboard a series of key technologies, some very smart folks, and a lot of on-the-ground services experience. Customers and industry-watchers showed interest (and skepticism, as you’d expect) as we brought the pieces together.

If you look back at our CA World announcements last year, you’ll see that we described the way cloud was changing IT and the solutions that we thought were needed. We talked about the IT role morphing into more of a supply chain orchestration job, focused on delivering IT service. We saw a need to understand those services, figure out ways to compare them, manage them, and control them.

But the market hasn’t been standing still. In fact, I think most would agree with me that the changes in IT as a result of cloud have accelerated. Our view that the IT function is shifting seems to be supported by some proof points (especially if you read some of the survey data I’ve seen in the past year). But that doesn’t mean we got everything perfect, right out of the gate. By being in the game early, we’re in a prime seat to watch the evolution. And react.

Evolving and targeting to match how enterprises and service providers adopt cloud

It’s now a little more than a year later, and we’re evolving our cloud portfolio. Today’s announcements are a set of next steps, and they reflect some pragmatic reactions to what we’ve seen. We’re enhancing the offerings we already have. We’ve built some new ones. And all of these are driven by what customers are saying and doing.

Here are some of the highlights, as I see them:

More than ever before, cloud means choice. Looking at cloud forces lots of internal and external decisions. As I’ve noted previously, these are decisions about technology, about organizational structure, about IT ownership and policy. With all of these options, there is no “one size fits all” for cloud. Instead, you have to make your own, very specific choices. And you want to have a portfolio of options that can help you regardless of which choices you need to make for your business. We, as a partner in that business, need to enable you to have your cloud, your way.

A broad portfolio to work from is a plus. The work to enable customers to use and provide cloud computing means a bunch of topic areas need to be covered. Management and security really end up jumping to the top of the list. (The CA portfolio is well-tuned to cover that emphasis, I might add.)

We see a lifecycle of decisions, and a set of capabilities at steps along with way. We think customers need to plan, design, deliver, secure, and assure their cloud efforts. And then constantly optimize these decisions for what’s best for their business.

Enterprises and service providers have very different needs and will make different choices. Enterprise and service providers are doing an interesting dance. Each sees benefit – and profit – in cloud computing, and is adopting it pragmatically. Enterprises are trying to evolve what they have invested in already, while maintaining the control they require and processes they’ve built up. That lets them continue with the heterogeneous components they have. That doesn’t lock them into a proprietary (and probably quite costly) “cloud stack.” Unless they want to be. In some cases, that’s a useful trade-off. But it still needs to be managed and secured.

Service providers are, in many cases, leading the charge to cloud, looking for ways to quickly deliver cloud services but to do so in a way that is going to mean differentiation and revenues, while building margin. Those that don’t won’t be around long. They are feeling pressure from big guys like Amazon and Rackspace. They’re trying to find the right niche. They’re trying to balance the right infrastructure with the financial structure to result in a winning (and sustainable) formula.

As a result of these differences, you’ll see sets of solutions from CA Technologies that address these very specific needs, but help make the connection between the two – the world of hybrid clouds – possible and appealing.

Finally, if you add new perspectives, experience with customers, and resources to some pretty innovative technology, you can move the needle. Several of today’s announcements show the combined effort of the vision of entrepreneurs that joined CA Technologies through the cloud acquisitions and the organic development efforts since then. A lot of these folks have been working on cloud since long before the term “cloud” existed.

Several of those are near and dear to my heart, and I’ll highlight those here:

CA Business Service Insight 8.0. We’re calling it 8.0, because the previous 7 versions were called Oblicore Guarantee and were focused on service level management. However, the work done on CA Business Service Insight since last year opens up new territory. The latest release gives enterprises information about their existing services and the ability to compare and contrast what they are doing internally with services they could choose externally. All this, while also managing the service levels from what they acquire from outside. In addition, CA Business Service Insight’s connection to the Service Measurement Index and Cloud Commons will become more and more intriguing as it matures.

CA AppLogic 3.0. The ability to work at an application level rather than dwelling on low-level hypervisor questions takes a huge step up with the addition of VMware support in this release. Now, you can think in terms of virtual business services instead of ESX or Xen. That’s an important extension to the vision that the 3Tera team brought to CA Technologies, especially if you’re an enterprise.

Service providers are probably still interested in the financial equation of using Xen, but now have new options in working with enterprises who’ve made big VMware investments. And, frankly, that’s everyone at this point. The new languages, VLAN tagging, and role-based access features are probably even more interesting to service providers and how they make money from a cloud business using CA AppLogic as their cloud platform. The service provider ecosystem that’s building around CA AppLogic is should get a mention here, too, but that’s worthy of its own post.

I’m personally pleased to see Cassatt capabilities woven in here, too (check out the Global Fabric Controller to see my previous company’s influence).

Learning pragmatically

There are a lot of moving parts here, mostly driven by the huge number of options that the cloud now presents. In my opinion, CA Technologies made a pretty prescient decision to jump into this market with both feet (and wallet), and to do so early. Much of what you’re seeing come to market here has benefited from early moves by both the innovators CA acquired -- and by CA itself.

The resulting time and experience have infused our offerings (and those of us working on them) with what I’d call a healthy amount of pragmatism. This pragmatism is something that I think will serve CA Technologies, its ecosystem partners, and our collective customers well as cloud computing continues to evolve.

And, of course, it’s good to see all those hours I’ve spent waiting for flights are paying off in interesting ways.

Back to school -- for the cloud? Try not to forget the multiple paths for virtualization & cloud

Summer vacation is really a bad idea.

At least, that’s what TIME Magazine reported a few weeks back. Despite our glorified, nostalgic memories of endless hours on the tire swing above the old water hole (or, more likely, trying to find towel space on a lounge chair by the gym’s overcrowded pool), apparently kids forget stuff when they aren’t in school.

So, now that everyone’s headed back to the classroom and hitting the books again, they’ve got to jog their memories on how this learning stuff worked.

Luckily, as working adults who think about esoteric IT topics like virtualizing servers and actually planning cloud computing roll-outs, we can say this is never an issue. Right? Anyone? Bueller? Bueller?

However, with VMworld imminent and people returning from vacations, it’s a good time to reiterate what I’ve been hearing from customers and others in the industry about how this journey around virtualization and cloud computing goes.

Some highlights (take notes if you’d like; there might be a short quiz next period):

Rely on the scientific method. You’re going to hear lots of announcements at VMworld next week. (In fact, many folks jumped the gun and lobbed their news into the market this week.) In any case, be a good student and diligently take notes. But then you should probably rely a bit on the scientific method. And question authority. Know what you need or at least what you think you need to accomplish your business goal. Look at any/all of our vendor announcements through that lens. You’ll probably be able to eliminate about two-thirds of what you hear next week from VMware and all its partners (and, of course, I realize that probably includes us at CA Technologies, too). But that last third is worth a closer look. And some serious questions and investigation.

The answers aren’t simply listed in the back of your textbook. Meaning what? Well, here's one thing for starters: just because you’re knee-deep in virtualization doesn’t mean you’re automagically perfectly set up for cloud computing. Virtualization is certainly a key technology that can be really useful in cloud deployments, but as I've noted here before, it’s not sufficient all by itself. The NIST definition of cloud computing (and the one I use, frankly), doesn’t explicitly mention virtualization. Of course, you do need some smart way to pool your computing resources, and 15,000 VMworld attendees can’t be wrong…right? (Go here for my write-up on last year’s VMworld event.) But, just keep that in mind. There’s more to the story.

In fact, there may be more than one right answer. There isn’t one and only one path to cloud computing. My old BEA cohort Vittorio Viarengo had a piece in Forbes this week talking about virtualization as the pragmatic path to cloud. It can be. I guess it all depends what that path is and where it goes. It just may not be ideally suited for your situation.

On the “path to cloud computing,” to borrow Vittorio’s term, there are two approaches we’ve heard from folks:

Evolution: No, Charles Darwin isn’t really a big cloud computing guru (despite the beard). But many companies are working through a step-by-step evolution to a more dynamic data center infrastructure. They work through consolidation & standardization using virtualization. They then build upon those efforts to optimize compute resources. As they progress, they automate more, and begin to rely on orchestration capabilities. The goal: a cloud-style environment inside their data center, or even one that is a hybrid of public and private. It’s a methodical evolution. This method maps to infrastructure maturity models that folks like Gartner talk about quite a bit.

Revolution: This is not something you studied in history class involving midnight rides and red coats. If organizations have the freedom (or, more likely, the pressure to deliver), they can look at a more holistic cloud platform approach that is more turn-key. It’s faster, and skips or obviates a lot of the steps mentioned in the other approach by addressing the issues in completely different ways. The benefit? You (a service provider or an end user organization) can get a cloud environment up and running in a matter of weeks. The downside? Many of the processes you’re used to will be, well, old school. You have to be OK with that.

Forrester’s James Staten explained ways to deliver internal clouds using either approach in his report about why orgs aren’t ready for internal clouds in the first place. Both the evolutionary and the revolutionary approaches are worthy of more detail in an additional post or two in the near future, I think. But the next logical question – how do you decide what approach to take? – leads to the next bit of useful advice I’ve heard:

When in doubt, pick ‘C’. Even customers picking a more evolutionary approach won’t have the luxury of a paint-by-numbers scenario. Bill Claybrook’s recent in-depth Computerworld article about the bumpy ride that awaits many trying to deliver private clouds underscores this. “Few, if any, companies go through all of the above steps/stages in parallel,” he writes. “In fact, there is no single ‘correct’ way to transition to a private cloud environment from a traditional data center.”

So, the answer may not only be a gradual evolution to cloud by way of increasing steps of virtualization, automation, and orchestration. And it may not only be a full-fledged revolution. Instead, you want to do what’s right for each situation. That means the co-existence of both approaches.

How do you decide? It’s probably a matter of time. Time-to-market, that is. In situations where you have the luxury of a longer, more methodical approach, the evolutionary steps of extending virtualization, automation, and standardization strategies is probably the right way to go. In situations where there is a willingness, eagerness, or, frankly, a need to break some glass to get things done, viva la revolution! (As you probably can guess, the CA 3Tera product falls into this latter category.)

Learn from the past. Where people have gotten stuck with things like virtualization, you’ll need to find ways around it. Sometimes that will be helped by tools from folks like VMware themselves, broader management tools from players like, oh, say CA Technologies or a number of others. Sometimes that help will need to be in the form of experts. As I previously posted, we’ve just brought a few of these experts onboard with the 4Base Technologies acquisition, and I bet there will be a few consulting organizations in the crowd at VMworld. Just a hunch.

Back to Claybrook’s Computerworld article for a final thought: “[O]ne thing is very clear: If your IT organization is not willing to make the full investment for whatever part of its data center is transitioned to a private cloud, it will not have a cloud that exhibits agile provisioning, elasticity and lower costs per application.”

And that’s enough to ruin anyone’s summer vacation. See you at Moscone.

If you are attending VMworld 2010 and are interested in joining the San Francisco Cloud Club members for drinks and an informal get-together on Wednesday evening before INXS, go here to sign up.

Forrester's Staten: Realities of private and hybrid clouds aren't what you're expecting

James Staten does not pull punches. And for an IT industry analyst, that’s a good thing.

I first met James a few years back when he joined Forrester during my time at Cassatt. I heard him do a couple presentations at that year’s Forrester IT Forum, had some briefing sessions with him, and realized that with James, friendly conversations quickly turn into very specific advice and commentary. Even better, it was advice and commentary that was very much on target.

For those of you who don’t know James, he is a principal analyst in Forrester Research’s Infrastructure & Operations practice, helping IT ops folks make sense of topics like virtualization, cloud computing, IT consolidation best practices, and other data center- and server-focused issues. From my perspective, James has been a great addition to Forrester, especially in his role as one of the earliest voices helping describe the impact and meaning of cloud computing.

So, I thought I’d turn him loose on a couple current cloud computing topics and see if we couldn’t find a few things to argue over. In a good way, of course. Here’s that interview:

Jay Fry, Data Center Dialog: There’s lots of debate about how cloud computing takes hold in an organization. Some see people starting with public clouds. There is also lots of conversation about private clouds. In your discussions with customers, what path do you see end user organizations taking now – and is it even possible to make any generalizations?

James Staten, Forrester: Most of the time enterprises start by experimenting with public clouds. They are easy to consume and deliver fast results. And typically the folks in the enterprises who do so are application developers and they are simply trying to get their work done as fast and easy as possible and see IT ops as slow, expensive, and a headache to deal with.

In response to this, IT ops likes the idea of a private cloud – their word, not mine. This usually translates into an internal cloud and the desire is to transform parts (if not all) of their virtual server environment into a cloud. Usually this transformation happens only in name, not in operation -- and that’s where a disconnect arises. IT ops pros tend to rebut public cloud by saying, “Hey, I can provision a VM in minutes, too.” But that’s not the full value of cloud computing. Fast deployment is just the beginning.

DCD: How do you see hybrid cloud environments coming into the picture? How soon will that be a reality?

James Staten: It’s a reality for some firms today, but not in the way that many people think. A hybrid between your internal data center and a public cloud isn’t very realistic due to latency, cost, and security. A hybrid between your internal cloud and a public cloud isn’t realistic either. (See the above answers for why.) What is realistic is a hybrid between dedicated hosting and cloud hosting within the same hosting provider. USA.gov is doing just such a thing in Terremark’s Alexandria, VA data center (Forrester customers can read a case study on this here). This kind of a hybrid allows the two separate environments to share the same backbone network and security parameters. And it allows the business service being supported to match the right type of resources to the right parts of the service.

For example, you may not want or need elastic scalability for your database tier, so it’s more stable and economical to host it on a dedicated resource with a predictable 12-month contract. But you have a lot of resource consumption volatility in the app and web tiers and so they are best served being hosted in a cloud environment.

DCD: I’ve written here before about rogue deployments using cloud computing from business users without the explicit buy-in of the IT department. Do you see that as likely or commonplace? How should IT deal with these?

James Staten: They are extremely common place as evidenced by our role-based research, which shows that when you ask application developers if their company is using cloud, you get back that 24% say yes; you ask the same question to IT ops pros and get back that 5% say yes. Clearly app dev is using the cloud and cimcumventing IT ops in doing so. It’s no surprise. They see IT ops as slow and rigid.

How should IT ops respond? First, you can’t be draconian and say, “Don’t use cloud.” Those who are parents know how well that works. Instead, IT ops needs to add value to this activity so they are invited into this process. They need to embrace the use of public clouds by asking how they can help.

DCD: What are some of the big changes that you see underway with regard to cloud computing this year so far? Has anything really surprised you?

James Staten: This year will be all about understanding where to start for most companies, how to move from testing the waters for the percentage who have already gotten this far, and how to optimize your cloud deployment for those who have already moved into production on public clouds.

For IT ops, this year is about roadmapping your transformation for running a virtual server environment to running an internal cloud. They won’t get there in one year but they can build the plan and start moving now.

Nothing has really surprised me in the past 12 months but I look forward to seeing the innovative ways companies will devise to take advantage of clouds in the future. We’ve seen some very promising starts.

DCD: Acquisitions are shaping up to be a major storyline this year from my point of view. And I’m not just saying that because of CA’s recent moves with Cassatt, Oblicore, 3Tera, and Nimsoft; there have been many. You were recently commenting on Twitter about confusion coming from Oracle about the management products related to their Sun acquisition. What role do you think acquisitions are (and should) play in shaping what customers have to choose from in the cloud computing space?

James Staten: For the most part, acquisitions in a new emerging space are about speed to market. The leading software companies could adapt their existing enterprise products to incorporate and integrate cloud computing, but that’s hard, as there are already 50-100 other priorities on the roadmap that keep the installed base happy. Acquisitions also inject new blood (new thinking, new technologies, perhaps even new culture) into an existing player and that is often sorely needed to drive a sense of urgency around addressing a new market opportunity because the immediate revenue opportunity in the new market is much smaller. This is at the heart of Clayton Christensen’s Innovator’s Dilemma. Cloud computing looks very much like a disruptive innovation and according to Christensen’s theory, the disruption can be a king-maker for those that lead the disruption (and make a pauper out of those being disrupted). So that fuels the belief that acquisitions are necessary.

DCD: You wrote one of the earliest analyst reports on cloud computing in March 2008 (“Is Cloud Computing Ready for the Enterprise?”). You’ve also been on top of “how to” topics for end users with research to help clarify some of the fuzziness around cloud computing (“Which Cloud Computing Platform is Right for You?” [April 2009], “Best Practices: Infrastructure-as-a-Service” [Sept. 2009], etc.). Given how the cloud computing category – and the industry debate – has evolved, would you approach some of your earlier research differently now? Any conclusions you would change?

James Staten: No, I feel proud of the research we have done as it was backed by real-world interviews with those who are leading this evolution, rather than theories and leaps of faith about what might occur. We were also very clear in our objective of clarifying what was truly new and different about cloud computing to help guide customers through what we knew would be a hype-filled world. It’s unfortunate the industry has latched on so tightly to the term “private cloud” because in our opinion it is a very nebulous and thus meaningless concept. Heck, anything can be made private. But then again, cloud computing isn’t most precise term, either.

DCD: If many years from now we’re thinking back on how cloud computing got started versus how it ended up, there will be an interesting storyline about the degree of difference between the two. How much of the hype around cloud computing should be ignored and how fundamental do you think it will end up being? What’s going to be the thing that really makes cloud more mainstream in your opinion?

James Staten: The one part about the hype that I think should be ignored is the belief that everything will be cloud in the future. While this is a nice, disruptive statement that draws a ton of discussion and "what if" thinking, it simply isn’t realistic. If it were, there would be no mainframes today. Everything old is still running in enterprise data centers; we simply contain these technologies as we move to adopt new ones that arise.

And cloud computing is definitely not an answer to every question. Some applications have no business in the cloud and frankly will be less efficient if put there.

It’s through better understanding of what the cloud is good for and what it isn’t that IT will move forward. At the heart of cloud computing is the idea of shared empowerment – that by agreeing to a standardized way of doing something, we can share that implementation and thus garner greater efficiencies from it. That concept will manifest itself in many ways over the coming years. SaaS is a classic example that is delivering new and greater functionality to customers faster. IaaS is a great example because when multiple customers share the same pool of resources the overall utilization of the pool can be higher and thus the cost of the infrastructure can be spread more effectively across a bigger customer set, lowering the cost of deployment for all. Take this further and you can imagine many more scenarios:

· Why buy any software when it is more efficient to rent it only when you need it? We haven’t seen that model in SaaS yet.

· Why write code when you can more easily construct a business service by stringing service elements together? This is the core of SOA and builds upon the Java construct of reusable libraries. How far can we take that?

· Why store anything yourself when storing things on the Internet allows that data to be anywhere you are, whenever you need it to be – and to be self-correcting? I’ve been using Plaxo for over 5 years to keep track of all my business contacts so I don’t really care if I lose my mobile phone, laptop, or other device that stores this information. Now that Plaxo links with Facebook and LinkedIn, all the people in my address book can update their own records and I get this info as soon as it is synced. That’s distributed data management in the cloud. Can this same model be applied to other data and storage problems?

DCD: So, what area do you think is the next battleground?

James Staten: There are many areas that will be effected by cloud computing. One market that greatly interests me right now is HPC/grid computing. Loosely-coupled applications are a great fit for the cloud today and flip the economics of grid on their head. There are some incredible examples of companies using this combination to transform how they do business in healthcare, financial services, government and many other fields. Business intelligence is fertile ground for change due to this and the influx of MapReduce. I’m really looking forward to seeing what changes in this arena.

DCD: What do you believe will be some of the more interesting starting points for customers’ cloud-based activity?

James Staten: You gotta start with test and development because this is the one area where any company can get immediate benefit. Every enterprise has a queue of test projects waiting to get lab resources. Use the cloud as an escape valve for these projects. Then take a look at your web applications. If they have traffic volatility they are natural fit for cloud.

DCD: Do you see any dead-ends that some customers are heading down that others should be careful to avoid?

James Staten: It’s a total dead end to think that you can simply buy a “cloud in a box” and suddenly you have an internal cloud. Internal cloud isn’t a “what,” it’s a “how.” How you operate the environment is far more the determiner of [the benefit of] cloud computing than the infrastructure and software technologies it is based on. This is true for service providers, too. Just because an ISP has expertise in managing physical or virtual servers doesn’t mean they can effectively run a cloud. Sure, some of the cloud building block technologies can help you get there, but this is totally an operational efficiency play.

DCD: Forrester recently changed its blogging policy for its analysts to require that any content in research-related areas be posted only to official Forrester blogs. Did this move make it harder or easier to blog, and do you think it’s going to help customers in the long run?

James Staten: It made it significantly easier for me as I strongly believe in separation between work and personal life, and equally believe in freedom of expression – and thankfully, so does Forrester. While I can’t speak for every analyst (nor can I speak for Forrester on this topic), I can say personally that this change in policy was a good one. Prior to this we had team blogs, rather than blogs for each analyst, and if someone wanted their own stage, they had to go outside to do it. Now our blogging platform gives every analyst their own outlet while preserving the aggregation of blog content by client role. We also moved to a blogging system that makes it much, much easier for me to author and publish blog entries myself. Anything that makes me more productive and helps clients consume our value is a good thing.

…

Thanks to James for spending the time for this extended interview. Of course, given that James is a pretty intense endurance runner during his off hours (he’s shooting for 8 marathons and 6 half-marathons this year – and 50 marathons by his 50th birthday), a marathon interview seemed somewhat appropriate.

If you have thoughts or feel an urge to disagree with James about any of topics we touched upon here, feel free to add your comments.

From private clouds to solar panels: more control and uniqueness, but are they worth it?

Andi Mann of EMA wrote recently that failures are endemic to public clouds. And, by the way, that’s OK. In fact, says Andi, failures are normal part of what your IT infrastructure needs to be able to deal with these days.

Even if you take it as a given that we’ll hear about cloud service failure after failure in the news from now on (a daunting prospect in and of itself), public clouds surprisingly still set a pretty high bar for internal IT. Andi’s figures put some public cloud uptime numbers at 3 to 3.5 “nines,” as in 99.9 or 99.95% uptime – that's 5-10 minutes of downtime each week.

Now, if you’re hoping to get a lot of the public cloud computing benefits but do so on-premise by creating a private cloud infrastructure, there's a serious amount of work and investment required to match public cloud availability for all of your applications. Andi pins “normal” cloud computing outages at 5-10 times less likely than those in internal data centers.

“CIOs who are planning to build their own private cloud have a surprisingly high bar to reach,” blogs Andi.

Sounds like it may not be worth the effort for private clouds then, eh?

Actually, think again. It just might be, but for other reasons than you might think.

Private clouds hold what’s most unique about your organization

Mike Manos recently had some interesting observations from his time at Microsoft and at his recently-ended stint at Digital Realty Trust (sounds like he’s heading to greener pastures at Nokia). In response to James Hamilton at Amazon (thanks to Dave O’Hara for pointing out the discussion), Mike postulated that the things that make private clouds quite interesting, despite the high bar, are the way they encapsulate the unique components of an organization.

In other words, the most tailored and specific things about your IT environment are the best argument for a private cloud.

That rings true for me. Cloud computing is a way to pay for only what you need, and a way to have compute, storage, and other resources appear and disappear to support your demands, as those demands appear and disappear. There are components of what IT does for your company that are not unique. Those sound perfect to move to external, public cloud computing services at some point. They are commodities, and should be handled as such. Maybe not now, but eventually.

The more specific, complex, special pieces of IT seem logical to be the ones you keep inside as you get started down the cloud path. Those take the kid gloves and your special expertise, at least for now.

The push to get the most from those important, unique pieces of IT is giving enterprises a strong incentive to pursue a cloud-style architecture in-house. To again quote Andi Mann’s EMA research, private clouds are the preferred approach to cloud computing for 75% of the organizations they polled, far ahead of the interest in public clouds.

Are private clouds a temporary fix or a permanent fixture?

With all of that as a background, how permanent are private clouds? Here's a quick detour to help answer that:

Chris Hoff of Cisco collected commentary at his Rational Survivability blog on the topic of private clouds recently by weighing in on the appropriateness of the IT-as-electricity analogy that Nick Carr brought mainstream with The Big Switch. His quick take was that private clouds might be like batteries (thought he didn’t go too far explaining his concept, beyond labeling it an “incomplete thought” to get conversation going). However, a couple of his commenters had an analogy I liked better: that of a solar power generator.

So, is a solar power generator a good analogy for a private cloud? You’re generating “IT power” for your own use, using your own resources. Unlike what the “battery” analogy implies, a private cloud implementation is not what you’d call temporary. In fact, as Manos was thinking about private clouds in the blog noted above, one of his comments was that “there’s no such thing as a temporary data center.” Or a temporary private cloud infrastructure, I’d add. Like it or not, most IT projects, even if they are done for an ostensibly short period of time, end up living long beyond their intended sunset.

Private clouds will be no different. Many (like Gartner) see private clouds as a stepping stone or a temporary requirement until the public cloud addresses all of the roadblocks people keep complaining about. But once that infrastructure to add/subtract, build/collapse things in your IT environment is in place, you should be able to get a lot of use from it. And it will live on. This is similar, in fact, to the situation if you had taken the time and effort to get that solar installation up and running.

As things progress, I predict this “either/or” kind of language (as in “public clouds” or “private clouds”) that we’ve been seeing will fall by the wayside. I think Manos is right: “and” will be the way of the future. We’ll aim for use of the public cloud where it makes sense. And we’ll keep using private clouds – leveraging their reflection of organizational uniqueness, coupled with an unintentional permanence because, well, they work. We’ll find a way to take advantage of both.

Paving the way for hybrid clouds

This scenario, by the way, makes hybrid clouds the end state, a situation that Hoff sees as “the way these things always go.” Scott Hammond, CEO of newScale, uses an example that reiterates this: “The data center looks like my dad’s basement.” In other words, IT continues to be a strange mishmash of the new, combined with all that’s come before. That’s reality.

So a conversation that started by questioning whether public cloud computing service outages are endemic or even a problem, which then shifted to how private clouds can hold unique value for organizations, ends up connecting the two.

Of course, hybrid clouds will require an additional level of thinking, management, and control. That’s a topic that will have to get a unique post of its own one of these days.

In the meantime, I’ll leave you to ponder what other cloud computing metaphors we might be able to unearth in Scott’s dad’s basement. It just might be worth it. Especially if we find something to help those solar panels pay off.

Watching cloud computing trends for 2010: the vision, customer reality, & downstream impact

We’ve crossed into a new decade (or not, if you’re a numbers purist), and it seems to be an appropriate time for a little reflection, and maybe a chance to get some feel for where things are headed in the 2010 for IT operations, especially as they look at what cloud computing is going to mean for them.

Last year, I rattled off a Top 10 list of Top 10 lists. This year, I, for one, am suffering from a bit of Year-End Top 10 Prediction Fatigue, so I’ll hold off on that for the moment. Instead, I thought I’d check the stats from this Data Center Dialog blog as a way to get a bead on things that people have been interested in here. That way I’m not just pulling commentary out of thin air. Plus, it’s more scientific that way, right?

So, to mark Data Center Dialog’s a-bit-more-than-one-year anniversary, here's a look back at the most popular posts over the past 6 months. My guess is it gives some indication of what people will be considering for the initial months of 2010 as well.

Beyond definitions: looking for vision…and then practical cloud considerations

It’s probably no surprise that the most popular new post here was also the one that explained a bit about the biggest news story to involve us (now former) Cassatt folks: the acquisition of the Cassatt assets and expertise by CA in June. I provided a bit of commentary on the acquisition just before taking an extended few weeks off in Berlin prior to starting my current gig at CA. It’s not too far-fetched to predict that in the months ahead there will be lots more details to talk about regarding what we’ve all been doing at CA since then. (That’s an easy prediction, for sure.)

Fumble! What not to do at a cloud computing conference – The endlessly repeated ploy of starting panel sessions at cloud computing events with the question, “So what is cloud computing?” finally took its toll on me in November. The result was a bit of a (popular) rant about why the people working on cloud computing need to move on to much more useful questions. At least that’s the only way I’m going to be able to sit through another cloud computing conference.

Judging by the numbers, I think you’re with me.

Not that talking about definitions was bad. 2009 was a year in which the definitions of cloud computing (public/private, internal/external, hybrid, and the like) came into focus as the discussion evolved throughout the year. To prove the point, the most popular entry of the last 6 months was the same entry that was the most popular of the first 6 months of the year: Are internal clouds bogus? That post was followed closely by one that described the shifts in the discussion toward hybrid clouds – and the speed with which the combination of public and private cloud computing was likely to become a reality (answer: it’ll take a bit; there are some missing pieces still). In fact, my highlight blog entry that tracked the evolution of the private cloud from the front row seat I’d had was also a favorite.

So, yes, there was a place for the definitional conversation. But real-world information about what customers are doing now was in great demand (and still is, say the stats). This pragmatism is heartening and it propelled reader interest in the entry I did on the 451 Group’s cloud computing customer panel at their ICE conference, alongside a post from earlier in the year listing actual customer questions that our field team had been getting about private clouds. There’s nothing quite like getting things from the horse’s mouth.

Here’s something that was perhaps part of that same trend about getting in better sync with reality: measurement of what is actually going on in data centers (even when it’s showing a trend toward upholding long-established patterns of inefficiency) was also of interest. I saw that as good news, especially since we also had lots of interest in our post from earlier in 2009 discussing the fact that many data center managers don’t actually know what their servers are doing. The first step to a solution is understanding what the problem is, right?

Notable Data Center Dialog interviews: Steve Hamm of BusinessWeek, Bill Coleman, and Mark Bramfitt

Some of the Data Center Dialog interviews (a feature I started at the beginning of 2009 with Al Gillen of IDC) were a few of the most popular posts in the second half of the year. The most read interview? It was one in which I turned the tables on a member of the so-called mainstream media and interviewed him: BusinessWeek’s Steve Hamm had some interesting insights on Silicon Valley in general. It didn’t hurt that he linked to the interview from his blog, too, of course. Interestingly, he has now done what many journalists are doing out of necessity -- changed careers. Steve noted via Twitter a few weeks back that he’s now at IBM.

Also interesting to our readers were the conversations I published about two folks well-known in the world of IT management talking about their Next Big Things. Bill Coleman, my former CEO, gave his take on where cloud computing is now (just Version 1.0, he said) and what he’s working on after Cassatt. Mark Bramfitt talked about his move from a leading role in PG&E’s data center energy efficiency programs to private industry in a two-part interview just published at the end of December. Both Bill and Mark included some candid thoughts on what’s gone well and not so well in their previous roles.

The longer-term implications of cloud computing

We also saw interest in some of the posts pondering what cloud computing might mean to the industry as a whole. Will it mean less will be spent on IT, or, in fact, help accelerate growth? And what about the oft-noted bogeyman of automation? Will the cloud finally mean that automation takes center stage without being cast as the human-hating Skynet from the Terminator flicks? That topic generated some interest for sure.

And, of course, Twitter…

And, as you might expect, our readers were in alignment with the rest of the industry (world?) in its interest in Twitter in the past few months. I used VMworld as a case study of 7 ways that IT conferences can be improved by using Twitter – and 2 ways it makes them worse. That one seemed especially popular with folks who found us via – you guessed it – Twitter.

So what does this all mean for 2010? I have no idea. But I’d bet a couple of these trends will continue to be important. The discussion around how private, public, and hybrid cloud computing will work will certainly continue. I’m expecting, however, that it becomes more focused around the day-to-day practicalities that end user IT departments need to know.

I’ll do my best to make sure I continue to interview folks of interest in the industry with useful perspectives that will benefit IT operations and those doing big thinking about the many ins and outs of cloud computing.

And, Data Center Dialog will continue to be a place to get a pulse on topics at the forefront of the way data centers – and IT in general -- are being run and managed. As customers continue their focus on cloud computing, this blog will too. Thanks for being part of the dialog.

The Great Internal Cloud Debate: Where are we now?

In case you haven't been spending 24x7 keeping track of the industry chatter on the internal cloud and/or private cloud issue, I thought I'd point you to some recent relevant discussions. And maybe highlight what sounds something like a consensus that seems to be building about how this concept will affect (and even benefit) IT, shocking though that may be to you.

One of the most methodically thought-through and extensively discussed sets of definitions for clouds-that-aren't-really-what-everyone-meant-by-cloud-computing-in-the-first-place that I've seen recently was proposed by Chris Hoff (@Beaker on Twitter), which came complete with visual aids (thank you for that, Hoff, actually). Hoff's original point was to try to add some clarity to the "vagaries of cloudcabulary" as he described it -- and to show why using the HIPPIE (Hybrid, Public, Private, Internal, and External) terms for clouds interchangeably (as, ahem, I've kinda been doing myself around here) really doesn't help matters.

In cloud computing, there are lots of hairs to split on where the physical (er, or virtual) compute resource is located, who owns it, who manages it, who can access it -- and how. And, it turns out that after much debate, the private cloud term is the one that seems to be the squishiest. Hoff ended up with something that a lot of people liked (read his post, updates, and the comments to get the full picture), but I'm betting that the precision with which his definitions have been sculpted will be lost on many. He acknowledges that, too, in saying "I don't expect people to stop using [the] dumbed down definitions" and points specifically to comparing "private" clouds to "internal" ones as a prime offender.

So when is an internal cloud private? Or vice versa?

Since this internal v. private cloud distinction isn't one that we've really been making on this blog up to this point, I think it's worth explaining what we mean by each in light of the issues Hoff raised.

When we talk "internal clouds" here, we are mainly talking about using what you already have in your data center to create a dynamic resource pool, managed through policy-based automation software like what Cassatt provides. That means we are, for the most part, ignoring the status of a lot of the other key issues that Hoff discusses in our initial conversations. It's not that management and access (to name a few) aren't important, but they are topics that we add to the discussion along the way with customers. They are just not necessarily the first words out of our mouths.

Why?

Because we're trying to highlight what we think is the most important value to Cassatt customers: being able to leverage the concept of cloud computing by using what you already own inside your data center. In beginning this discussion about improving the efficiency of the data center resources an organization already has, the "internal cloud" moniker seems a fair, if somewhat imprecise, starting point. But you have to start somewhere.

Of course, after heading down that path a bit with a customer, the "private cloud" term may be the one that actually makes the most sense to describe what they are doing or working toward. It may be that the customer's ideal set-up includes both internal and external resources (I'm talking location here), and may need to be used by people and resources inside/outside the company, but still need to be trusted and integrated sufficiently to be considered part of that company's internal compute infrastructure. Hybrid cloud situations could definitely fall into this category, as they begin to move from the realm of PowerPoint to that of reality. And in all those cases, we should absolutely use the private cloud term.

So, we'll endeavor to be more precise in what we mean. Thanks for the pointers in the right direction, Hoff.

And, by the way, private cloud computing is suddenly everywhere

Having just said that there is a distinction between how someone uses private and internal clouds as a label, I am forced to note that the IT press and analyst communities seem to have latched onto the "private cloud" term much more aggressively, regardless of any distinctions. Or maybe those publishing recently have been following the debate (some certainly have on Twitter). I'll let you decide. In any case, here are a couple write-ups on private (and internal) clouds worth noting of late:

· InformationWeek’s Charlie Babcock covered “Why ‘Private Cloud’ Computing Is Real – And Worth Considering” pretty thoroughly. He argues that even though no single piece of an internal cloud architecture may look like a breakthrough, "private clouds represent a convergence of trends holding great promise for enterprise computing," enabling users to tap computing power without a lot of know-how. If your IT guys can master virtualization, he says, you'll master the private cloud (despite virtualization not being a requirement, it seems to be a good measuring stick). And, notes Charlie, "internal clouds aren't just a more efficient way of maintaining old data center practices." Instead, you have to rethink how you do things. Craig Vosburgh did a whole Data Center Dialog post about that topic if you're interested.

· Forrester's James Staten explained his view on how to "Deliver Cloud Benefits Inside Your Walls." While James does use both the internal and private cloud nomenclature, his first report in their "private cloud" series published April 13, 2009, puts Forrester's stake in the ground on the topic. While their definition is a little too virtualization- and developer-based for my tastes, I can't disagree with James that "the end result looks a lot like organic IT" -- the term Forrester has been using for a dynamic, utility-style data center since 2002.

· "Private Cloud Computing Is Real -- Get Over It," said Gartner's Tom Bittman in one in a series of blog posts on the topic. Tom has been pretty clear and pragmatic in his posts on this topic. Whether the name is precisely accurate is not the important point, he says. Instead, it's the idea. And he's in the middle of writing a bunch of research that, if his blog posts are any indication, will put Gartner's full weight behind the concept.

· Also from InformationWeek: what GE is doing with their private cloud, and what private cloud tools are hitting the market. (Yep, Cassatt got a quick mention.)

· 451 Group/Tier 1 Research explained that “The Sky’s the Limit: How Cloud Computing Is Changing the Rules” in their recent webcast. William Fellows recounted BT, Betfair, and Bechtel examples of how real customers are using private (and even hybrid) clouds in this webcast, created from a new report of theirs. Customer examples like this (and the ones in the InformationWeek articles) are great to see.

So where are we now?

To borrow a phrase, we've come a long way, baby. Frankly, even from when Cassatt started actively using the "internal cloud" term aggressively in customer-facing conversations in the middle of 2008 or when I first blogged on the topic here ("Are internal clouds bogus?"), there's been a notable change in both quality and quantity of discussions.

On the quality side of things: the conversation is no longer about whether this concept makes sense, but instead about who is doing it and the distinctions necessary for other companies to really get there. (Our own example: our recent webcast was explicitly about the steps toward creating an internal cloud.) This qualitative step forward is a good sign that the hype is starting to get outpaced by a little bit of real-world execution.

As for quantity, let's just say that my Google alerts for "private clouds" and "internal clouds" are going crazy. For fun (more or less), I set up "dueling Google alerts" on these two specific phrases a few months back. Some days they are more weighted toward one term, some days toward the other ("private clouds" won today, 8 mentions to 6). But the reality is that if I didn't have Google limiting their appearance in my inbox to only once a day, I wouldn't be able to keep my head above the, well, clouds.

Like the Big Dig, ex-IDC analyst John Humphreys believes cloud computing will 'take time'

In the last post, I interviewed John Humphreys, formerly the resident virtualization guru at IDC, now with the virtualization and management group within Citrix. John characterized Citrix as moving beyond criticism that they aren't doing enough with their XenSource acquisition and, in fact, taking the bull by the horns -- offering XenServer for free and focusing on aspects of heterogeneous management.

That first crack at being able to manage diverse types of virtualization in the data center is certainly needed. It's one of the first steps down a path that hasn't been well-trodden so far (and it has been especially ignored by the virtualization vendors themselves). OK, but how might that all fit into the industry conversation around cloud computing? Glad you asked...

Jay Fry, Data Center Dialog: John, I asked your old buddy Al Gillen at IDC how he thought virtualization connected to (and was distinct from) cloud computing. I'd love to get your thoughts on that, too.

John Humphreys, Citrix: I see virtualization as the foundation to any "cloudy" infrastructure -- public or private. In order for any cloud to live up to the promises, it must be able to deliver services that are isolated and abstracted from the underlying infrastructure. Clearly virtualization delivers on both of those requirements in spades!

In my opinion, the opportunity is to build workflow and policies on top of the virtualized infrastructure. Today those workflows are built at the siloed application or VM level, but I believe it will require policies and workflows that exist at the IT service level. To me, execution on this sort of vision will take a long term, multi-year commitment.

DCD: The big virtualization vendors -- Citrix, VMware, and Microsoft -- have all also talked about their cloud computing visions. While I like that VMware talks about both internal and external clouds, they seem to think that everyone will be virtualizing 100% of their servers no questions asked, and that no other virtualization technologies will exist in a data center. That, to me, puts them (and their vision) out of synch with the reality of a customer's actual data center. What's your take on this?

John Humphreys: First and foremost, I agree -- I simply don't see customers going 100% virtual any time soon. There are too many "cultural barriers" or concerns in place to do that.

The second point I'd make is that to me, cloud is still years away from becoming a mainstream reality. Just as a point of comparison, x86 virtualization, measured by VMware, is over 10 years old and now approximately 20% of servers are being virtualized each year. These things take time.

Finally, I'd point out that in the near- to mid-term, the ability to federate between internal and external clouds is a huge hurdle for the industry.

Concepts like "cloudbursting" are appealing but today are architecturally dependent. In addition to the technical ability to move services between internal and external data centers, security and regulatory impacts to cloud are "hazy" at best.

DCD: You've now had a chance to view the virtualization market from two different angles -- as an analyst at IDC and now as an executive working for a vendor in the space. What about the space do you see now that you didn't catch before in your analyst role?

John Humphreys: The move for me has been really eye-opening and educational from a lot of different perspectives. I think the one that most drew me to the role is the level of complexity that vendors must deal with in making any decision or product changes.

In the analyst realm, the focus is exclusively on strategy. When you jump over the fence, the strategy is still critical, but it is the details in the execution of that strategy that ultimately will define the success or failure of any move. That means not only being able to define a successful strategy but being able to communicate it to the organization, getting the sales teams to support the moves, coordinate the infrastructure changes that must occur internally, address supply chain issues, work with partners, etc.

As an analyst, I knew I was only seeing the first piece of the cycle, so I made the move so I could experience "the rest of story."

Being from Boston, I see a metaphor in the Big Dig and the Chunnel projects. Being an analyst is like planning the Chunnel project, while being part of a technology vendor is like planning and executing the Big Dig. The Big Dig planners had to worry about 300 years of infrastructure and needed to put all sorts contingency plans in place to ensure the successful execution. That "be prepared" requirement for flexibility appeals to me.

DCD: What's the most interesting thing that you see going on in this space right now?

John Humphreys: I see some very interesting business models being developed that leverage the cloud computing concept. I believe the industry is on the cusp of seeing a host of new ideas being introduced. And, perhaps contrary to others, I believe the down economy is a perfect incubator as expectations over the near term are lowered, giving these start-ups the opportunity to more fully develop these new and great ideas and business models. I expect we'll start to see the impact of all this innovation in the next 3-5 years.

...

Data center change: A Big Dig?

Thanks to John for the interview. I know the analogy to the Big Dig was something John meant in the context of how you get an organization building a product to do something big -- and how you have to make sure you're taking into account all the existing structures. However, I'm thinking it's a good one for data centers in general making transitions to a new model. Here's what I mean:

Your goal is to change how your data center runs to drastically cut your costs and improve how stuff gets done. There's a lot of infrastructure technology from Citrix, VMware, Cassatt, and a lot of others that can help you: virtualization, policy-based automation, management tools, the works. But there's all your critical infrastructure that's already in place and (mostly) working that you have to be careful not to disrupt. It's a big job to work around it and still make progress. Kinda like they had to do with the Big Dig.

But, hey, I'm not from Boston, so maybe the analogy breaks down for IT projects. I kind of hope so, actually, since cost overruns and big delays certainly aren't what we're all aiming for. In IT, you certainly have a greater ability to do smaller, bounded projects that show real return -- and still make notable, tangible improvements to running your business. Those of you who lived through the Big Dig probably know better than I on how close of a match this is.

On the hybrid public/private cloud capabilities, I think John's on target. The industry conversations about this capability (moving computing from inside your data center, out to the cloud, and then back again) are reaching a fever pitch, but there a few things that have to get solved before this is going to work. (Here's my recent post on hybrid clouds if you want to dive deeper on the topic.) But it's certainly one of the models that IT ops is going to want to have at its disposal in the future.

The approach we're taking at Cassatt is to help people think about how they might do "cloudbursting" by starting the work on creating a cloud internally first. At the same time, customers often begin experimenting with external cloud services. That early experience on both sides of the internal/external cloud divide will be a big help. (And, we've built our software with an eye toward eventually making cloud federation a reality.)

There is one thing I might quibble with John on that I didn't during the interview -- his supposition that virtualization is going to be at the core of any "cloudy" infrastructure. My take is that while the concept of separating the software and applications from the underlying hardware infrastructure is a key concept of creating a more dynamic infrastructure, you can still make headway here without having to virtualize everything.

In fact, we've heard a great deal of interest around the internal cloud computing approach, especially when it leverages what someone already has running in their data center -- physical or virtual. Virtualization can be a useful component, but being 100% virtualized is not a requirement. I was pretty critical of VMware's assumptions around this topic in a previous post. The Citrix approach that John walks through above is definitely describing a more realistic, heterogeneous world, but still has some assumptions you'll want to be careful of if you're looking into it.

So, if you are starting on (or are in the messy middle of) your own data center Big Dig -- exploring virtualization, the cloud, and all the infrastructure impact that those might have -- feel free to leave a comment here on how it's going.

If you can get past the Jersey barriers, of course.

IDC: Downward Directions for IT in 2009 leave room for cloud computing uptick

IDC's 44th annual Directions conference in San Jose this week may be the longest running IT conference in the world, but it didn't pull any punches on the economy. From John Gantz's opening keynote through every track session I attended, the analysts recounted what anyone running a data center knows all too well: IT spending is pulling way back. IDC wisely did a mid-year course-correction on their 2009 spending prediction at the end of last year, and they used some of these revisions at the conference to show how far -- and how fast -- things have headed down. As I was sitting in the audience, I started to wonder if even those revisions were deep enough. Only cloud computing escaped the dour forecast (more on that in a minute).

Here's a quick summary of the key points I took away from the conference, focusing on IDC's take on the macro-level IT environment, the impact of the economy on running a data center, and -- the lone bright spot -- how cloud computing figures into all this. On that last point, let's just say Frank Gens, the day's cloud presenter, was positively giddy to be the one guy who got to deliver good news. The highlights:

The economy has us in a dark, dark place -- but IT is needed now more than ever

John Gantz, IDC's chief research officer, summed up the effect of the economy at the start of the day: "I don't think we've been here before. We're in new territory. We're in the dark" because we don't have a very good handle on what the economy's going to do next. Gantz noted that IDC has ratcheted down IT spending predictions for this year to nearly flat over 2008 (up only 0.5%). That doesn't take into account any effect from the Obama stimulus package (or those from other governments elsewhere in the world). IDC told their analysts not to try to quantify stimulus package impact, said Gantz, but to assume they "won't make things worse." Let's hope. One positive note: 2010's growth rate looks positively robust, but of course that's because it's building on the catastrophe that 2009 is working out to be.

However, says Gantz, the bad economy is not slowing down the increase in mobile Internet users, the adoption of non-traditional computing devices, nor is it putting the brakes on the amount of data being gathered or user interactions per day (predicted to increase to 8.4 times its current rate in the next 4 years). And that's all something for IT to deal with.

So, said Gantz, amid this "extinction event," there are incredible new demands for management. "The economic crisis changes everything and it changes nothing. We have a new, new normal." The current situation merely forces the issue on seeing and doing things differently. "If everything is crashing down around you," said Gantz, "now is a good time to take a risk. Now is a period of opportunity." He noted companies like Hyatt, GE, RIM, FedEx, HP, and IBM had all been started in recessions (I've also written about great innovations during previous downturns).

What opportunities did he see in particular right now? Gantz noted enterprise social media, IT outsourcing, virtualization software, and Internet advertising (really). Of particular note: virtualization management software. Which has a big impact on IDC's view of what's happening in the data center...

The move to more modular, pay-as-you-go data centers -- with warnings about virtualization management

Michelle Bailey, presenting her content from IDC's Data Center Trends program, seemed very concerned about how hard and complex managing a data center had become, and believed that we're going to see customers making moves to simplify things out of necessity.

The recession, said Bailey, "changes the decision on where to hold the [data center] assets." Its main impact is to push data center managers to move "from a fixed price model to a variable pricing model," to move costs from cap ex to op ex.

Virtualization has had a huge impact so far, and will continue to do so, according to Matt Eastwood, IDC group vice president for enterprise platforms. In fact, there will be more VMs than physical servers deployed in 2009. "It will be the cross-over year," said Eastwood.

However, that drives big, big concerns on how data center managers are going to cope, said Bailey. "The thing I worry about the most with virtualization is the management consequences. There's no way to manage this with the processes and tools in place today." In fact, Bailey is so worried that she thinks this "virtualization management gap" might stall the virtualization market itself as users search for management solutions. "I’m worried that customers may have gone too far and may have to dial it back," she said. "The challenge in the server virtualization world is that people aren't used to spending a lot of money on systems management tools."

When we at Cassatt talk to customers about this, we've found that they know there is a virtual management problem and are actively trying to address it. The approach we talk to these customers about is having a coherent strategy for managing all of your data center components based upon the application service levels you need, regardless of whether the compute resources are physical or virtual. Having a separate management stack for each virtualization vendor and another one for their physical systems is not appealing, to say the least.

One of Bailey's other most important points was that there isn't just one type of data center -- there are actually three:

1. Enterprise-style data centers focus on SLAs, cost containment, and are dealing with space issues.
2. Hosting/outsourcer data centers focus on doing what's necessary to meet customer demand.
3. Web 2.0/telco-style data centers are all about cost efficiency and growth.

Trying to compare how you run your data center with one that has a different set of goals is not productive and will get you focused on the wrong things -- and result in more of a mess.

She did say, however, no matter what type of data center you are running, to look at doing things in a much more modular way, as a way to simplify. Bailey called "massively modular" the blueprint for the future data center. This helps break down big problems into smaller, more manageable ones, and ensures that you don't have to be absolutely correct in your 20-year vision for your data center. She sees things like containerized data centers becoming more standardized and less proprietary, making this modular approach more complimentary than disruptive to what data centers are already doing. And, with power and cooling still a huge problem for data centers, IT ops and facilities need help from both a more modular approach and the "pretty sophisticated" power management tools that exist. (I like to think that she was thinking of us at this point in her presentation.)

Cloud computing is on track to move to the mainstream -- and show actual growth despite the economy

Bailey had a healthy dose of cloud computing skepticism in her break-out presentation: "Anything that has money attached to it can’' be [in the cloud] for another 10 years," she said, clearly paving the way for big organizations with security, compliance, and lock-in concerns to give this cloud model a try, but to do so within their own data centers as an internal cloud.

In Frank Gens' keynote on cloud computing, he acknowledged a lot of the concerns that companies have been expressing about going to an external cloud, however, was very upbeat. "The idea of cloud is of very, very high interest to CIOs in the market right now," he said. Last year IDC predicted that 2009 would be "the year of moving from the sandbox to the mainstream," said Gens. "We are certainly on that path right now."

Why? Maybe not for the reasons you might think (cost). Gens corroborated comments from Gartner's Tom Bittman at their Data Center Conference back in December: the No. 1 reason that people want to move to the cloud is that "it's fast" to do so.

This new cloud model hasn't yet bulldozed the old model for IT, according to IDC, for reasons we've heard (and Michelle Bailey mentioned above): deficiencies in security, performance, availability, plus problems integrating with in-house IT. Gens sees cloud computing beginning the move across Geoffrey Moore's chasm toward mainstream adoption as a result of a couple things: performance-level assurances and being able to connect back to on-premise systems.

"Service level assurances are going to be critical for us to move this market [for cloud computing] to the mainstream," said Gens. And, customers want the ability to do hybrid public/private cloud computing: "They want a bridge and they want it to be a two-way bridge" between their public and private clouds.

And, despite all the economic negativity, IDC painted a pretty rosy picture for cloud computing, noting that it's where the new IT spending growth would be happening. Gens described it as the beginning of the move to a more dynamic deployment of IT infrastructure, and part of an expanding portfolio of options for the CIO.

"We’re right where we were when the PC came along or when the Internet first came out," said Gens. As far as directions go, that's pretty much "up."

Up next: comments on Nicholas Carr's closing keynote at IDC Directions San Jose. Slides from the IDC presentations noted above are available for IDC customers in PDF format in their event archives at www.IDC.com.

Sorry, VMware: you don't need virtualization for cloud computing

The VMworld Europe PR blitz is in full swing (hats off to many of my old BEA marketing compatriots!). And as it was at VMworld in Vegas back in September, VMworld Europe is all about the cloud. The only problem (if you're VMware) is that the cloud isn't all about virtualization.

In fact, you don't need virtualization for cloud computing. Despite what they'd like you to think. Blasphemy? Maybe, but let me explain...

While I was away on vacation last week a great discussion took place on this topic, started by Christofer Hoff's simple, incomplete thought that he lobbed to his readers: "How many of you assume that virtualization is an integral part of cloud computing? From your perspective do you assume one includes the other? Should you care?"

The thing that got him asking the question was a difference in the way Google and Amazon deliver and define their respective cloud services (PaaS v. IaaS, from my perspective). I, too, noticed an assumptive thread that consistently weaves its way through most conversations about cloud computing (both the internal and external varieties): the assumption is that when people are talking about dynamic, on-demand, cloud-style resources, of course there will always necessarily be virtualization underneath it all.

Not true, actually.

First, though, credit needs to be given to market-leader VMware (currently sipping their share of kir royales in Cannes, I'm sure) and all the virtualization providers, really, for changing the conversation about how a data center can be run. For years, siloed and static application stacks kept anything dynamic or elastic from getting very far. Virtualization changed that. It allowed a first step, a separation of underlying hardware from the software running on top.

Virtualization has really opened up the thinking of IT ops folks. Now that these previously inseparable items are able to be sliced, diced, moved around, expanded, contracted, and the like, all bets are off. In fact, it's probably the change in thinking brought about by virtualization that allows us to even consider talking about cloud computing in the first place.

However, virtualization isn't the silver bullet that enables cloud computing. In fact, it isn't even required. It's one of several types of technologies that can be employed to help deliver the service your business requires from a set of compute resources, either in your data center (an internal cloud), outside your data center (an external cloud), or both (a hybrid of the two).

Internal clouds will mean a mix of physical and virtual

When we at Cassatt talk to customers about implementing internal clouds, it's a discussion about using what you already have running in your data center -- apps, servers, virtual machines, networks -- with no changes, but applied in a different way. Instead of the compute resources being dedicated to particular apps, the apps (with the help of our software) pull what's needed from a big pool (cloud) of available compute supply. If our customers are any indication about what data centers really look like, some of those compute resources will be physical, some virtual. And, with the economy the way it is, people want to squeeze every bit of capability out of whatever mishmash they are running, even if it's not ideal. That means there will be a little bit of everything.

Alessandro Perilli of virtualization.info quoted VMware CEO Paul Maritz as saying that starting later this year, when the first generation of vSphere platforms will be out, there will be no technical reason not to virtualize 100% of your data center. I bet not. Why? We asked the VMware users that came by our booth at VMworld in September '08 that same question: "You're eventually going to virtualize everything, right?" Every one of them responded, "No, no, of course not." They cited cost, performance, management, and a host of other reasons. Translation: reality gets in the way. So, if you aren’t going to virtualize everything, aren't you still going to want to make the best use of all your data center resources? The answer is yes from what we hear.

External and hybrid clouds: will anybody have the same infrastructure?

External clouds might be a different story. You'll use whatever it is that your external service providers have. They could have a fully virtualized set-up. Or, they may have a more mixed environment. Again, Perilli quoted Maritz saying that virtualization is the only viable way to do cloud computing. Nope. It's an option. Of course, if you can apply your internal IT ops expertise to your external cloud work, too, it's a win (as Christofer Hoff notes in another post). It just won't always be possible.

When it comes to hybrid clouds -- moving from internal to external clouds and back, or some federated way of mixing and matching compute power from both -- the ability to be able to leverage physical and virtual, and even to leverage VMware, Citrix, Microsoft, Parallels Virtuozzo, etc., is going to be really important. Nobody is going to have the same stuff. The real world is heterogeneous.

Which brings me to some general commentary about VMware's European announcements:

Things that sounded good from the VMware announcements

There were definitely some items that sounded positive from VMworld Europe this week. Maritz talked about interoperability, and was quoted by Alex Barrett of SearchServerVirtualization.com as saying, "What we fear is the emergence of a couple highly proprietary uberclouds. There's an old joke about a California hotel that you can check in to but that you can't check out of. We don't think that should happen." (Of course, I always thought that was the Roach Motel, but maybe the point's the same.)

Kudos to Maritz for talking up internal clouds and the move to hybrid/federated clouds. Chuck Hollis from EMC underscored Martiz's validation of the private cloud concept, calling it a "big thing" in his post, because "of all the different cloud models I've heard...this is the first one that I think can work for real-world enterprise IT." In creating their "software mainframe," Maritz talks about wanting to help change the decision about whether to run a workload on an internal or external cloud from being a can't-change-it architectural one to being an operational one. Management will be at the "service level and not at the plumbing level." To me, that sounds like the right vision, and a great reason to base your infrastructure on policy-based automation. But I'm a little biased on that topic.

Some things VMware forgot to mention in Cannes (and not by accident)

As with the initial announcement of vCloud and the VDC-OS (now vSphere) back in September, though, there are some underlying problems when you try to fit the vision slides with the reality (Ken Oestreich did a good overview of these back then). First, you'll probably have more than one virtualization technology in your IT systems. This is either by design or by happenstance (like, say, from acquisitions), but it's going to be the reality in many cases. VMware's vision of "hyper-vising the data center" (as Chris Mellor at The Register calls it) ignores that.

Second, and more fundamentally (and as I noted already), you'll have both physical and virtual servers in your data center. You'll want to think through an internal cloud strategy that takes that into account.

Reuven Cohen’s post on the announcements hits the nail on the head on both of these points: "As for being interoperable, VMware is saying that its various management tools will only work on top of the VMware hypervisor. In other words, physical servers and servers virtualized by Microsoft, Citrix or any other vendor will not be compatible with the vCloud initative. Summarized, we're interoperable as long as it's VMware."

It's not unexpected that VMware would ignore these points. But *you* need to keep them in mind in trying to reconcile cloud computing versus virtualization. Virtualization can certainly help deliver a cloud computing infrastructure either inside your data center, or via a cloud service provider. But it's only one of the components, not the main driver. Delivering your business requirements as efficiently as possible -- that's the important thing.

I recommend digging through the comments in Hoff's blog entry that I mentioned at the start of this post. There are a stack of good comments there from Alessandro Perilli of virtualization.info, Andre Gironda, Anthony Chaves, James Urquhart, and others. James says "abstraction is what is important to cloud computing, not virtualization -- a big difference." Mike Rothman of securityincite.com views "cloud computing as what is being delivered and virtualization as one of the ways to deliver it."

Great points. See what I miss when I go on vacation?