Is cloud a turkey for IT vendors? Thinking through its impact on end user IT spending

Amid preparations for Thanksgiving feasts here in the U.S., a couple recent predictions from IT industry experts could leave vendors with much less to be thankful for this year. I’ve seen more than one pundit say that even though cloud computing is the Next Big Thing for IT, it will, in the end, mean that less money is being spent on IT.

Is cloud computing really going to mean doing more with less?

Matt Asay, for example, recently noted a new Goldman Sachs report called “A Paradigm Shift for IT: The Cloud” found that even though CIOs may be “loosening the purse strings on IT spending, IT vendors may want to hold off their celebrations.” Why? “Much of this spending appears to be headed for deflationary forces like cloud computing [and] virtualization.”

At SYS-CON’s Cloud Computing Expo earlier this month, I heard GoGrid’s CEO John Keagey announce an IT vendor’s moment of triumph and death knell in one breath. “The dream of utility computing has arrived,” he said, followed quickly with: “I believe we’re at the height of IT spending right now. You’ll see the IT economy shrink to half its current size.”

And, finally, during a panel at Interop last week reported by Joe McKendrick, AT&T’s Joe Weinman, raised doubts about the sustainability of cloud computing economics, describing a scenario in which they break down as enterprise management requirements come into play. “I’m not sure there are any unit-cost advantages that are sustainable among large enterprises,” he said. He expects adoption of external cloud computing in some areas, and private capabilities for others.

Now that’s not the kind of market that the purveyors of cloud computing hype were likely hoping for. In fact, it looks to be a bit of a, um, turkey. “An economic rebound never looked so dire,” noted Asay. “That’s unless you’re an IT buyer, of course.”

More on Matt’s last comment in a moment. First, where are these comments coming from?

Goldman Sachs expects a big bump from cloud computing…

The dire remarks Asay picked up are interspersed in that pretty straightforward Goldman Sachs report he mentioned that covers the basics of cloud computing. (In fact, a big section of the report is itself entitled “Cloud Computing 101: definition, drivers and challenges,” so you shouldn’t expect big leaps forward in analysis.)

However, there are a couple of interesting observations from the report about the importance of cloud computing and its relation to the IT spending environment:

· Cloud computing is “similar in importance to the transition from mainframes to client-server computing in the 1980s.”
· The new data center approaches and architectures required for cloud computing “are coming at an auspicious time: they are likely to coincide with a meaningful cyclical upturn in IT spending over the next couple of years.” One underlying reason for the uptick: the underinvestment that the bad economy has caused in the past 18 months or so.
· “Our checks suggest that enterprise IT demand has hit an inflection point and is now beginning to improve.”

Hang on, that all sounded a bit positive, at least to my ears. They expect growth from re-architecting systems “upon a foundation of virtualization and increased automation.” Goldman (and Asay) note the typical (read: big) players that will be taking advantage of that.

So this sounds rosy for 2010 and the years just beyond. Goldman says “we expect a meaningful increase in demand in the technologies that enable virtualization and increase the availability and security of external Clouds.”

…and now the “but”:

· But, longer term, cloud computing could actually “drive some headwinds for the IT industry” because they see virtualization (one of the interesting components that can be involved in cloud computing) as a “deflationary technology.” Goldman sees IT spending concentrating into the hands of fewer buyers: mainly cloud providers, hosting vendors, and large enterprises. They think better utilization will mean less pressure to spend, and when they do, it’ll be a buyer’s market.

For SaaS, however, there was less uncertainty. SaaS offerings were seen to be expanding rapidly and leading other “external” cloud services. In fact, it is now an “unstoppable shift” from on-premise to on-demand delivery. Evangelos Simoudis, managing director of Trident Capital, posted his thoughts from the Goldman event that linked with the report, which underscored their SaaS discussion: “Two years ago,” wrote Simoudis, “the discussion was whether the enterprise will adopt any SaaS applications. Today the conversation is about which on-premise applications will be replaced by their SaaS equivalent.”

As always, baby steps to start

For areas of cloud computing other than SaaS, however, two themes emerged from the Goldman report that matched what other reputable reports have been saying. First, “for large enterprises, Cloud Computing is still very early.” And, second, “most industry participants we spoke with expect large enterprises to take baby steps in their adoption of Cloud Computing resources, especially for business-critical activities.”

Goldman highlights cloud management

Additional comments from Simoudis are relevant here in helping analyze what Goldman’s thoughts might mean. “Infrastructure-as-a-Service (IaaS) requires management skills that most IT shops today don’t have,” he said. As a result, it’s easy to see why Goldman also noted that “systems management [is] likely to become a key component of the platform over time.” Allan Leinwand of Panorama Capital, also speaking during the Interop panel I mentioned previously, underscored this management gap. “There’s a gap between what the enterprise is used to doing behind the firewall and what the cloud providers can do.”

Goldman sees traditional systems management players (BMC and CA being two they call out) going to market with more IT automation capabilities for provisioning workloads in both physical and virtual environments, as well as extending capabilities to the public cloud. But hang on to your hats, they also predict an eventual battle for where these capabilities go between systems management and platform vendors such as VMWare and Microsoft. Something to watch, for sure.

“CIOs don’t want a single, ubiquitous stack for cloud computing,” predicts Simoudis, “but the ability to work simultaneously with multiple stacks coming from different vendors.” CA’s certainly betting on that approach. Goldman’s published take on CA is that it (along with others) has cloud computing as a “potential new growth vector.” That’s a comment I agree with, or (obviously) I wouldn’t still be here.

The opposite of deflationary: Cloud as another channel for growth?

So back to the question. Is IT spending headed for a permanent downshift? As enterprises find their way to the cloud, using baby steps or otherwise, we’ll certainly see if this deflationary impact is for real.

Now, I know that this whole cloud computing revolution is supposed to first and foremost be an economic one. As in, good for the economics of running IT. No one said anything about it being good for the IT vendors. Unless, of course, they figure out how to get on the side of the customers (always a good idea, if you ask me).

And if they do figure out how to side with the customers, maybe the cloud is in fact another channel for growth. Someone recently reminded me how early e-commerce over the Internet looked back in 1999 (Black Friday was much different back then – and there wasn’t even a Cyber Monday). At that time, many analysts predicted the demise of the brick-and-mortar retail store. Instead, the Web has become another channel to sell goods; it has given people another channel to sell and buy.

Is that a good way to look at what will happen with cloud computing as well? Or maybe that’s the wrong metaphor.

Well, it’s at least worth a thought between slices of pumpkin pie. Gobble, gobble.

IT management nirvana? Smells like virtual and physical control

I was very amused by the headline on Denise Dubie's Network World article this week about CA's big multi-product announcement. It noted that CA and other management vendors were working toward IT management "nirvana" -- a state that IT has been pretty far away from. Especially when virtualization gets involved.

So, what's the main difference between where we are now and what she described? "Now" might be described (with a little help from a certain '90s grunge band of the same name) as "Come As You Are," "Nevermind," or something equally dire.

"IT management nirvana," on the other hand, requires a coherent way to control your IT environment, regardless of whether you're talking about physical or virtual components. The good news: I think CA is addressing that combined requirement pretty well, and this week's announcements help.

One of the things I've liked about the CA story that I've heard since I joined is the way in which the company directly addresses the day-to-day, pragmatic interests of users. Breaking down management silos is one of the keys to that. It's also worth noting that a bunch of other aspects of IT control -- big, grown-up, important concepts -- are front and center. Management, governance, automation, and security are all in the first paragraph of CA's press release regarding virtualization management -- topics that were certainly not highlighted nor addressed by many industry players even just six months ago.

Getting to "IT management nirvana" may not be exactly only about pragmatism, but nevertheless, I think it's some of the practical parts of this week's announcements that are worth noting. In particular:

· Single pane of glass for physical & virtual management. The root-cause analysis capabilities of the new CA Spectrum Service Assurance product take both the physical & virtual into account. It's designed to offer one place to display the impact of the physical & virtual IT infrastructure on the services it supports. Mike Vizard's CTO Edge write-up about this announcement gives you a peek into what the product's interface looks like, in case you're curious.

· Provisioning across physical & virtual infrastructures. Enabling application configuration management and dynamic resource provisioning is hard. Being able to do that across internal physical, virtual, and even external cloud environments is really hard. CA Spectrum Automation Manager has now added this, plus another neat little trick: rapid physical-to-virtual and virtual-to-virtual server provisioning.

· Cross-virtualization management. Add to the things mentioned above broad support for virtualization technologies like VMware vSphere, Citrix Xen, IBM LPAR, and Sun Solaris, and it's another reason it's worth noting. There are a few more holes to fill (Microsoft Hyper-V comes to mind), but it makes a great cross-VM story. What CA Spectrum Automation Manager now has in its cross-virtualization support is beyond what we were doing at Cassatt. You can see some earlier discussion of across-multiple-hypervisor management in this post.

· For VMware lovers out there, there's even more. A couple of the other products (you can find details about which ones in the CA press release) help discover more about VMware and performance issues, including database performance before, during, and after VMware VMotion migrations. In fact, there are a bunch more virtualization features scattered across many product lines that work in conjunction with your VMware technology. (Again, it's kind of nice to have answers to a lot of the broad management questions that customers are asking by virtue of an extensive product portfolio.)

Making a bet that integrated management is better

You'll notice that CA is putting its weight behind the concept that having a unified management capability is more efficient and more powerful for an IT organization. That's not unexpected given the breadth of management capabilities that CA can offer a customer. But, it's also in line with the complex environments that large customers actually have. But is it how they want to manage things?

Mike Vizard, again in his CTO Edge article, weighed in with one view: "At the moment," said Vizard, "customers seem to be favoring [an] integration strategy between existing systems management tools and providers of dedicated virtual machine management tools. Over the long haul, odds are good that customers will ultimately want to see more convergence of these tools rather than continuing to pay separate licensing fees for both," wrote Vizard.

Delivering tools that can provide this convergence, and seeing customers have great success with them, is the thing that will tip the balance. And in the end, that's probably a balance that will favor the customer. Vizard said something similar back in September when CA announced the deal to acquire NetQoS, postulating that "it will also ultimately prove a lot less expensive [to have unified management tools] than managing a whole slew of point products."

All that certainly sounds more like IT management nirvana (and certainly more harmonious and under control than the aforementioned namesake band from Seattle often was). Hopefully, these tools and others it inspires in the marketplace will gets us a step closer.

And, if any of my Nirvana references don't quite hit the mark: All Apologies. Blame Denise. Or her headline writer.

Inklings of what you really need for cloud management

I've seen a number of articles in recent weeks recounting some of the issues that customers are going to have if they really, truly want to start using cloud computing in some capacity. A number of the presenters at this week's Cloud World Expo said much the same.

And it's not pretty. For example:

David Linthicum, in an Intelligent Enterprise piece recently, rattled off some pretty serious issues to consider before deciding whether an application is at all a fit for any sort of cloud architecture. It included compliance, privacy, fear, and control. Along with his suggestions for working through these items, he concluded there are some apps where using a cloud will never make sense. "Cloud computing," he writes, "is almost never cost effective or a good technical fit for all applications." [Emphasis mine.]

Gordon Haff of Illuminata had a similar Top 10 list of observations about where we are today in the cloud computing market. Several of his observations noted how complicated the real world always is, especially when compared with the PowerPoint version of our plans. Some "very real and very thorny" IT questions start to come up as people really start doing cloud computing, says Haff. Some of the tough ones he noted were security, compliance (again), data portability, and even legal matters, like what happens when the FBI shows up at a hosting provider with a search warrant.

How do you start to jump over these serious hurdles?

Several of the presentations I saw today at Cloud World in San Francisco from people like James Urquhart (Cisco), Sam Charrington (Appistry), CA's own Stephen Elliot, and Joe Weinman of AT&T hit the point home. Cloud computing has a number of big issues to solve. As Weinman said in the closing keynote panel, "The enterprise data center is not going to go away" just because cloud computing appeared on the scene. (In fact, if you believe Weinman, "AT&T has been in the cloud business since 1878," a comment which amused the audience.) The question, he said, becomes "how do you make all this stuff work optimally with all the trade-offs?"

One of the missing components needed to make sure a big IT shop can take advantage of some sort of cloud architecture -- internal, external, or somewhere in between -- is management. Pretty complex and thorough management, actually. The advent of serious cloud management capabilities is going to signal that this cloud computing stuff is going to be possible for mere mortals.

Maybe I wasn't paying enough attention to the "big guys" before (call it a start-up induced blind spot, perhaps), but several very established players are jumping into the space, including CA, the new home of Cassatt employees and technology assets. CA used this week's Next Generation Data Center and Cloud World and whatever-other-names-IDG-is-using-this-year Conference to announce Amazon EC2-related capabilities across a large number of its products. The CA press release is posted here.

Is EC2 support an important thing?

Sure, the CA EC2 support is nice, but in fact I think the more interesting angle on this announcement is that a company like CA is starting to weave cloud capabilities into its entire (very broad) product line. Since CA started talking about its activity in the cloud space last November, they've now brought data center automation, application performance management, database management, and service management into the mix. And systems management is underway.

Moves like what CA is doing are starting to answer a lot of the "thorny questions" that customers are asking. Sure, there are more solutions and capabilities to deliver. But, if the industry discussion and conference chatter is any indication, people are starting to ask questions about how they cover a lot of the messier areas to make cloud computing work. And the first steps at answering those questions are appearing.

There's lots of work to do, but the more we as an industry talk about the real issues customers are going to have when incorporating cloud computing in their IT portfolio, and the less we sweep it under a layer of hype, the better.

Virtualization complexity is not going away, so plan for reality

Earlier this year, Cassatt ran our second annual Data Center Survey, getting responses from several hundred data center-related people in our database. Last year's survey (register to download it here) focused mainly on data center energy efficiency. This year, we asked those same questions again -- and then some. Since the issues in the data center have shifted, so have our questions. We hit a broad set of topics -- from virtualization to cloud computing to the impact of the economy on data center efficiency projects.

We’ll be publishing the overall results in a couple weeks, but I have been sifting through the survey data and found some interesting tidbits about virtualization that I thought were worth noting, especially in light of the press announcement we made today (more on that in a moment).

VMware is still the runaway leader, and virtualization is ready for primetime

No news there. More than 81% of our respondents have VMware currently deployed in their environment. But it certainly didn't stop there. Citrix and non-Citrix derivatives of Xen were installed in nearly 23% of organizations. Microsoft Hyper-V topped 17%. Other virtualization technologies were listed as well, including Parallels Virtuozzo Containers, Solaris Containers/Zones, IBM LPARS, and a smattering of others.

We also asked where virtualization was being used. The answer: everywhere. Almost 66% of respondents said they were using virtualization in not only development and test environments but also in production. That's up 3% from our 2008 survey. This year, only 13% were using virtualization solely in dev/test. Virtualization, as if we didn't already know, is all over the place and being relied on to support important applications.

Standardizing on one virtualization technology? IT ops folks don't see it happening

We asked our survey-takers to characterize how they plan to support virtualization in their data centers: did they expect to standardize on just one technology or support multiple? Only 23% said that they plan to support one and only one virtualization technology. Painful as it might be from a management standpoint, nearly 68% of respondents acknowledged that their data centers would have hypervisors from multiple vendors. They either:

· Are already supporting multiple virtualization technologies -- and don't expect that to change (23%)
· Are planning to support multiple virtualization technologies in the future (17%) or
· Hope to support only one virtualization technology, but fully expect that others will appear over time (28%)

Sounds like they have a pretty good handle on what's coming at them.

Managing physical and virtual systems with the same management stack?

Many people seemed to be planning to manage both their virtual servers and more traditional physical server environment with the same tool set. In fact, 41% told us exactly that. Well, that would be nice, wouldn't it? That answer seems to be a bit of wishful thinking, especially if you look at what people are doing right now (like what tools they're using). However, it's one of the things we at Cassatt want to make more and more possible, so we applaud their foresight on this. However, nearly 29% hadn't really made a strategic choice yet about how to manage these mixed physical and virtual environments, which tells me that customers are still thinking about this one. This also puts a big chunk of the organizations out there behind the eight ball, for sure, given how far adoption of virtualization has come. No firm management strategy? Yikes.

How you can plan for all this complexity: prepare for reality

All of these data points point me back to the design point we try to keep in mind when creating Cassatt products: reality. If you read today's announcement, we talked about delivering our new GA-level support for Parallels Virtuozzo Containers. This is in addition to the VMware and Xen support we already have in Cassatt Active Response, and the support for controlling physical servers, too (running Linux, Solaris, Windows, and AIX so far).

Most of the discussions about dynamic data centers or internal clouds that you hear from VMware and others don't bring up any of the real-world diversity that exist in IT shops. Maybe this is one of the reasons that some end users are highly skeptical of the whole concept of more flexible, shared, dynamic infrastructures. You can't talk about improving how someone runs their data center if you're not talking about the actual data center that they have to live with every day. (To be fair, as John Humphreys mentioned in my recent interview with him, Citrix at least acknowledged that having a tool that manages Xen and Hyper-V would be useful in their recent announcements. It's a start, for sure.)

What we're continuing to hear directly from customers and through surveys like the one I previewed a bit here is that virtualization complexity is here to stay. It can't be ignored. Instead, it needs to be taken into account: you'll have applications that use VMware. You'll acquire someone (or be acquired by someone) running Xen. Or Microsoft Hyper-V. Or Virtuozzo. Or (deep breath) all of the above. It's not what you'd like, nor something that's going to make your life as an IT operations professional easier in the short run. But it's reality.

And, when you add in the fact that you still have real, actual, physical servers running in your environment that need care and feeding to support applications, you have a physical and virtual management challenge that warrants an approach that smoothes the running of IT and reduces expenses -- rather than the opposite.

There's more about Cassatt Active Response 5.3, out today, in our press release, and overview product info is on our product pages, if you're interested.

And, incidentally, there are a couple of other interesting points worth highlighting from the survey data that I'll try to write about in upcoming posts prior to the whole report going public.

Interview with IDC's Al Gillen: in 2009, virtualization will help bridge to the cloud

When I first started this blog a few months back, I told you we'd do our best to bring you a cogent collection of useful ideas, opinions, and other relevant tidbits from those steeped in the world of running data centers. So far, that's generally taken the form of blogging from industry events (like my posts from the Gartner Data Center Conference in Vegas in early December) or commentary on IT ops topics sparked by an article or event. Our chief engineer, Craig Vosburgh, even got in on the act last month, launching the first in a series of posts scrutinizing the necessary components of cloud computing, relying on his experience in this space (and his highly tuned B.S. detector, I might add) to help cut through the hype.

And now for something completely different.

In that first blog post, I promised we'd feature give-and-take interviews with people you should know, in and around the world of IT. We're starting the new year off with a bang, in fact: today's post is the first of these interviews -- an interview with Al Gillen, program vice president of system software at IDC. When we talked to Al about Cassatt's "internal cloud" capabilities announcement in November, he had some interesting observations about virtualization, cloud computing, and where the industry is headed.

I recently had a chance to ask him about some of these comments in a little more detail, and I'm posting his responses today and tomorrow. By the way, Al was just recently promoted to run the Enterprise Virtualization Software research team at IDC following the departure of John Humphreys to Citrix. Nicely done, Al.

In today's excerpt, Al notes some shifts in the reasons people are virtualizing servers. In 2009, he says, it won't all be about consolidation. But, as the shift toward virtualization continues, the need for a way to manage virtual systems (in parallel with physical servers, I might add) becomes much more urgent. He believes this move toward virtualization is actually a bridge to cloud computing.

Here’s an excerpt of that interview:

Jay Fry, Cassatt Data Center Dialog: Congrats on the expanded role at IDC, Al. When we talked last you mentioned the industry was starting to see the "secondary impact of virtualization" -- one of which being a necessary intersection between systems management and virtualization. Can you explain a little about what you are seeing out there?

Al Gillen, IDC: The first wave of virtualization adoption was heavily weighted to consolidation and server footprint reduction. Those use cases were a natural fit for virtualization, and to be honest, there is still lots of consolidation to take place in the industry. But users find out very quickly that server consolidation does nothing to simplify the management of the operating systems and layered software, and in some cases, can make it more complex, especially if there is any balancing going on aboard the virtualized servers. Therefore, organizations that did not have a strong management solution in place before beginning to adopt virtualization will be finding that is their next most important acquisition.

DCD: How do you see the virtualization market changing in 2009?

Al Gillen: Our survey data is finding that consolidation is no longer the stand-out favorite use case for virtualization. To be sure, consolidation is not going away, but other use cases are growing in popularity as a primary objective for deploying a virtualized solution. Use cases such as high availability and disaster recovery are emerging as strong use cases today. The other factor in play here is that a lot of the largest consolidation opportunities -- not all of them, but many of them -- already have either been consolidated or are committed to a consolidation strategy and to a given vendor. As a result, the "green field" for large scale consolidations is becoming less green and less big. However, the opportunity for larger numbers of smaller scale consolidations is still enormous.

DCD: How will the economy affect how organizations are adopting virtualization?

Al Gillen: The economic conditions are likely to drive customers more quickly toward virtualization simply because the IT departments, which often are seen by management as a cost center, will be charged with reducing expenses, while still delivering the services the business needs to be successful. Being charged with "doing more with less" has been the mandate that has helped many of the transitional technologies that we now consider mainstream to get a first foothold. Can you say Linux? x86? Or Windows?

DCD: You probably guessed I was going to ask a cloud computing question. How do you think what's going on with virtualization intersects with or is different from cloud computing?

Al Gillen: There is no question that virtualization will serve as the bridge to bring customers from today's physical data center to the compute resource that will serve as tomorrow's data center. Virtualization serves several roles in this transition, including a "proof of concept" that you really can pool multiple smaller machines into a larger resource and begin to allocate and consume the resulting superset of resources in an intelligent, scalable, and reliable manner. But virtualization also will be the lubricant that makes it possible to slide software stacks off the hardware foundations to which the software used to be permanently attached. That said, cloud will be adopted very much on a transitional basis, and will penetrate different verticals and different platforms at different rates. I believe we will be talking about the transition to cloud computing for the next 15 years.

Next time: more from Al on cloud computing and the impact of the economic meltdown.