Public/private 'hybrid' cloud computing: Sooner or later?

It's good to know that amid bank and technology stock prices cratering, and joblessness hitting 15-year highs here in California, and the feel-good sobriety of Barack Obama's inauguration, the tech industry is still up for a good debate. The debate on the legitimacy of internal clouds (called private clouds by many) raged on last week in the blogosphere, Twitter, and even on the official sites of industry publications and an analyst or two. The internal cloud topic continues to strike a chord.

But as all of this discussion swirled around, some of the ground started shifting a bit, too. If internal clouds are interesting to IT, then a hybrid of internal clouds and external clouds must be really interesting, right? I say maybe not quite yet. Read on.

Listening to strong voices in the internal cloud dialog

First off, here are some of the places you should have been reading last week (consider this a "how to" for catching up on some recent conversations about internal clouds): Rich Miller's Data Center Knowledge, James Urquhart's Wisdom of Clouds, Elastic Vapor, Chuck's Blog, Rational Survivability, among others. The Cloud Connect event in Mountain View was probably ground zero for all this.

But it didn't stop there. Gartner's Tom Bittman weighed in on private clouds as part of discussing Cisco's "unified computing" announcements (a subject for another day). "There is huge industry energy pushing in the direction that will make internal computing more real-time, on demand, adaptive, dynamic, unified," Tom said. "...What was custom will become packaged, and we will see a growth both in the numbers of cloud computing providers and in the number of organizations that feel they are building 'private clouds' to be used only by their internal customers." A pretty strong endorsement, there.

One of the other Gartner bloggers, Dan Sholler, revealed this "shocking truth about 'private clouds'" among others: "Private clouds (or whatever we end up calling them) are about combining the thing that infrastructure folks are doing with the things that the app folks are doing."

Here's where things got interesting: the concept of hybrid cloud computing

Throughout these discussions, many pondered the future of cloud computing. In his thorough, well-articulated piece on the reason private clouds make sense, Chuck Hollis of EMC envisioned a future phase of cloud computing that includes "federated service providers that provide customers choice." It's not hard to extrapolate from there to include your internal systems as one of those choices. But, says Chuck, "it's early days indeed. Near as I can tell, there are only a few service providers who've built environments from the ground up to [receive] virtualized applications and [information] -- and provide back to IT the control they need."

Then, this weekend, John Foley of InformationWeek published 10 predictions about cloud computing in 2009 (I guess I wasn't the final Top 10 list after all!). "IT departments will create public-private hybrid clouds. They'll use virtualization, APIs, and platforms like Elastra's Cloud Server to devise cloud-like environments in their own data centers that work seamlessly with public cloud services. ...Some experts talk of 'private clouds' as an alternative to public clouds, but hybrid clouds mix the best of both worlds."

So, in addition to arguing about whether internal clouds are something most large enterprises will pursue in the near term (for what it's worth, we are working with some significant organizations, both commercial and public sector, that are already partway through internal cloud implementations, even if that's not exactly the term they're using), this conversation also moved forward to the next step: hybrid clouds. That is, as John explained, the possibility of using both public and private. Both external and internal cloud computing. Simultaneously.

Why hybrid cloud computing is even more interesting than internal clouds. And farther off.

"Regular" cloud computing is about leveraging outside compute resources "by the drink." Internal cloud computing is about applying those same concepts to the infrastructure you already have in your data center. Internal cloud computing, though, is a reaction to the current (but not necessarily permanent) inadequacies of using cloud services. These are inadequacies like having to rewrite apps in order to run them using cloud-based resources, or not being able to handle security or compliance sufficiently. Or like the lock-in that can result from using many of the services available today.

Now, if you could get your internal cloud up and running AND could iron out the devil-in-the-details issues with external cloud computing, a hybrid between the two does, indeed, sound like the best of both worlds. You would have an internal infrastructure that uses your existing hardware, software, and networks very efficiently by dynamically balancing your computing demand with your supply. And, when even your internal supply is inadequate (or doesn't meet your policies for some reason), you can lean on the cloud providers to help you through a sudden, unexpected spike in demand from customers (like, say, to watch the aforementioned inauguration using the laptop in your office, without receiving the message I got from CNN's site, which went something like: "Congratulations, you made it, but so did everyone else. You are now in line for the next available spot for our live Internet video feed." I stuck with the nytimes.com, by the way).

With hybrid clouds, you'll need some way to make it OK to move workloads out to the cloud and back again. You'll need a policy engine so your systems can figure out when to do this and with what. And those are just a couple serious requirements off the top of my head for starters.

And I guess that's my point. The 451 Group and others have called this hybrid cloud capability the Holy Grail of computing. I've heard our CEO, Bill Coleman, talk about a world where (eventually) your systems are checking constantly with your set of external compute providers and comparing what they can supply with what you have going internally, and then having your infrastructure direct your apps to run here, there, or everywhere, depending on how prices or other conditions look on any given day/hour/minute. (This is where "follow-the-moon" computing comes in -- picture one option where your workloads run wherever computing is cheapest in the world at any given moment.)

But like tooling up to make hybrid cars, this is not something that's going to appear overnight in your IT department. That's why I'm betting we'll see some pretty robust implementations of both cloud-style architectures running inside your data center and outside of it in the cloud before you see anyone start mixing the two.

It's worth quoting Dan Sholler at Gartner again at this point: "[A]s with all innovations, the chatter will die down as we all get our hands dirty implementing this in practice." If you're seeing evidence one way or the other, I'd love to hear about it.