Sunday, January 25, 2009

Public/private 'hybrid' cloud computing: Sooner or later?

It's good to know that amid bank and technology stock prices cratering, and joblessness hitting 15-year highs here in California, and the feel-good sobriety of Barack Obama's inauguration, the tech industry is still up for a good debate. The debate on the legitimacy of internal clouds (called private clouds by many) raged on last week in the blogosphere, Twitter, and even on the official sites of industry publications and an analyst or two. The internal cloud topic continues to strike a chord.

But as all of this discussion swirled around, some of the ground started shifting a bit, too. If internal clouds are interesting to IT, then a hybrid of internal clouds and external clouds must be really interesting, right? I say maybe not quite yet. Read on.

Listening to strong voices in the internal cloud dialog

First off, here are some of the places you should have been reading last week (consider this a "how to" for catching up on some recent conversations about internal clouds): Rich Miller's Data Center Knowledge, James Urquhart's Wisdom of Clouds, Elastic Vapor, Chuck's Blog, Rational Survivability, among others. The Cloud Connect event in Mountain View was probably ground zero for all this.

But it didn't stop there. Gartner's Tom Bittman weighed in on private clouds as part of discussing Cisco's "unified computing" announcements (a subject for another day). "There is huge industry energy pushing in the direction that will make internal computing more real-time, on demand, adaptive, dynamic, unified," Tom said. "...What was custom will become packaged, and we will see a growth both in the numbers of cloud computing providers and in the number of organizations that feel they are building 'private clouds' to be used only by their internal customers." A pretty strong endorsement, there.

One of the other Gartner bloggers, Dan Sholler, revealed this "shocking truth about 'private clouds'" among others: "Private clouds (or whatever we end up calling them) are about combining the thing that infrastructure folks are doing with the things that the app folks are doing."

Here's where things got interesting: the concept of hybrid cloud computing

Throughout these discussions, many pondered the future of cloud computing. In his thorough, well-articulated piece on the reason private clouds make sense, Chuck Hollis of EMC envisioned a future phase of cloud computing that includes "federated service providers that provide customers choice." It's not hard to extrapolate from there to include your internal systems as one of those choices. But, says Chuck, "it's early days indeed. Near as I can tell, there are only a few service providers who've built environments from the ground up to [receive] virtualized applications and [information] -- and provide back to IT the control they need."

Then, this weekend, John Foley of InformationWeek published 10 predictions about cloud computing in 2009 (I guess I wasn't the final Top 10 list after all!). "IT departments will create public-private hybrid clouds. They'll use virtualization, APIs, and platforms like Elastra's Cloud Server to devise cloud-like environments in their own data centers that work seamlessly with public cloud services. ...Some experts talk of 'private clouds' as an alternative to public clouds, but hybrid clouds mix the best of both worlds."

So, in addition to arguing about whether internal clouds are something most large enterprises will pursue in the near term (for what it's worth, we are working with some significant organizations, both commercial and public sector, that are already partway through internal cloud implementations, even if that's not exactly the term they're using), this conversation also moved forward to the next step: hybrid clouds. That is, as John explained, the possibility of using both public and private. Both external and internal cloud computing. Simultaneously.

Why hybrid cloud computing is even more interesting than internal clouds. And farther off.

"Regular" cloud computing is about leveraging outside compute resources "by the drink." Internal cloud computing is about applying those same concepts to the infrastructure you already have in your data center. Internal cloud computing, though, is a reaction to the current (but not necessarily permanent) inadequacies of using cloud services. These are inadequacies like having to rewrite apps in order to run them using cloud-based resources, or not being able to handle security or compliance sufficiently. Or like the lock-in that can result from using many of the services available today.

Now, if you could get your internal cloud up and running AND could iron out the devil-in-the-details issues with external cloud computing, a hybrid between the two does, indeed, sound like the best of both worlds. You would have an internal infrastructure that uses your existing hardware, software, and networks very efficiently by dynamically balancing your computing demand with your supply. And, when even your internal supply is inadequate (or doesn't meet your policies for some reason), you can lean on the cloud providers to help you through a sudden, unexpected spike in demand from customers (like, say, to watch the aforementioned inauguration using the laptop in your office, without receiving the message I got from CNN's site, which went something like: "Congratulations, you made it, but so did everyone else. You are now in line for the next available spot for our live Internet video feed." I stuck with the, by the way).

With hybrid clouds, you'll need some way to make it OK to move workloads out to the cloud and back again. You'll need a policy engine so your systems can figure out when to do this and with what. And those are just a couple serious requirements off the top of my head for starters.

And I guess that's my point. The 451 Group and others have called this hybrid cloud capability the Holy Grail of computing. I've heard our CEO, Bill Coleman, talk about a world where (eventually) your systems are checking constantly with your set of external compute providers and comparing what they can supply with what you have going internally, and then having your infrastructure direct your apps to run here, there, or everywhere, depending on how prices or other conditions look on any given day/hour/minute. (This is where "follow-the-moon" computing comes in -- picture one option where your workloads run wherever computing is cheapest in the world at any given moment.)

But like tooling up to make hybrid cars, this is not something that's going to appear overnight in your IT department. That's why I'm betting we'll see some pretty robust implementations of both cloud-style architectures running inside your data center and outside of it in the cloud before you see anyone start mixing the two.

It's worth quoting Dan Sholler at Gartner again at this point: "[A]s with all innovations, the chatter will die down as we all get our hands dirty implementing this in practice." If you're seeing evidence one way or the other, I'd love to hear about it.


Anonymous said...

How does cloud computing get over last mile connectivity issues that many rural communities face?

Anonymous said...

I think that hybrid clouds are the future. Organizations will setup their resources to handle the load 95% of the time and they will use a public cloud to facilitate the few times that their load exceeds their resources.

I do believe however that the technology is already here to allow this to work. Cluster Resources has a product called Moab that can do many of the tasks that are needed for this to work. It can help an organization to use their data center like a cloud which can increase utilization and decrease cost. You can see more information in my write-up or at

Jay Fry said...

Chris: I agree that hybrid clouds have a great future. In fact, we're betting on it here at Cassatt, too. Some of the policy-based capabilities you guys talk about will indeed be important for this to work.

So, it's not a matter of if, but when. My point was more that either public/private hybrid cloud activities will need to be pretty simplistic in these early days, or they'll need to wait for a couple more steps down the maturity curve. And that's when they will be more interesting.

Doesn't mean IT depts. shouldn't start looking into these today. We talk to orgs about a two-prong approach at the moment: try non-critical, greenfield apps in the cloud while also working on pooling the servers supporting an app or two internally to create a cloud-style environment inside. All of this has an eye toward eventual hybrid usage of both internal and external clouds together.

Jay Fry said...

As for how we get past the last-mile connectivity issue, I can't say I have a good answer. Most of the work we do is with large corporations and government entities where those issues are not barriers. Yet. Ideas/thoughts welcome.

John Brownlow said...

> How does cloud computing get over last mile connectivity issues that many rural communities face?

Last mile connectivity issues are not restricted to 'cloud computing'. Single point of failure is a network issue that any organization looking for non-stop access has to deal with. Generally we deal with that one of two ways.
1. Dedicated link with ISDN (dial) or satellite backup or
2. Separate physical connections from different service providers