Dana Gardner's BriefingsDirect: Apr 17, 2011

Friday, April 22, 2011

Cloud brokering: Building a cloud of clouds

 This guest BriefingsDirect post comes courtesy of Jason Bloomberg, managing partner at ZapThink.

By Jason Bloomberg

The essence of cloud computing—what makes a cloud, well, cloudy—is the fact it’s an

abstraction. The cloud hides underlying complexity while presenting a simplified interface to the consumer. Of course, abstractions are nothing new in the world of IT: compiled languages, graphical user interfaces, and SOA-based Business Services are all examples of abstractions. After all, everything we’re doing in IT boils down to zeroes and ones in the end. Layers of abstraction are how we deal with this never-ending stream of bits.

The business service abstraction in the SOA context provides flexible, loosely coupled access to application functionality and data. The cloud abstraction, on the other hand, delivers a shared pool of configurable computing resources of various types (processors, storage, etc.) that can be dynamically and automatically provisioned and released. The two approaches solve different problems, but nevertheless both simplify the underlying technical complexity while providing greater agility to the consumer of the respective abstractions.

It doesn’t matter how resilient your provider’s infrastructure is if they go out of business, or a denial of service attack takes them off the Internet.

Another critical benefit of both abstractions is increased fault tolerance. If something goes wrong beneath the abstraction, then it should be possible (at least in theory) to fail over to a backup or route around the problem without adversely impacting the consumer. In the case of business services, the intermediary (typically an ESB or XML appliance) handles this routing, while the underlying cloud provider infrastructure handles failover within the cloud.

That is, unless the problem is with the cloud provider itself. It doesn’t matter how resilient your provider’s infrastructure is if they go out of business, or a denial of service attack takes them off the Internet. Think of cloud providers as baskets. Do you really want to put all of your eggs in just one?

Enter cloud brokering

Cloud brokering is the capability that addresses this eggs-in-one-basket problem. A cloud broker provides cloud service intermediation, aggregation, and arbitrage across a set of cloud providers. The need for such cloud brokers, of course, is not lost on the community of cloud startups. Today, if there’s even a hint of a niche you’ll find several entrepreneurs jumping on it, and the nascent cloud broker market is no different. However, there is a twist to the current state of the cloud broker market: as far as I can tell, all the players in this space today include cloud brokering as an extension of their existing business model, rather than a pure play model in its own right.

In fact, most of the vendors offering cloud brokering are in the cloud management space. RightScale and Kaavo, for example, provide template-based cloud deployment. Build the template, and the tool will deploy your fully configured cloud instance in any of a number of cloud environments by following the template. CloudSwitch takes the template idea down a few notches to layer two of the OSI stack, which means your cloud instances will be identical down to the IP addresses and even the MAC addresses, independent of the cloud environment. A fourth player worth mentioning is enStratus, who touts cloud independence as part of cloud governance.

All the players in this space today include cloud brokering as an extension of their existing business model, rather than a pure play model in its own right.

There is another angle on the cloud brokering marketplace, however: as an extension of the cloud storage/sync market. This niche is already quite crowded, with players like Dropbox, Jungle Disk, Box.net, Wuala, and several more. A closely related market niche is the cloud backup market, featuring vendors like Mozy, Backblaze, Carbonite, CrashPlan, and Livedrive, to name a few. It’s not clear, however, if any of these vendors support cloud brokering. Instead, they all rely upon a single underlying cloud environment for each of their offerings. The inherent fault tolerance of each vendor’s chosen cloud infrastructure may be sufficient for many users, especially in the consumer and small business segments, but enterprises may require a higher degree of resilience.

One vendor, however, has apparently carved out a niche for themselves: Oxygen cloud. Oxygen cloud focuses on cloud-based sync and shared storage, but they have also taken the extra step to build cloud brokering into their offering. As a result, customers who want the benefits of sync and storage in the cloud without having to rely on a particular cloud provider have few if any options other than Oxygen cloud.

The ZapThink take

The ability to select among several public clouds is only one benefit of cloud brokering. It also

supports the ability for an organization to move application instances or data between private and public clouds. In other words, cloud brokering is at the heart of dynamic hybrid clouds.

When we talk about the various cloud deployment models—public, private, community, and hybrid—it’s the hybrid model that elicits the most head scratching. People wonder under what circumstances would it ever be worth the trouble to mix private and public clouds together. And they have a point: hybrid clouds sound like a huge hassle. Without cloud brokering, managing a hybrid cloud may be more trouble than it’s worth.

Cloud brokering, however, abstracts out the deployment model altogether, creating what we might even call a “cloud of clouds.” From the perspective of the consumer, all clouds might as well be hybrid clouds, because the decision whether to leverage on-premise or off-premise resources is simply part of the dynamic provisioning benefit of the cloud of clouds. The notion of a cloud of clouds that brokering enables, however, is a temporary phenomenon. Today we require visibility into the selection of individual cloud providers. Tomorrow, the brokering-based cloud of clouds will simply be the cloud.

ZapThink has no business relationship with any of the vendors mentioned in this ZapFlash. We’re simply calling ‘em like we see ‘em.

 This guest BriefingsDirect post comes courtesy of Jason Bloomberg, managing partner at ZapThink.

SPECIAL PARTNER OFFER

SOA and EA Training, Certification,
and Networking Events

In need of vendor-neutral, architect-level SOA and EA training? ZapThink's Licensed ZapThink Architect (LZA) SOA Boot Camps provide four days of intense, hands-on architect-level SOA training and certification.

Advanced SOA architects might want to enroll in ZapThink's SOA Governance and Security training and certification courses. Or, are you just looking to network with your peers, interact with experts and pundits, and schmooze on SOA after hours? Join us at an upcoming ZapForum event. Find out more and register for these events at http://www.zapthink.com/eventreg.html.

Tag-team of HP workshops provides essential path to IT maturity assessment and a data center transformation journey

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or download a copy. Learn more. Sponsor: HP.

The pace of change, degrees of complexity, and explosion around the uses of new devices and increased data sources are placing new requirements and new strain on older data centers. Research shows that a majority of enterprises are either planning for or are in the midst of data center improvements and expansions.

Deciding how to best improve your data center is not an easy equation. Those building new data centers now need to contend with architectural shifts to cloud and hybrid infrastructure models, as well as the need to cut total cost and reduce energy consumption for the long-term.

An added requirement for new data centers is to satisfy the needs of both short-and long-term goals, by effectively jibing the need for agility now with facility and staffing decisions that may well impact the company for 20 years or more.

All these fast-moving trends are accelerating the need for successful data center transformation (DCT). As a means to beginning such a DCT journey, to identify some proven ways that explore how to do DCT effectively, BriefingsDirect now examines two ongoing HP workshops as a means of accurately assessing a company’s maturity in order to know then how to best begin and take a DCT journey.

Join BriefingsDirect's Dana Gardner, Principal Analyst at Interarbor Solutions, as he interviews three HP experts on the Data Center Transformation Experience Workshop and the Converged Infrastructure Maturity Model Workshop: Helen Tang, Solutions Lead for Data Center Transformation and Converged Infrastructure Solution for HP Enterprise Business; Mark Edelmann, Senior Program Manager at HP’s Enterprise Storage, Servers, and Network Business Unit, and Mark Grindle, Business Consultant for Data Center Infrastructure Services and Technology Services in HP Enterprise Business.

Here are some excerpts:

Tang: What the world is demanding is essentially instant gratification. You can call it sort of an instant-on world, a world where everything is mobile, everybody is connected, interactive, and things just move very immediately and fluidly. All your customers and constituents want their need satisfied today, in an instant, as opposed to days or weeks. So, it takes a special kind of enterprise to do just that and compete in this world.

You need to be able to serve all of these customers, employees, partners, and citizens -- 0r if you happen to be a government organization -- with whatever they want or need instantly, any point, any time, through any channel. This is what HP is calling the Instant-On Enterprise, and we think it's the new imperative.

There are a lot of difficulties for technology, but also if you look at the big picture, we live in extremely exciting times. We have rapidly changing and evolving business models, new technology advances like cloud, and a rapidly changing workforce.

Architecture shifts

A Gartner stat: In the next four years, 43 percent of CIOs will have the majority of their IT infrastructure and organizations and apps running in the cloud or in some sort of software-as-a-service (SaaS) technology. Most organizations aren’t equipped to deal with that.

There’s an explosion of devices being used: smartphones, laptops, TouchPads, PDAs. According to the Gartner Group, by 2014, that’s less than three years, 90 percent of organizations will need to support their corporate applications on personal devices. Is IT ready for that? Not by a long shot today.

Last but not least, look at your workforce. In less than 10 years about half of the workforce will be millennials, which is defined as people born between the year of 1981 and 2000 -- the first generation to come of age in the new millennium. This is a Forrester statistic.

This younger generation grew up with the Internet. They work and communicate very differently from the workforce of today and they will be a main constituency for IT in less than 10 years. That’s going to force all of us to adjust to different types of support expectations, different user experiences, and governance.
Maturity is a psychological term used to indicate how a person responds to the circumstances or environment in an appropriate and adaptive manner.

Your organization is demanding ever more from IT -- more innovation, faster time to market, more services -- but at the same time, you're being constrained by older architectures, inflexible siloed infrastructure that you may have inherited over the years. How do you deliver this new level of agility and be able to meet those needs?

You have to take a transformational approach and look at things like converged infrastructure as a foundation for moving your current data center to a future state that’s able to support all of this growth, with virtualized resource pools, integrated automated processes across the data center, with an energy-efficient future-proofed physical data center design, that’s able to flex and meet these needs.

Edelmann: The IT Maturity Model approach consists of an overall assessment, and it’s a very objective assessment. It’s based on roughly 60 questions that we go through to specifically address the various dimensions, or as we call them domains, of the maturity of an IT infrastructure.

We've found it’s much more valuable to sit down face to face with the customer and go through this, and it actually requires an investment of time. There’s a lot of background information that has to be gathered and so forth, and it seems best if we're face to face as we go through this and have the discussion that’s necessary to really tease out all the details.

We apply these questions in a consultative, interactive way with our customers, because some of the discussions can get very, very detailed. Asking these questions of many of our customers that have participated in these workshops has been a new experience. We're going to ask our customers things that they probably never thought about before or have only thought of in a very brief sort of a way, but it’s important to get to the bottom of some of these issues.

From that, as we go through this, through some very detailed analysis that we have done over the years, we're able to position the customer’s infrastructure in one of five stages:
The first stage, which is where most people start, is in Stage 1; we call that Compartmentalized and Legacy, which is rather essentially the least-mature stage.
From there we move to Stage 2, which we call Standardized.
Stage 3 then is Optimized.
Stage 4 gets us into Automated and a Service-Oriented Architecture (SOA), and,
Stage 5 is more or less IT utopia necessary to become the Instant-On Enterprise that Helen just talked about. We called that Adaptively Sourced Infrastructure.
We evaluate each domain under several conditions against those five stages and we essentially wind up with a baseline of where the customer stands.

As a result of examining the infrastructure’s maturity along these lines, we're able to establish a baseline of the maturity of the infrastructure today. And, in the course of interviewing and discussing this with our customers, we also identify where they would like to be in terms of their maturity in the future. From that, we can put together a plan of how to get from here to there.

Even further behind

Most of our customers find out that they are a lot further behind than they thought they were. It's not necessarily due to any fault on their part, but possibly a result of aging infrastructure, because of the economic situation we have been in, disparate siloed infrastructure as a result of building out application focused stacks, which was kind of the way we approached IT historically.

Also, the impact of mergers and acquisitions has kind of forced some customers to put together different technologies, different platforms, using different vendors and so forth. Rationalizing all that can leave them in kind of a disparate sort of a state. So, they usually find that they are a lot further behind than they thought.

We've been doing this for a while and we've done a lot of examinations across the world and across various industries. We have a database of roughly 1,400 customers that we then compare the customer’s maturity to. So, the customer can determine where they stand with regards to the overall norms of IT infrastructures.
It's a difficult and a long journey to get to that level, but there are ways to get there, and that’s what we're here for.

We can also illustrate to the customer what the best-in-class behavior is, because right now, there aren’t a whole lot of infrastructures that are up at Stage 5. It's a difficult and a long journey to get to that level, but there are ways to get there, and that’s what we're here for.

Grindle: This process can also be structured if you do the Data Center Transformation Experience Workshop first and then follow that up with the Maturity Model.

The DCT workshop was originally designed and set up based on HP IT’s internal transformation. It's not theoretical and it's also extremely interactive. So, it's based on exactly what we went through to accomplish all the great things that we did, and we've continued to refine and improve it based on our customer experiences too. So, it's a great representation of our internal experiences as well as what customers and other businesses and other industries are going through.

During the process, we walk the customer through everything that we've learned, a lot of best practices, a lot of our experiences, and it's extremely interactive.

Then, as we go through each one of our dimensions, or each one of the panels, we probe with the customer to discuss what resonates well with them, where they think they are in certain areas, and it's a very interactive dialog of what we've learned and know and what they've learned and know and what they want to achieve.

The outcome is typically a very robust document and conversation around how the customer should proceed with their own transformation, how they should sequence it, what their priorities are, and true deliverables -- here are the tasks you need to take on and accomplish -- either with our help or on their own.
It’s a great way of developing a roadmap, a strategy, and an initial plan on how to go forward with their own transformational efforts.

Designed around strategy

It's definitely designed around strategy. Most people, when they look at transformation, think about their data centers, their servers, and somewhat their storage, but really the goal of our workshop is to help them understand, in a much more holistic view, that it's not just about that typical infrastructure. It has to do with program management, governance, the dramatic organizational change that goes on if you go through transformation.

Applications, the data, the business outcomes, all of this has to be tied in to to ensure that, at end of the day, you've implemented a very cost-effective solution that meets the needs of the businesses. That really is a game-changing type of move by your organization.

The financial elements are absolutely critical. There are very few businesses today that aren’t extremely focused on their bottom line and how they can reduce the operational cost.

Certainly, from the HP IT experience, we can show, although it's not a trivial investment to make this all happen, the returns are not only normally a lot larger than your investment, but they are year-over-year savings. That’s money that typically can be redeployed to areas that really impact the business, whether it's through manufacturing, marketing, or sales. This is money that can be reinvested in the business, and allowed to help grow the areas that really will have future impact on the growth of the business, while reducing the cost of your data centers and your operation.
Even though you're driving down the cost of your IT organization, you're not giving up quality and you are not giving up technology.

Interestingly enough, what we find is that, even though you're driving down the cost of your IT organization, you're not giving up quality and you are not giving up technology. You actually have to implement new technologies and robust technologies to help bring your cost down. Things like automations, operational efficiency, ITIL processes all help you drive the saving while you are allowed to upgrade your systems and your environments to current technologies and new technologies.

And, while we're on the topic of cost savings, a lot of times when we are talking to customer about transformation, it's normally being driven by some critical IT imperative, like they're out of space in their data center and they're about to look at building out a new data center or perhaps a obtaining a collocation site. A lot of times we find that we sit down and talk with them about how they can modernize their application, tier their storage, go with higher density equipment, virtualize their servers, they actually can free up space and avoid that major investment of the new data center.

I am working with a company right now that was looking at going to eight data centers and by implementing a lot of these new technologies -- higher virtualization rates, improvements to their applications, and better management of their data on their storage. We're trying to get them down into two data centers. So right there is a substantial change. And, that’s just an example of things that I have seen time and time again, as we've done these workshops.

It's all about walking through the problems and the issues that are at hand and figuring out what the right answers are to meet their needs, while trying to control the expense.

Tang: Both workshops are great. It's not really an either/or. I would start with the Data Center Transformation Experience Workshop, because that sets the scene in the background of how I start to approach this problem. What do I think about? What are the key areas of consideration? And, it maps out a strategy on a grander scale.

The CI Maturity Model Assessment specifically gets into when you think about implementation. Let's dive in and really drill deep into your current state versus future state when it comes to the five domains.

You say, "Okay, what would be the first step?" A lot of times, it makes sense to standardize, consolidate. Then, what is the next step? Sometimes that’s modernizing applications, and so on. That’s one approach we have seen.

In the more transformational approach, whereby you have the highest level of buy-in, all the way up to the CIO and sometimes CFO and CEO, you lay out an actual 12-18 month plan. HP can help with that, and you start executing toward that.

A lot of organizations don’t have the luxury of going top-down and doing the big bang transformation. Then, we take that more project-based approach. It still helps them a lot going through these two workshops. They get to see the big picture and all the things that are possible, but they start picking low-hanging fruit that would yield the highest ROI and solve their current pain points.
A lot of organizations don’t have the luxury of going top-down and doing the big bang transformation.

Edelmann: Often, the journey is a little bit different from one customer to the other.

The Maturity Model Workshop you might think of as being at a little lower level than the Data Center Transformation Workshop. As a result of the Maturity Model Workshops, we produce a report for the customer to understand -- A is where I'm at, and B is where I'm headed. Those gaps that are identified during the course of the assessment help lead a customer to project definitions.

In some cases, there may be some obvious things that can be done in the short term and capture some of that low-hanging fruit -- perhaps just implement a blade system or something like that -- that will give them immediate results on the path to higher maturity in their transformation journey.

Multiple starting points

There are multiple starting points and consequently multiple exit points from the Maturity Model Workshop as well.

Grindle: The result of the workshop is really a sequence series of events that the customer should follow up on next. Those can be very specific items, like gather your physical server inventories so that that can be analyzed, to other items such as run a Maturity Model Workshop, so that you can understand where you are in each of the areas and what the gaps are, based on where you really want to be.

It’s always interesting when we do these workshops, because we pull together a group of senior executives covering all the domains that I've talked about -- program management, governance -- their infrastructure people, their technology people, their applications people, and their operational people, and it’s always funny, the different results we see.

I had one customer that said to me that the deliverable we gave them out in the workshop was almost anti-climatic versus what they learned in the workshop. What they had learned during this one was that many people had different views of where the organization was and where it wanted to go.
It’s a great learning collaborative event that brings together a lot of the thoughts on where they want to head.

Each was correct from their particular discipline, but from an overarching view of what are we trying to do for the business, they weren’t all together on all of that. It’s funny how we see those lights go on as people are talking and you get these interesting dialogs of people saying, "Well, this is how that is." And someone else going, "No, it’s not. It’s really like this."

It’s amazing the collaboration that goes on just among the customer representatives above and beyond the customer with HP. It’s a great learning collaborative event that brings together a lot of the thoughts on where they want to head. It ends up motivating people to start taking those next actions and figuring out how they can move their data centers and their IT environment in a much more logical, and in most cases, aggressive fashion than they were originally thinking.

The place to learn more would be hp.com/go/dct. To learn more about the CI Maturity Model, you can go to hp.com/go/cimm.