Message Bus bets its cloud-native messaging service will improve the art of email delivery

Message Bus has a pedigreed CEO, an impressive list of customers and partners, and technology that makes its cloud-based service highly scalable and resilient, yet the young company's goal is simple: help customers keep their legitimate email messages out of recipients' spam folders.

With Twitter co-founder Jeremy LaTrasse at the helm, Message Bus is navigating the often dark waters of email delivery so that its customers don't have to. The company's Global Delivery Network, launched in mid-November, aims to be to email and mobile messaging what Amazon Web Services are to cloud computing and Dropbox to cloud storage.

Currently, one in five legitimate emails is either blocked or routed to the spam folder.

The service is a cloud-native application, meaning that it's not tied to the underlying infrastructure of a single cloud service provider. Therefore Message Bus can scale and move its customers' workloads across different cloud infrastructures as needed (the company says it currently deploys on Joyent, Amazon Web Services and Rackspace cloud services). This approach avoids the scale limitations of working with a single cloud service provider, as well as the possibility of service disruption if a provider experiences an outage.

But it takes more than the right architecture to provide an effective message delivery service. Message Bus has done extensive relationship building with top ISPs including AOL, Microsoft and Google to understand what they expect from a trusted sender and sticks to those guidelines, resulting in a higher likelihood that legitimate emails make it to the inbox.

"More than 90 percent of all mail worldwide ends up in one of those places; if there’s no trust with those ISPs then the message won’t make it into the box," says LaTrasse. "So we had the idea to build best practices into the network, so everyone who sends through our service follows them. We made the relationships happen, and all our customers benefit, as well as their recipients."

Out of control

Currently, one in five legitimate emails is either blocked or routed to the spam folder, says Message Bus, making it difficult for companies relying on email as a primary driver of revenue and brand recognition to get their message across. What's more, the cost and complexity of launching messaging campaigns across multiple channels (email, mobile and social messaging, etc) is spinning out of control.

Customers of the Global Delivery Network don't need dedicated messaging hardware or personnel; instead they build a virtual SMTP bridge to send their messages across Message Bus' network. This significantly reduces upfront infrastructure costs as well as ongoing staffing, says LaTrasse, and allows customers to focus on the content of the messages, knowing that they'll be delivered in a manner that's effective, secure, and complaint.

If there’s no trust with those ISPs then the message won’t make it into the box.

At the same it unveiled the Global Delivery Network Message Bus launched a free reporting service called Discover that informs customers of email senders who may be abusing their domain name for illicit or unauthorized purposes. And late in November the company announced an enhancement to its service with the deployment of Opscode's Hosted Chef to automate configuration, environment and application management across the multiple cloud infrastructures powering the company's service.

Message Bus lists American Greetings, MyFitnessPal, and Telly among its early users.

(BriefingsDirect contributor Cara Garretson provided editorial assistance and research on this post. She can be reached on LinkedIn at http://linkd.in/T6trhH.)

You may also be interested in:

• Message Bus lands $11M from North Bridge, True Ventures to be the Dropbox for cloud messaging
• SendGrid and Twilio team up to offer email, SMS, voice and push notifications for the enterprise market
• Study: Cloud computing becoming pervasive, and IT needs to take control now
• Capgemini's CTO on how Cloud computing exposes the duality between IT and bsuiness transformation
• Market confidence in cloud soars, especially among service providers, says North Bridge survey

 

New strategies now needed to simplify data backup and protection in complex enterprise IT environments

Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy. Sponsor: Quest Software.

The latest BriefingsDirect IT trends discussion targets enterprise backup, why it’s broken, and how to fix it.

Nowadays the backup of enterprise information and associated data protection are fragmented, complex, and inefficient. But new approaches are helping to simplify the data-protection process, keep costs in check, and improve recovery speed and confidence.

Joining us to share insights on how data protection became such a mess -- and how new techniques are being adopted to gain comprehensive and standard control over the data lifecycle -- are John Maxwell, Vice President of Product Management for Data Protection at Quest Software, now part of Dell, and George Crump, Founder and Lead Analyst at Storage Switzerland, an analyst firm focused on the storage market. The chat is moderated by Dana Gardner, Principal Analyst at Interarbor Solutions.  [Disclosure: Quest Software is a sponsor of BriefingsDirect podcasts.]

Here are some excerpts:

Gardner: Why has something seemingly as straightforward as backup become so fragmented and disorganized?

Maxwell: Dana, I think it’s a perfect storm, to use an overused cliché. If you look back 20 years ago, we had heterogeneous environments, but they were much simpler. There were NetWare and UNIX, and there was this new thing called Windows. Virtualization didn’t even really exist. We backed up data to tape, and a lot of data was in terabytes, not petabytes.

Flash forward to 2012, and there’s more heterogeneity than ever. You have stalwart databases like Microsoft SQL Server and Oracle, but then you have new apps being built on MySQL. You now have virtualization, and, in fact, we're at the point this year where we're surpassing the 50 percent mark on the number of servers worldwide that are virtualized.

John Maxwell

Now we're even starting to see people running multiple hypervisors, so it’s not even just one virtualization platform anymore, either. So the environment has gotten bigger, much bigger than we ever thought it could or would. We have numerous customers today that have data measured in petabytes, and we have a lot more applications to deal with.

And last, but not least, we now have more data that’s deemed mission critical, and by mission critical, I mean data that has to be recovered in less than an hour. Surveys 10 years ago showed that in a typical IT environment, 10 percent of the data was mission critical. Today, surveys show that it’s 50 percent and more.

Crump: I would dovetail into what he just mentioned about mission criticality. There are definitely more platforms, and that’s a challenge, but the expectation of the user is just higher. The term I use for it is IT is getting "Facebooked."

High expectations

I've had many IT guys say to me, "One of the common responses I get from my users is, 'My Facebook account is never down.'" So there is this really high expectation on availability, returning data, and things of that nature that probably isn’t really fair, but it’s reality.

One of the reasons that more data is getting classified as mission critical is just that the expectation that everything will be around forever is much higher.

George Crump

The other thing that we forget sometimes is that the backup process, especially a network backup, probably unlike any other, stresses every single component in the infrastructure. You're pulling data off of a local storage device on a server, it’s going through that server CPU and memory, it’s going down a network card, down a network cable, to a switch, to another card, into some sort of storage device, be it disk or tape.

So there are 15 things that happen in a backup and all 15 things have to go flawlessly. If one thing is broken, the backup fails, and, of course, it’s the IT guy’s fault. It’s just a complex environment, and I don’t know of another process that pushes on all aspects of the environment in one fell swoop like backup does.

Gardner: So the stakes are higher, the expectations are higher, the scale and volume and heterogeneity are all increased. What does this mean, John, for those that are tasked with managing this, or trying to get a handle on it as a process, rather than a technology-by-technology approach?

Maxwell: There are two issues here. One, you expect today's storage administrator, or sysadmin, to be a database administrator (DBA), a VMware administrator, a UNIX sysadmin, and a Windows admin. That’s a lot of responsibility, but that’s the fact.

A lot of people think that they are going to have as deep level of knowledge on how to recover a Windows server as they would an Oracle database. That’s just not the case, and it's the same thing from a product perspective, from a technology perspective.

Is there really such thing as a backup product, the Swiss Army knife, that does the best of everything? Probably not.

Is there really such thing as a backup product, the Swiss Army knife, that does the best of everything? Probably not, because being the best of everything means different things to different accounts. It means one thing for the small to medium-size business (SMB), and it could mean something altogether different for the enterprise.

We've now gotten into a situation where we have the typical IT environment using multiple backup products that, in most cases, have nothing in common. They have a lot of hands in the pot trying to manage data protection and restore data, and it has become a tangled mess.

Gardner: Before we dive a little bit deeper into some of these major areas, I'd like to just visit another issue that’s very top of mind for many organizations, and that’s security, compliance, and business continuity types of issues, risk mitigation issues. George Crump, how important is that to consider, when you look at taking more of a comprehensive or a holistic view of this backup and data-protection issue?

Disclosure laws

Crump: It's a really critical issue, and there are two ramifications. Probably the one that strikes fear in the heart of every CEO on the planet is all the disclosure laws that exist now that say that, when you lose a customer’s data, you have to let him know. Unfortunately, probably the only effective way to do that is to let everybody know.

I'm sure everybody listening to this podcast has gotten more than one letter already this year saying their Social Security number has been exposed, things like that. I can think of three or four I've already gotten this year.

So there is the downside of legally having to admit you made a mistake, and then there is the legal requirements of retaining information in case of a lawsuit. The traditional thing was that if I got a discovery motion filed against me, I needed to be able to pull this information back, and that was one motivator. But the bigger motivator is having to disclose that we did lose data.

And there's a new one coming in. We're hearing about big data, analytics, and things like that. All of that is based on being able to access old information in some form, pull it back from something, and be able to analyze it.

That is leading many, many organizations to not delete anything. If you don't delete anything, how do you store it? A disk-only type of solution forever, as an example, is a pretty expensive solution. I know disk has gotten a lot cheaper, but forever, that’s a really long time to keep the lights on, so to speak.

We need to step back, take inventory of what we've got, and choose the right solution to solve the problem at hand, whether you're an SMB or an enterprise.

Gardner: Let's look at this a bit more from the problem-solution perspective. We have multiple platforms, we have operating systems, hypervisors, application types, even appliances. What's the solution?

Maxwell: The problem is we need to step back, take inventory of what we've got, and choose the right solution to solve the problem at hand, whether you're an SMB or an enterprise.

But the biggest thing we have to address is, with the amount and complexity of the data, how can we make sysadmins, storage administrators, and DBAs productive, and how can we get them all on the same page? Why do each one of these roles in IT have to use different products?

George and I were talking earlier. One of the things that he brought up was that in a lot of companies, data is getting backed up over and over by the DBA, the VMware administrator, and the storage administrator, which is really inefficient. We have to look at a holistic approach, and that may not be one-size-fits-all. It may be choosing the right solutions, yet providing a centered means for administration, reporting, monitoring, etc.

Gardner: Is there anything different and specific about backup that makes this even harder to move from that point solution, best-of-breed mentality, into more of a comprehensive process standardization approach?

Demands and requirements

Crump: It really ties into what John said. Every line of business is going to have its own demands and requirements. To expect not even a backup administrator, but an Oracle administrator that’s managing an Oracle database for a line of business, to understand the nuances of that business and how they want to keep things is a lot to ask.

When backup is broken, the default survival mechanism is to throw everything out, buy the latest enterprise solution, put the stake in the ground, and force everybody to centralize on that one item. That works to a degree, but in every project we've been involved with, there are always three or four exceptions. That means it really didn’t work. You didn't really centralize.

Then there are covert operations of backups happening, where people are backing up data and not telling anybody, because they still don't trust the enterprise application. Eventually, something new comes out. The most immediate example is virtualization, which spawned the birth of several different virtualized specific applications. So bringing all that back in again becomes very difficult.

I agree with John. What you need to do is give the users the tools they want. Users are too sophisticated now for you to say, "This is where we are going to back it up and you've got to live with it." They're just not going to put up with that anymore. It won't work.

So give them the tools that they want. Centralize the process, but not the actual software. I think that's really the way to go.

Gardner: So we recognize that one size fits all probably isn’t going to apply here. We're going to have multiple point solutions. That means integration at some level or multiple levels. That brings us to our next major topic. How do we integrate well without compounding the complexity and the problems set? John?

We’re keenly interested in leveraging those technologies for the DBAs and sysadmins in ways that make their lives easier and make sure they are more productive.

Maxwell: We've been working on this now for almost two years here at Quest, and now at Dell, and we are launching in November, something called NetVault XA. “XA” stands for Extended Architecture. We have a portfolio of very rich products that span the SMBs and the enterprise, with focus on virtual backup, heterogeneous backup, instantaneous snapshots and deep application recovery, and we’re keenly interested in leveraging those technologies for the DBAs and sysadmins in ways that make their lives easier and make sure they are more productive.

NetVault XA solves some really big issues. First of all, it unifies the user experience across products, and by user, I mean the sysadmin, the DBA, and the storage administrator, across products. The initial release of NetVault XA will support both our vRanger and NetVault Backup, as well as our NetVault SmartDisk product, and next year, we'll be adding even more of our products under NetVault XA as well.

So now we've provided a common means of administration. We have one UI. You don’t have to learn something different. Everyone can work on the same product, yet based on your login ID, you will have access to different things, whether it's data or capabilities, such as restoring an Oracle or SQL Server database, or restoring a virtual machine (VM).

That's a common UI. A lot of vendors right now have a lot of solutions, but they look like they're from three, four, or five different companies. We want to provide a singular user experience, but that's just really the icing on the cake with NetVault XA.

If we go down a little deeper into NetVault XA, once it’s is installed, learning alongside vRanger, NetVault, or both, it's going to self identify that vRanger or NetVault environment, and it's going to allow you to manage it the way that you have already set about from that ability.

New approach

We're really delivering a new approach here, one we think is going to be unique in the industry. That's the ability to logically group data and applications within lines of business.

You gave an example earlier of Oracle. Oracle is not an application. Oracle is a platform for applications, and sometimes applications span databases, file systems, and multiple servers. You need to be looking at that from a holistic level, meaning what makes up application A, what makes up application B, C, D, etc.?

Then, what are the service levels for those applications? How mission critical are they? Are they in that 50 percent of data that we've seen from surveys, or are they data that we restored from a week ago? It wouldn’t matter, but then, again, it's having one tool that everyone can use. So you now have a whole different user experience and you're taking up a whole different approach to data protection.

Gardner: There really seems to be a drilling down into these technologies and surfacing information to such a degree that it strikes me as similar to what IT service management (ITSM) did for managing IT systems at a higher level. We're now bringing that to a discrete portion backup and recovery. Does that sound about right, George, or did I overstate it?

We're really delivering a new approach here, one we think is going to be unique in the industry. That's the ability to logically group data and applications within lines of business.

Crump: No, that's dead-on. The benefits of that type of architecture are going to be substantial. Imagine if you are the vRanger programmer, when all this started. Instead of having to write half of the backend, you could just plug into a framework that already existed and then focus most of your attention on the particular application or environment that you are going to protect.

You can be releasing the equivalent of vRanger 6 on vRanger 1, because you wouldn’t have to go write this backend that already existed. Also, if you think about it, you end up with a much more reliable software product, because now you're building on a library class that will have been well tested and proven.

Say you want to implement deduplication in a new version of the product or a new product. Instead of having to rewrite your own deduplication engine, just leverage the engine that's already there.

One common means

Maxwell: By having one common means -- whether you're a DBA, a sysadmin, a VMware administrator, or a storage administrator -- you are all on the same page. You can have people all buying into one way of doing things, so we don't have this data being backed up two or three times.

But the other thing that you get, and this is a big issue now, is protecting multiple sites. When we talk about multiple sites, people sometimes say, "You mean multiple data centers. What about all those remote office branch offices?" That right now is a big issue that we see customers running into.

The beauty of NetVault XA is I can now have various solutions implemented, whether it's vRanger running remotely or NetVault in a branch office, and I can be managing it. I can manage all aspects of it to make sure that those backups are running properly, or make sure replication is working properly. It could be halfway around the country or halfway around the world, and this way we have consistency.

Speaking of reporting, as you said earlier, what about a dashboard for management? One of our early users of NetVault XA is a large multinational company with 18 data centers and 250,000 servers. They have had to dedicate people to write service-level reports for their backups. Now, with NetVault XA, they can literally give their IT management, meaning their CIO and their CTOs, login IDs to NetVault XA, and they can see a dashboard that’s been color coded.

It can say, "Well, everything is green, so everything is protected," whether it's the Linux servers, Oracle databases, Exchange email, whatever the case. So by being able to reduce that level of complexity into a single pane of glass -- I know it's a cliché, but it really is -- it's really very powerful for large organizations and small.

I can manage all aspects of it to make sure that those backups are running properly, or make sure replication is working properly.

Even if you have two or three locations and you're only 500 employees, wouldn’t it be nice to have the ability to look at your backups, your replicas, and your snapshots, whether they're in the data center or in branch offices, and whether you're a sysadmin, DBA, storage administrator, to be using one common interface and one common set of rules to all basically all get on the same plane?

Dispersed operations

So it's having a means to take an inventory and ensure that the servers are being maintained, that everything is being protected, because next to your employees, your data is the most important asset that you have.

Data is everywhere now. It’s in mobile devices. It certainly could be in cloud-based apps. That's one of the things that we didn’t talk about. At Quest we use seven software-as-a-service (SaaS)-based applications, meaning they're big parts, whether it's Salesforce.com or our helpdesk systems, or even Office 365. This is mission-critical corporate data that doesn’t run in our own data center. How am I protecting that? Am I even cognizant of it?

The cloud has made things even more interesting, just as virtualization has made it more interesting over the past couple of years. With NetVault XA, we give you that one single pane of glass with which you can report, analyze, and manage all of your data.

Mobile devices

Gardner: Just to be clear John, this console is something you can view as a web interface, and I'm assuming therefore also through mobile devices. I'm going to guess that at some point, there will perhaps be even a more native application for some of the prominent mobile platforms.

Maxwell: It’s funny that you mentioned that. This is an HTML5-based application. So it's very new, very fresh, and very graphical. If you look at the UI, it was designed with tablets and laptops in mind. It's gotten to where you can do controls with your thumbs, assuming you're running this on a tablet.

In-house, and with early support customers, you can log into this remotely via laptops, or tablet computing. We even have some people using them on mobile phones, even though we're not quite there yet. I'm talking about the form factor of how the screens light up, but we will definitely be going that way. So a sysadmin or storage administrator can have at their fingertips the status of what’s going on in the data-protection environment.

What's nice is because this is a thin client, a web UI, you can define user IDs not only for the sysadmins and DBAs and storage administrators, but like I said earlier, IT management.

So if your boss, or your boss’ boss, wants to dial in and see the health of things, how much data you’re protecting, how much data is being replicated, what data is being protected up in the cloud, which is on-prem, all of that sort of stuff, they can now have a dashboard approach to seeing it all. That’s going to make everyone more productive, and it's going to give them a better sense that this data is being protected, and they can sleep at night.

If you don’t have a way to manage and see all of your data protection assets, it's really just a lot of talk.

Gardner: Is there anything here going forward that will make having a process approach to a data lifecycle and backup and recovery even more important?

Maxwell: Dana, you hit on something that's really near and dear to my heart, which is data deduplication. We have a very broad strategy. We offer our own software-based dedupe. We support every major hardware based dedupe appliance out there, and we're now adding support for Dell’s DR Series, DR4000 dedupe appliances. But we're still very much committed to tape, and we're building initiatives based on storing data in the cloud and backing up, replicating, failover, and so forth.

One of the things that we built into NetVault XA that's separate from the policy management and online monitoring is that we now have historical data. This is going to give you the ability to do some capacity management and capacity planning and see what the utilization is.

How much storage are your backups taking? What's the most optimum number of generations? Where are you keeping that data? Is some data being kept too long? Is some data not being kept long enough?

For every ounce of flexibility, it feels like we have added two ounces of complexity, and it's something we just can't afford to deal with.

By offering a broad strategy that says we support a plethora of backup targets, whether it's tape, special-purpose backup appliances, software-based dedupe, or even the cloud, we're giving customers flexibility, because they have unique needs and they have different needs, based on service levels or budgets. We want to make them flexible, because, going back to our original discussion, one size doesn’t fit all.

Crump: Just to tie in with what John said, we need flexibility that doesn’t add complexity. Almost everything we've done so far in the environment up to now, has added flexibility, but also, for every ounce of flexibility, it feels like we have added two ounces of complexity, and it's something we just can't afford to deal with. So that's really the key thing.

Looking forward, at least on the horizon, I don't see a big shift, something like virtualization that we need to be overly concerned with. What I do see is the virtual environment becoming more and more challenging, as we stack more and more VMs on it. The amount of I/O and the amount of data protection process that will surround every host is going to continue to increase. So the time is now to really get the bull by the horns and institute a process that will scale with the business long-term.

Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy. Sponsor: Quest Software.

You may also be interested in: 

• For Dell’s Quest Software, BYOD puts users first -- and with IT’s blessing
• New Levels of Automation and Precision Needed to Optimize Backup and Recovery in Virtualized Environments
• Ocean Observatories Initiative: Cloud and Big Data come together to give scientists unprecedented access to essential climate insights
• Case Study: Strategic Approach to Disaster Recovery and Data Lifecycle Management Pays Off for Australia's SAI Global
• Columbia Sportswear extends deep server virtualization to improved ERP operations, disaster recovery efficiencies

HP BSM software newly harnesses big-data analysis to better predict, prevent, and respond to IT issues

HP this week announced a new version of its HP Business Service Management (BSM) software to endow IT organizations with big data analysis capabilities across mobile, hybrid, and cloud IT environments.The goal: To significantly improve the performance and availability of software services.

As organizations have adopted virtualization and cloud technologies, the complexity to effectively monitor trouble across these systems has skyrocketed. And, with the rise of shared services, IT no longer knows or controls all the technologies supporting their businesses.

So HP has broadened its BSM solutions to deliver better end-to-end visibility into IT applications and services  by exploiting powerful, real-time and historical analytics. With enhanced BSM, IT can anticipate performance and trouble issues before they happen. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

“IT organizations are looking for new ways to deliver predictable service levels," said Ajei Gopal, senior vice president and general manager, Hybrid and Cloud Products, Software at HP. “The new HP Business Service Management software delivers end-to-end operational intelligence to help IT make better decisions and improve service levels in complex, dynamic IT environments.”

Operational analytics

New to HP BSM is HP Operational Analytics (OpsAnalytics), a capability that delivers ongoing intelligence about the health of IT services by automating the correlation and analysis of consolidated data, including reams of machine data, logs, events, topology, and performance information.

OpsAnalytics is enabled through the integration of HP ArcSight Logger, a universal log management solution, with correlation capabilities of HP Operations Manager i (OMi), and the predictive analytics of HP Service Health Analyzer (SHA). This combination delivers deep visibility and insight into nearly any performance or availability issue, so, says HP, IT operators can:

• Remediate known problems before they occur with predictive analytics that forecast problems and prioritize issues based on business impact
• Proactively solve unknown issues by collecting, storing, and analyzing IT operational data to automatically correlate service abnormalities with the problem source
• Resolve incidents faster with knowledge based on historical analysis of prior similar events that contains search capabilities across logs and events.

HP BSM further helps clients maximize IT investments with end-to-end visibility across heterogeneous environments, enabling clients to:

• Ensure service availability with a 360-degree view of IT performance, gathered by aggregating data from disparate sources into a single dashboard using out-of-the-box connectors to a range of management frameworks, including IBM Tivoli Enterprise Console and IBM Tivoli Monitoring and Microsoft System Center
• Resolve and improve performance of applications running in OpenStack and Python cloud environments with diagnostics that pinpoint performance bottlenecks
• Improve availability of web and mobile applications through greater insight into client side performance issues.

HP also enables virtualization administrators to diagnose and troubleshoot performance bottlenecks in highly virtualized environments with HP Virtualization Performance Viewer (vPV), which helps reduce operational resources by up to 70 percent and decrease time to problem resolution by up to 50 percent, and is available as a free download, said HP.

The free versions of HP Virtualization Performance Viewer (vPV) and HP ArcSight Logger are available to download from www.hp.com/go/vpv and www.hp.com/go/opsanalytics respectively.

You may also be interested in:

• Right-sizing security and information assurance, a core-versus-context journey at Lake Health
• HP Discover Performance Podcast: McKesson Redirects IT to Become a Services Provider That Delivers Fuller Business Solutions
• Investing Well in IT With Emphasis on KPIs Separates Business Leaders from Business Laggards, Survey Results Show
• Expert Chat with HP on How Better Understanding Security Makes it an Enabler, Rather than Inhibitor, of Cloud Adoption
• Expert Chat with HP on How IT Can Enable Cloud While Maintaining Control and Governance
• Expert Chat on How HP Ecosystem Provides Holistic Support for VMware Virtualized IT Environments

Right-sizing security and information assurance, a core-versus-context journey at Lake Health

Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy. Sponsor: HP.

Welcome to the latest edition of the HP Discover Performance Podcast Series. Our next discussion examines how regional healthcare services provider Lake Health in Ohio has matured from deploying security technologies to becoming more of a comprehensive risk-reduction practice provider internally for its own consumers.

We learn how Lake Health's Information Security Officer has been expanding the breadth and depth of risk management there to a more holistic level -- and we're even going to discuss how they've gone about deciding which risk and compliance services to seek from outside providers, and which to retain and keep on-premises.

Here to explore these and other security-related enterprise IT issues, we're joined by our co-hosts for this sponsored podcast, Chief Software Evangelist at HP, Paul Muller, and Raf Los, Chief Security Evangelist at HP.

And we also welcome our special guest, Keith Duemling, Information Security Officer at Lake Health. The discussion is moderated by Dana Gardner, Principal Analyst at Interarbor Solutions. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

Here are some excerpts:

Gardner: Many people are practicing IT security and they're employing products and technologies. They're putting in best practices and methods, of course.

But you have a different take. You've almost abstracted this up to information assurance -- even quality assurance -- for knowledge, information, and privacy. Tell me how that higher abstraction works, and why you think it's more important or more successful than just IT security?

Duemling: If you look at the history of information security at Lake Health, we started like most other organizations. We were very technology focused, implementing one or two point solutions to address specific issues. As our program evolved, we started to change how we looked at it and considered it less of a pure privacy issue and more of a privacy and quality issue.

Go back to the old tenets of security, with confidentiality, integrity, and availability. We started thinking that, of those three, we really focused on the confidentiality. But as an industry, we haven't focused that much on the integrity -- and the integrity is closely tied to the quality.

Information assurance

So we wanted to transform our program into an information-assurance program, so that we could allow our clinicians and other caregivers to have the highest level of assurance that the information they're making decisions based on is accurate and is available, when it needs to be, so that they feel comfortable in what they are doing.

As background, Lake Health is a not-for-profit healthcare system. We’re about 45 minutes outside of Cleveland, Ohio. We have two freestanding hospitals and approximately 16 satellite sites of different sizes that provide healthcare to the citizens of the county that we’re in and three adjacent counties.

We have three freestanding 24×7 emergency rooms (ERs), which treat all kinds of injuries, from the simple broken fingers to severe car accidents, heart-attacks, things of that nature.

It's not just protecting information from being disclosed, but it's protecting information so that it's the right information.

We also have partnerships with a number of very large healthcare systems in the region, and organizations of that size. We send some of our more critically injured patients to those providers, and they will send some of their patients to us for more localized, smaller care closer to their place of residence.

We’ve grown from a single, small community hospital to the organization that we have now.

I've been with Lake Health for a little under eight years now. I started as a systems administrator, managing a set of Windows servers, and evolved to my current position over time.

Typically, when I started, an individual was assigned a set of projects to work on, and I was assigned a series of security projects. I had a security background that I came to the organization with. Over time, those projects congealed into the security program that we have now, and if I am not mistaken, it's in its third iteration right now. We seem to be on a three-year run for our security program, before it goes through a major retrofit.

So it's not just protecting information from being disclosed, but it's protecting information so that it's the right information, at the right time, for the right patient, for the right plan of care.

From a high level, the program has evolved from simple origins to more of a holistic type of analysis, where we look at the program and how it will impact patient care and the quality of that patient care.

Gardner: It sounds like what I used to hear -- and it shows how long I have been around -- in the manufacturing sector. I covered that 20 years ago. They talked about a move toward quality, and rather than just looking at minute or specific parts of a process, they had to look at it in total. It was a maturity move on behalf of the manufacturers, at that time.

Raf Los, do you see this as sort of a catching up time for IT and for security practices that are maybe 20 years behind where manufacturing was?

Raf Los

Los: What Keith’s group is going, and where many organizations are evolving to, is a practice that focuses less on “doing security” and more on enabling the enterprise and keeping quality high. After all, security is simply a functionof -- one of the three pillars -- of quality. We look at does it perform, does it function, and is it secure?

So it's a natural expansion of this, sort of a Six Sigma-esque approach to the business, where IT is catching up, as you’ve aptly put it. So I tend to agree with it.

Gardner: Of course, compliance is really important in the healthcare field. Keith, tell us how your approach may also be benefiting you, not just in the quality of the information, but helping you with your regulatory and compliance requirements too?

Duemling: In the approach that we’ve taken, we haven’t tried to change the dynamics of that significantly. We've just tried to look at the other side of the coin, when it comes to security. We find that a lot of the controls that we put in place for security benefit from an assurance standpoint, and the same controls for assurance also benefit from a security standpoint.

As long as we align what we're doing to industry-accepted frameworks, whether it’d be NIST or ISO, and then add the healthcare-specific elements on top of that, we find that that gives us a good architecture to continue our program, and to be mindful of the assurance aspect as well as the security side.

Add-on benefits

One of the other benefits of the approach is that we look at the data itself or the business function and try to understand the risks associated with it and the importance of those functions and the availability of the data. When we put the controls and the protective measures around that, we typically find that if we're looking specifically at what the target is when we implement the control, our controls will last better and they will defend from multiple threats.
So we're not putting in a point solution to protect against the buzzword of the day. We're trying to put in technologies and practices that will improve the process and make it more resilient from both what the threats are today and what they are in the future.

Paul Muller

Muller: A couple of observations ... The first is that we need to be really careful when we think about compliance. It's something of a security blanket, not so much for security executives. I think InfoSec security executives understand the role of compliance, but it can give business leaders a false sense of security to say, "Hey, we passed our audit, so we're compliant."

There was a famous case of a very large financial-services institution that had been through five separate audits, all of which gave them a very clear bill of health. But it was very clear from some of the honey pots they put in place in terms of certain data that they were leaking data through to a market-based adversary. In other words, somebody was selling their data, and it wasn’t until the sixth audit that it uncovered the source of the problem.

So we need to be really careful. Compliance is actually the low bar. We're dealing with a market-based adversary. That is, someone will make money from your data. It's not the nation-state that we need to worry about so much as the people who are looking to exploit the value of your information.

Of course, once money and profit enter the equation, there are a lot of people very interested in automating and mechanizing their attack against your defense, and that attack surface is obviously constantly increasing.

The challenge, particularly in examples such as the one that Keith is talking about, comes in the mid-sized organizations. They've got all of the compliance requirements, the complexity, and the fascinating, or interesting, data from the point of view from a market-based adversary. They have all of that great data, but don't necessarily have the scale and the people to be able to protect that.

Balancing needs

It's a question of how you balance the needs of a large enterprise with the resources of a mid-sized organization. I don't know, Keith, whether you've had any experience of that problem.

Duemling: I have all too many times experienced that problem that you’re defining right there. We find that technology that helps us to automate our situational awareness is something that's key for us. We can take the very small staff that we have and make it so that we can respond to the threats and have the visibility that we need to answer those tough questions with confidence, when we stand in front of the board or senior management. We're able to go home and sleep at night and not be working 24×7.

Los: Keith, let me throw a question at you, if you don't mind. We mentioned automation, and everybody that I have with this conversation with tends to -- I don't want to say oversimplify -- but can have an over-reliance on automation technology.

In an organization of your size, you’re right smack in the middle of that, too big not to be a target, too small to have all the resources you've ever wanted to defend yourself. How do you keep from being overrun by automation -- too many dashboards, too many red lights blinking at you, so you can actually make sense of any of this?

Duemling: That's actually one of the reasons we selected HP's ArcSight. We had too many dashboards for our very small staff to manage, and we didn’t want Monday to be the dashboard for Product A, Tuesday for Product B, and things of that nature.

So we figured we would aggregate them and create the master dashboard, which we could use to have a very high-level, high-altitude view, drill down into the specific events, and then start referring them to subject-matter experts. We wanted to have just those really sensitive events bubble up to the surface, so that we could respond to them and they wouldn’t get lost in the maze of dashboards.

We wanted to have just those really sensitive events bubble up to the surface, so that we could respond to them and they wouldn’t get lost in the maze of dashboards.

Gardner: How did you unify all of these different elements under what you call a program for security? What were some of the steps you needed to take? We heard a little bit about the dashboard issue, but I'm trying to get a larger perspective on how you unified culture around this notion of information assurance?

Duemling: We started within the information and technology department where we had to really do an evaluation of what technologies we had in place? What are different individuals responsible for, and who do they report to? Once we found that there was this sprinkling of technology and responsibilities throughout the department, we had to put together a plan to unify that all into one program that has one set of objectives, is under one central leadership, and has its clear marching orders.

Then once we accomplished that, we started to do the same thing across the entire organization. We improved our relationship within IT, not just with sub-departments within IT, but then we also started to look outside and said, "We have to improve our relationship with compliance and we have to improve our relationship with physical security."

So we’re unifying our security program under the mantra of risk, and that's bringing all the different departments that are related to risk into the same camp, where we can exchange notes and drive towards a bigger enterprise focused set of objectives.

Los: At the end of the day, what security is chartered with, along with most of the rest of IT, as I said earlier, is empowering the organization to do its work. Lake Health does not exist for the sole purpose of security, and clearly they get that.

That's step one on this journey of understanding what the purpose of an IT security organization is. Along the broader concept of resiliency, one of the things that we look at in terms of security and its contribution to the business is, can the organization take a hit and continue, get back up to speed, and continue working?

Not if, but when

Most organization technologists by now know it’s not a question of if you’re going to be hacked or attacked, but a question of when, and how you’re going to respond to that by allowing the intelligent use of automation, the aligning towards business goals, and understanding the organization, and what's critical in the organization.

They rely on critical systems, critical patient-care system. That goes straight to the enterprise resiliency angle. If you get hacked and your network goes down, IT security is going to be fighting that hack. At the same time, we need to realize how we separate the bad guys from the patient and the critical-care system, so that our doctors and nurses and support professionals can go back to saving lives, and making people’s lives better, while we contain the issue and eradicate it from our system.

It's more than just about security, and that's a fantastic revelation to wake up to every morning.

Gardner: Are there some other returns on investment (ROI), maybe it's a softer return like an innovation benefit or being able to devote more staff to innovation?

Duemling: I'd put forward two paybacks. One is about some earlier comments I heard. We, as an organization, did suffer a specific event in our history, where we were fighting a threat, while it was expected that our facilities would continue operating. Because of the significant size of that threat, we had degraded services, but we were able to continue -- patients were able to continue coming in, being treated, things of that nature.

That happened earlier in our program, but it didn’t happen to the point where we didn’t have a program in place. So, as an organization, we were able to wage that war, for lack of a better term, while the business continued to function.

So we can demonstrate more of an ROI through an improvement in situational awareness and security intelligence.

Although those were some challenging times for us, and luckily there was no patient data directly or indirectly involved with that, it was a good payoff that we were able to continue to fight the battle while the operations of the organization continued. We didn't have to shut down the facilities and inconvenience the patients or potentially jeopardize patient safety and/or care.

A second payoff is, if we fast forward to where we are now, lessons learned, technologies put in place, and things of that nature. We have a greater ability to answer those questions, when people put them to us, whether it's a middle manager, senior manager, or the board. What are some of the threats we're seeing? How are we defending ourselves? What is the volume of the challenge? We're able to answer those questions with actual answers as opposed to, "I don't know," or "I'll get back to you."

So we can demonstrate more of an ROI through an improvement in situational awareness and security intelligence that we didn't have three, four, or five years earlier in the program’s life. And tools like ArcSight and some of the other technologies that we have, that aggregate that for us, get rid of the noise, and just let us hone in on the crown jewels of the information are really helpful for us to answer those questions.

System of record

Gardner: How about looking at this through the lens of a system of record perspective, an architectural term perhaps, has that single view, that single pane of glass, allowed you to gain the sense that you have a system of record or systems of record. Has that been your goal, or has that been perhaps even an unintended consequence?

Duemling: It's actually kind of both. One, it retains information that sometimes you wish you didn't retain, but that's the fact of what the device and the technology are in the solution and it’s meeting its objective.

But it is nice to have that historical system of record, to use your term, where you can see the historical events as they unfold and explain to someone, via one dashboard or one image, as a situation evolves.

Then, you can use that for forensic analysis, documentation, presentation, or legal to show the change in the threat landscape related to a specific incident, or from a higher level, a specific technology that's providing its statistical information into ArcSight, but you can then do trending and analysis on.

It is also good to get towards a single unified dashboard where you can see all of the security events that are occurring in the environment or outside the environment that you are pulling in, like edit from a disaster recovery (DR) site. You have that single dashboard where if you think there's a problem, you can go to that, start drilling down, and answer that question in a relatively short period of time.

Muller: I'll go back to Keith’s opening comments as well. Let's not undervalue the value of confidence -- not having to second guess not just the integrity of your systems and your applications, but to second guess the value of information. It's one thing when we're talking about the integrity of the bank balance of a customer. Let's be clear that that's important, but it can also be corrected just as easily as it can be modified.

When you're talking about confidence in patient data, medical imaging, drug dispensations, and so forth, that’s the sort of information you can't afford to lack confidence in, because you need to make split-second decisions that will obviously have an impact on somebody’s life.

Let's not undervalue the value of confidence -- not having to second guess not just the integrity of your systems and your applications.

Duemling: I would add to that. Like you were saying, you can undo an incorrect or a fraudulent bank transfer, but you cannot undo something such as the integrity of your blood bank. If your blood bank has values that randomly change or if you put the wrong type of blood into a patient, you cannot undo those without there being a definitely negative patient outcome.

Los: Keith, along those lines, do you have separate critical systems that you have different levels of classifications for that are defended and held to a different standard of resilience, or do you have a network wide classification? I am just curious how you figure out what gets the most attention or what gets the highest concentration of security?

Duemling: The old model of security in healthcare environments was to have a very flat type of architecture, from both networking, support, and a security standpoint. As healthcare continues to modernize for multiple reasons, there's a need to build islands or castles. That’s the term we use internally, "castles," to describe it. You put additional controls, monitoring, and integrity checks in place around specific areas, where the data is the most valuable and the integrity is the most critical, because there are systems in a healthcare environment that are more critical than others.

Obviously, as we talked about earlier, the ones that are used for clinical decision making are technically more critical than the ones that are used for financial compensation as it results from treating patients. So although it's important to get paid, it's more important that patient safety is maintained at all times.

Limited tools

We can't necessarily defend all of our vast resources with the limited set of tools that we have. So we've tried to pick the ones that are the most critical to us and that's where we've tried to put all the hardening steps in place from the beginning, and we will continue to expand from there.

We look at every security project with the mindset of how we can do this the most effectively and with the least amount of resources that are diverted from the clinical environment to the information security program.

That being said, security as a service, cloud-based technology, outsourcing, whatever term you would like use, is definitely something that we consider on a regular basis, when it comes to different types of controls or processes that we have to be responsible for. Or professional services in the events of things like forensics, where you don’t do it on a regular basis, so you may not consider yourself an expert.

We've tried to pick the ones that are the most critical to us and that's where we've tried to put all the hardening steps in place.

We tend to do an evaluation of the likelihood of the threat materializing or dependence on the technology, what offerings are out there, both as a service and premise-based, what it would take from an internal resource standpoint to adequately support and use a technology. Then, we try and articulate that into a high-level summary of the different options, with cost, pros and cons related to each.

Then, typically our senior management will discuss all of those, and we'll try and come to the decision that we think makes best for our organizations, not just for that point, but for the next three to five years. So some initiatives have gone premise-based and some have gone security-as-a-service based. We are kind of a mix.

Gardner: It's interesting that a common thread for successful organizations is knowing yourself well. It's also an indicator of maturity, of course.

You have had a good opportunity to know yourself and then to track your progress. Is that helping you make these decisions about what's core or context in the design of your risk-mitigation activities?

What you do well

Duemling: Yes, it is. You have to know what you do well and also you have to know the areas where you, as an organization, are not going to be able to invest the time or the resources to get to a specific comfort level that you would feel would be adequate for what you are trying to achieve. Those are some of the things where we look to use security as a service.

We don't want to necessarily become experts on spam filtering, so we know that there are companies that specialize in that. We will leverage their investment, their technology, and their IP to help defend us from email-borne threats and things of that nature.

We're not going to try and get into the business of having a program or to create an event-correlation engine. That's why we're going to go out and look for the best-of-breed technologies out there to do it for us.

IT security is just another enabler in the business and we should really continue to treat it that way and work towards that goal.

We'll pick those different technologies, whether it's as a service or premise-based and we'll implement those. That will allow us to invest in the people that know our environment the best and intimately and who can make decisions based on what those tools and those managed services tell them.

They can be the boots on the ground, for lack of a better term, making the decisions that are effective at the time, with all the situational awareness that they need to resolve the problem right then and there.

Gardner: For those of our listeners who are perhaps juggling quite a few security products or technologies and they would like to move into this notion of a program, and would like to have a unified view -- any thoughts about getting started, any lessons learned that you could share?

Duemling: I would say just a couple of bullet points. Security is more than just technology. It really is the people, the process, and the technology. You have to understand the business that you are trying to protect. You have to understand that security is there to support the business, not to be the business.

Probably most importantly, when you want to evolve your security and set up projects into an actual security program, you have to be able to talk the language of the business to the people who run the business, so that they understand that it’s a partnership and you are there to support them, not to be a drain on their valuable resources.

Los: I think he has put it brilliantly just now. IT security is a resource and also a potential drain on resources. So the less we can take away from anything else the organization is doing, while enabling them to basically be better, deliver better, deliver smarter, and save more lives and make people healthier, that is ultimately the goal.

If there's nothing else that anybody takes away from a conversation like this, IT security is just another enabler in the business and we should really continue to treat it that way and work towards that goal.

Lessons learned

Gardner: All right, last word to you today, Paul Muller. What sort of lessons learned or perhaps perceptions from the example of Lake Health would you amplify or extend?

Muller: I will just go back to some of my earlier comments, which is, let’s remember that our adversary is increasingly focused on the market opportunity of exploiting the data that we have inside our organizations -- data in all of its forms. Where there is profit, as I said, there will be a drive for automation and best practices. They are also competing to hire the best security people in the world.

But as a result of that, and mixed in with the fact that we have this ever-increasing attack surface, the vulnerabilities are increasing dramatically. The statistic I saw from just October is that the cost of cyber crime has risen by 40 percent and the attack frequency has doubled in the last 12 months. This is very real proof that this market forces are at work.

Cyber crime has risen by 40 percent and the attack frequency has doubled in the last 12 months. This is very real proof that this market forces are at work.

The challenge that we have is educating our executives that compliance is important, but it is the low bar. It is table stakes, when we think about information and security. And particularly in the case of mid-sized enterprises, as Raf pointed out, they have all of the attractiveness as a target of a large enterprise, but not necessarily the resources to be able to effectively detect and defend against those sorts of attacks.

You need to find the right mix of services, whether we call it hybrid, whether we call it cloud or managed services, combined with your own on-premises services to make sure that you're able to defend yourself responsibly.

Gardner: I'd like to thank our supporter for this series, HP Software, and remind our audience to carry on the dialogue with Paul Muller through the Discover Performance Group on LinkedIn, and also to follow Raf on his popular blog, Following the White Rabbit.

You can also gain more insights and information on the best of IT performance management at http://www.hp.com/go/discoverperformance.

And you can always access this and other episodes in our HP Discover Performance Podcast Series at hp.com and on iTunes under BriefingsDirect. Thanks!

Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy. Sponsor: HP.

You may also be interested in:

• HP Discover Performance Podcast: McKesson Redirects IT to Become a Services Provider That Delivers Fuller Business Solutions
• Investing Well in IT With Emphasis on KPIs Separates Business Leaders from Business Laggards, Survey Results Show
• Expert Chat with HP on How Better Understanding Security Makes it an Enabler, Rather than Inhibitor, of Cloud Adoption
• Expert Chat with HP on How IT Can Enable Cloud While Maintaining Control and Governance
• Expert Chat on How HP Ecosystem Provides Holistic Support for VMware Virtualized IT Environments

Market confidence in cloud soars, especially among service providers, says North Bridge survey

Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy. Access the survey.

A new survey about cloud computing explores the business growth opportunities for buyers and consumers of cloud services alike, with surprising findings about confidence and a high degree of ongoing experimentation.

The multi-year annual survey on the cloud market provides a springboard for examining some of the implications for where the growth opportunities are and where the inhibitors for the growth may be.

For more details on the survey, go to:

 http://www.zdnet.com/blog/btl/the-future-of-cloud-computing-9-trends-for-2012/80511

To learn more about where the cloud business has been and where it’s going, BriefingsDirect sat down with Michael Skok, Partner at North Bridge Venture Partners. The interview is conducted by Dana Gardner, Principal Analyst at Interarbor Solutions.

Before joining North Bridge in 2002, Skok had himself been an entrepreneur and CEO in the software business for 21 years.

Hear the results of this multi-year annual survey on the cloud market.

He founded, led, and attracted more than $100 million in venture backing to his investments in multiple successful software companies. As a venture capitalist himself, Skok has invested in many entrepreneurs who have together built more than a $1 billion of value, focusing on large market-changing technologies and disruptive business models such as software as a service (SaaS), cloud computing, open source, and mobile.

Current representative investments include Acquia, Akiban, Apperian, Demandware (NYSE:DWRE), and Unidesk, as well as Actifio and Revolution Analytics.

Skok's passion for innovation and entrepreneurship is also fueling his work mentoring and developing the next generation of entrepreneurs. For example, he is currently developing and leading workshops such as the "Startup Secrets" series with the Harvard i Lab. You can follow him at www.mjskok.com and @mjskok.

Here are some excerpts from the conversation:

Gardner: Is there anything to indicate from your survey and your experience lately that there is a waning interest or enthusiasm for cloud?

Skok: Obviously there's an increasing interest in understanding cloud, but as cloud has captured so much attention, there is also a significant interest in understanding what the real applications and potential for it are. People are trying to get beyond the hype, at this stage, to understand the practical applications and opportunities.

Gardner: Is it fair to say that confidence is up because the perceived risks are down, or are we still working through how confident people are and whether there are significant risks here?

Skok: Maybe the best way to answer that is to give you some specific data from the survey, and rather than have my commentary, it will give you the market’s viewpoint on this. That’s one of the key reasons we run the survey -- to try to understand what vendors and customers believe are some of the key issues, both driving and inhibiting the cloud.

So I'll jump in and give you some of the inhibitors first to answer your question on risk, for example, and then perhaps we can talk about some of the drivers.

On the inhibitors, one of the things that’s interesting this year is that, if you look back to 2011, 10 percent of the survey respondents would have said that the cloud is just too risky, and they gave many reasons last year. This year, we're down to 3 percent. So that’s a significant drop.

Michael Skok

Now, I'd argue that 3 percent says that you're at a point where people are beginning to understand cloud better, because the issues that they are raising are things like data sovereignty and the Patriot Act. Those are very real issues that are unlikely to just disappear, and they are beyond just cloud. They have to do with the reality of how people have to run their businesses.

The good news is that 12 percent feel that the cloud still needs to mature. That's not so significant number, but it’s down from 26 percent in 2011. So again, people are starting to feel that the cloud is obviously meeting more of their needs.

When you look at the issues behind those 12 percent who are looking for greater maturity, there are things that again you would expect to see in an early-stage market -- things like security and compliance, and that’s very typical.

If you looked at any major trend that comes into the marketplace, if you looked at the initial early days of the web and eCommerce, people said things like, "We'll never put our credit cards on the web." Now, not only do we put our credit cards on the web, but we allow people to do Internet banking and take photos of the checks as a means to make deposits from their cellphones.

So things have come a long way, and that’s just the time-scale that it takes. It’s typically several years before things mature and get people confident in these kinds of applications.

Encouraged by results

So I'm encouraged by those results. The next obvious thing that comes out of the survey is how many people are still experimenting. About a third are experimenting, 34 percent to be precise, with concepts in the cloud, driving applications, and using the cloud in some innovative ways.

For example, you see companies like Bank of America, who do trials using the cloud, and if they are successful, they use the cloud’s elasticity to quickly expand their trials. If they're not, they just throw them away. That’s a great example of how the cloud is specifically enabling people to do trials and get to market faster and be more effective.

And the other side of the coin, the great news this year is the rapid growth in confidence overall in the marketplace. If you had asked how many people had complete confidence in 2011, you would have gotten an answer about 13 percent, and this year it was fully 50 percent.

So we're not quite at a tipping point, because you have to double-click on that 50 percent. You have to understand the split between vendors and customers, and vendors were over half. In fact, 56 percent of them have complete confidence in the cloud. So you're seeing net new development in cloud from independent software vendors (ISVs), absolutely the tipping point. You see very few companies starting up today that aren’t building in a cloud.

If you look at the customers, they're not quite at that same level of confidence.

But if you look at the customers, they're not quite at that same level of confidence. Just over a third, 37 percent in fact, have complete confidence. More of them are experimenting and waiting for it to mature, as we were just talking about, and some of them still feel it’s too risky.

So it’s a long answer to your question. I hope it gives you some substance backed up by the survey to get a sense of this, and I am happy to answer any questions behind that.

Gardner: It’s interesting that those who are in the cloud ecosystem themselves are very confident, and you'd think that they would have the most to lose. They're making their investments, but the longer tail toward the consumer side is still catching up to that.

It certainly seems optimistic for the market in general that those in the know -- those that are using these to build business -- that they themselves will be providing cloud services and are so confident.

Skok: It turns out that there’s an interesting representation of players in the survey here, in that we have got both vendors and users responding. There were over 785 in total, mostly C-suite, but more than a third of it are customers.

Of the vendors that are represented, we're covering everything from Amazon to Citrix, to some of the mid-tier players like Rackspace, Red Hat, and others, and also up-and-coming and emerging players, for example, Eucalyptus and Acquia.

Bridge the gap

So it’s a very good breadth of players to drill one level beneath this, and we did that. We tried to understand what’s going to bridge the gap between vendor’s confidence and user’s confidence and we heard five specific things.

Number one, people want more complete value propositions. A lot of what’s being sold at the moment is technology and what people really want is the second key thing, which is clear business benefits. And they want that in the form of case studies, which is the third thing that would help people.

The fourth thing is more proof of specific opportunities that are being addressed in their industry, the vertical specific applications if you will. The bottom line, the fifth thing, is that people want greater return-on-investment (ROI) case studies to be presented to them so that they can put that forward as they champion this on an economic basis.

So to answer your question in summary, Dana, what we'll see is this gap between the confidence in the cloud the vendors are seeing and what users are seeing it is going to get bridged, as we become more able to deliver on the benefits with specific examples that drop right to the bottom line.

The beauty of the survey is that it represents a broad swath, about 40 of the key vendors.

By the way, the full results of the survey are available on our site at mjskok.com. Just look under the "Industry for Cloud," and you'll see "Future Cloud."

This year’s survey is an opportunity to get a level set as to what’s going on in the industry, where are we, and to understand what’s going on in the key drivers and inhibitors, because everybody in the ecosystem is trying to understand how to better address the tsunami that’s rolling over the industry in cloud computing. The results were gathered in the summer of 2012, and they're continuously updated.

So the beauty of the survey is that it represents a broad swath, about 40 of the key vendors, both driving and enabling cloud, and also key buyers and C-suite members who are trying to evaluate and deploy cloud.

The idea behind the survey obviously is to enable both sides to get a better understanding of how to take actionable steps toward implementing what might be the next generation of IT. Pretty much everybody recognizes cloud as the platform on which not just applications and solutions are going to get built, but IT is going to transform to the next generation of providing itself as a service in an effective form.

Independent survey

For example, we're in constant conversations with these vendors and also with the CIOs to continue to keep them fresh. But while we sponsor it, 40 collaborators are driving it. Again, the details of that are on the web, but the point is that it’s an independent survey so that no one vendor is driving it, it’s a collaboration of the industry as a whole to ensure that it's an independent survey.

Gardner: One of the things that jumped out at me, as you were trying to define what we could start to call loosely "killer applications of the cloud," where this is going to get traction, clearly one of the areas was platform as a service (PaaS). So let’s address that. Then, there's also big data -- fast data, analytics in the cloud. How prominent were they in the survey in terms of the priorities or the endgame for these two types of uses?

Skok: That’s a great question. You only skipped one, so I'll cover it briefly. The most surprising thing is just how much SaaS has gained in the survey since last year.

We also worked with Goldman Sachs, to give credit to them, and some of the information is also pulled from the industry as a whole. We found that 67 percent of the survey respondents are already deploying SaaS applications, and the value that people are seeing is in the application solving real business problems.

Respondents were saying that 75 percent of them thought that they would be building software with PaaS in the next five years, which is a big jump.

Of course, SaaS is built on PaaS and infrastructure as a service (IaaS) too. The important thing that you are pointing out is that there was a significant jump of interest in PaaS this year. In fact, looking forward to the future, the respondents were saying that 75 percent of them thought that they would be building software with PaaS in the next five years, which is a big jump.

We have a viewpoint on that, and I'll come back at it in a second, but what’s interesting here is that people recognize that they're going to be building applications. Why would they build them in anything other than in a cloud-based manner? That’s what’s so interesting here.

Now, I'll come back to that, because there’s some interesting controversy around how PaaS will play out and that came out of the survey too. But to talk a little bit about what you were describing as key application areas, big data was certainly one of them. It was top of the list on what people thought would be changed by cloud. As far as which application categories would be disrupted most, big data was at the top of the list.

Beneath that, were others that wouldn't surprise you, for example, customer relationship management (CRM). With Salesforce having led that charge, it’s not surprising that people see that continue to be a key area.

What was exciting to me was that number three was eCommerce. In our own portfolio, for example, we saw one of my investments, Demandware, go public this year and that was real evidence to me that you're going to be able to build confidence in mission-critical applications.

eCommerce applications, like Demandware, are the front door representing major vendors and brands, and people can track the nature of their business literally second by second and measure how much revenue would be lost if eCommerce applications were down.

Mature and strong

So the fact that major retailers and brands now bet billion of dollars on eCommerce as a service gives you a sense that people feel like the technology is in place and mature, strong, and reliable enough for them to back it with their brand and have it at their front door. That was very interesting.

Gardner: Just to expand on that a bit, in addition to retail and consumer side eCommerce, we saw SAP acquire Ariba. So there is obviously some interest in the B2B side as well.

Skok: Exactly. The B2B side is very early, and there is tremendous potential there too. We think that’s relatively untapped and that there's great white space there. You're quite right.

Gardner: So continuing down your list.

Skok: The list obviously is long, but what we did was to look forward and try to understand some of the key areas that are driving cloud and some of the opportunities. I'll cover what we talked about as the future cloud formations and the potential opportunities for applications.

They fall into what we call five cloud formations, and we're specific in talking about formations, as opposed to cloud-washed opportunities. What we mean by that is that you've seen a lot of vendors try to bring out just another level of their application and host it in some shape or form and deliver it via the cloud. That’s really not what we're talking about here.

We think the future is in applications that have been built specifically for the cloud.

Those kinds of things that aren’t true multi-tenant applications that are born in the cloud, and we think they're not the real future here. We think the future is in applications that have been built specifically for the cloud and enable you to do things that you wouldn’t find possible should you not have had the cloud available to you.

The formations we talk about first are media and entertainment. People have gotten used to that with iTunes and their music and Netflix to get their movies online. That was a major revolution and it started initially with web ordering where Netflix was delivering physical DVDs. As the pipes got fatter, we could just physically deliver over the web, and you're seeing more and more of those opportunities.

If you look at gaming, it has also all gone online, and people are taking it for granted. That’s actually a lot of what drove the cloud initially. This media and entertainment formation is very real, here to stay, and we think has tremendous opportunity, especially as the mobile platform expands too.

The second key area is what we call social and collaboration. The social and collaborative cloud is very much understood by people who use Facebook in the consumer world. What's interesting is that it has moved into the enterprise with applications important to supply chain management that are enabling things like tighter inventory control.

Also, there's collaboration all the way down to the customer, so that people can get better service and support, and in many instances self-service, which has a great cost savings and ROI payback.

Easier to collaborate

You're seeing that now start to play out. People are getting used to the fact that it's so much easier to collaborate in the cloud than it is to try to send people on-premise applications to work with, when you want to collaborate with them. We'll see a great expansion of that going forward, too.

The third key area, which I would describe as almost a platform shift, is identified as mobile and that includes location data, too. Mobile, if you think about it, is not possible without the cloud. Again, it goes to a real, true cloud application.

These devices that we carry with us, smart as they are, are nothing without the connections back to the cloud, to be able to do everything from synchronizing our contacts, calendars, and email, to much more important and significant things, such as to connect back to business processes and provide such key information as price lists and contracts for the people in the field to be able to do their job in situ.

That’s a really important shift, and the incredible rise, it's unparalleled, of new devices like the iPad, which has been the fastest growing device ever, in both consumer and enterprise, are giving rise to new demands and new services.

eCommerce has really become something that people take for granted that they can do over the web.

What's perhaps obvious when you think about it, but less obvious in this context, is how much location data is being generated from that. We'll talk about that in terms of the big data formation in a second, but location data is providing new opportunities for new applications. That links nicely to the fourth key cloud formation that we think about. That's commerce and that includes payments.

eCommerce, as we were just talking about, has really become something that people take for granted that they can do over the web. It's not just Amazon anymore, as you said, it's even B2B commerce, for example, that companies are taking a lot of the supply chain, collapsing it, and taking out cost.

That’s being enabled by the cloud. As mobile payments and the payment system in general become more accessible by the cloud, which is more of a political challenge than it is a technical one, that will become a very interesting opportunity for new applications that will be spawned and connected back to the cloud.

All of those applications, as I started to hint at with location data, are generating a huge amount of data, and that’s giving rise to the big data cloud. Big data is interesting on two fronts. It's interesting because with every click and step we take we're creating information that is being collected in the cloud, in a form that you can consider part of the big-data opportunity.

What's interesting on the second side of the coin is that the cloud itself provides the kind of scale, indeed economy of scale, for crunching that data, analyzing it, and providing insight from it.

The fact that you can spin out an analysis of anything from the human genome to a click stream in the cloud, and then provide insight, in some cases in real time, to drive applications wherever they may be and reach them with things like your mobile devices, is really changing the game.

Cloud formations

So these five cloud formations: media and entertainment, social collaboration, mobile and location, eCommerce and payments, big data and analytics, are where we think cloud is dramatically changing the scope of the landscape.

When you look at them, what's really exciting here is what's happening at the intersection. I'd be happy to give you an example of that, if it's useful to you.

Gardner: What's very fascinating to me, Michael, is not just these impressive arenas that you have described on their own, but how they intersect and in many ways multiply each other -- being mobile, having the big data to crunch, relating that data into a commerce activity, and bringing that back out through collaboration or social activities. It's really the whole greater than the sum of the parts here. Please explain a bit where you think that is going or where the survey tells you it's going?

Skok: You said it very well. The sum is greater than the parts here, and you've obviously picked right up on it. We could give you many examples, but I'll take one that’s simple, so that everybody can relate to it.

It used to be that if you thought about going to see a movie, you would have to go and check your local listings, but obviously people are way beyond that today. We can go right online and if it's not available to you at Netflix, you can quickly check to see where it is available on your local cinema from your cellphone geo tag where you are and it can quickly tell you that the closest place to go to see the movie.

Of course, you can use commerce in the cloud to buy it on something like Fandango. Then what's interesting is that you can choose at that time to check out what your friends think of the movie, see the collaboration that’s been going on of reviews from people that you know, and decide whether it's that movie or something else you should see.

At the application level, the big game changer is going to be what I call social commerce.

So you're using all of the things we are just talking about, media and entertainment, social collaboration, mobile and location, commerce and payment, to do all of that.

What gets to be exciting is all that data that’s being generated, if you go and see the movie, or if you rate it yourself, it gets fed back to you in things like recommendations for the next movie you might want to see, or if you take your kids, the kind of merchandizing that follows up with offers to you, and payments that can drive you to make further additional purchases.

And that’s just a simple example. There are many others I can think of that are, exactly as you say, the whole being much greater than the sum of these individual client formations. It's really quite game changing.

Gardner: So who are the beneficiaries? Clearly there is a business to be had providing cloud services and in integrating process benefits across some of these domains. You can sell hardware and software. You can build new business models by either giving consumers things they couldn't get before or making what they had done before far more efficient and productive. But where is the margin?

This gets to the business of cloud. We see Amazon being very aggressive on price, maybe racing to the bottom on some of the commodity services for IaaS for example. And we certainly expect a lot of competition between the likes of Google and Microsoft for cloud and PaaS types of services. Salesforce of course is in there.

But where is the point in all of this where you could say, "Here is another Apple with the iPad. Here is the margin. Here is the place where the business is as revolutionary as the productive benefits of cloud activities?"

Three examples

Skok: Very good question. I'm going to give you three examples at the different levels: so one at the application level, one at the PaaS level, and then one at the infrastructure level. I hope that will be helpful.

At the application level, the big game changer is going to be what I call social commerce. It's the intersection of two of those cloud formations, if not three of them, which is social connections and recommendations, connected with eCommerce, and potentially mobile within there too.

You're going to see there is tremendous opportunity, because what people most rely on when they are actually buying things is their friends and trusted recommendations, and we're very early in that. Surely, people have begun to recognize the power of the like button, but we haven’t yet seen that translate into commerce. We're early in Facebook trying to realize that.

The other extreme, the eCommerce companies, are taking off doing what we call omni-channel commerce, connecting everything from bricks and mortar, and are also recognizing the power of being able to do that as people are out and about with the mobile devices and gaining data on, for example, local offers and so forth.

The next great opportunity is going to come in the combination between social and commerce, and it might involve mobile and local as well. We haven’t seen the next great company emerge from that, but we're certainly seeing many opportunities. At the application level, that’s probably a good example.

People are looking for more analytics, and more of the capabilities that are going to be specifically taking advantage of cloud scale.

To deliver on all of that, one of the things we're taking for granted is that the infrastructure is going to be in place to do all that. A part of the survey that we always take time to ensure we cover is to understand the things that people are actually spending money on right now.

If we look at the intersection between vendors and users, and in the survey it's a slide called "Rainmakers," at the bottom of the infrastructure stack there's still a tremendous amount to do to enable the kinds of applications that you and I are talking about here.

Some things are very basic, the things like single sign-on on authentication to enable this collaboration across the supply chain. More specifically, in mission-critical businesses, it's things like backup, archiving, and business continuity to ensure that all this information is being stored and managed on a significantly scalable basis.

When we looked at all that, the thing that stood out, which is not going to surprise you probably, given that we talked about big data, is that people expect one of their greatest areas of spend to be analytics.

So at the infrastructure level, I think we are going to see some of the things that I talked about that are basic, like next generation of single sign-on. But the big thing that came out was that people are looking for more analytics, and more of the capabilities that are going to be specifically taking advantage of cloud scale.

Insights in real time

Whether that’s using things like Hadoop or next generation NoSQL or NewSQL, our capability is to get those kind of insights in real-time. In the end, the more data that’s being generated, the more we're going to have to step up the scale of analytics to provide insight in an effective time scale.

Those two would exemplify the application opportunities and the infrastructure opportunities. In the middle, as we talked about earlier, there’s a great deal of interest in PaaS, and it's less clear to me what the opportunity is for a specific breakout.

I'll say both what the survey revealed and what it didn’t reveal, which is interesting. We talked about how it revealed that there is a strong interest in PaaS, but when we dig in with vendors, what we see is that the vendors are actually at the bottom of the stack. The IaaS vendors, people like Amazon, VMware, and others, are actually trying to add more capabilities to their IaaS platform, to enable them to feel more like a PaaS.

If you look at Amazon, they've added numerous new services to make themselves more platform like, and they have become the de facto standard there. So they are moving from the bottom upward.

But you also see the SaaS vendors, exemplified by Salesforce.com, introducing their PaaS, like Force.com, to extend the use of their infrastructure or their applications to be more platform like too. There's a pretty big squeeze from the top and the bottom that’s making it difficult to see what will be the white space for a PaaS vendor.

People have historically very rarely made money out of tools. I don't think it will be any different in the cloud.

The honest truth is that I can describe the first two, what the opportunities for the SaaS and IaaS are, but it's not clear to me where the white space is in PaaS, and it feels like it's getting squeezed, if that makes sense.

Gardner: So to sum up, perhaps there is a significant business to be had up and down the spectrum, infrastructure, hardware-software, facilities, management, building out the applications, but perhaps one of the larger two opportunities that's yet to be solidified or clear is in the analytics and in PaaS.

Now, in the past, development was often a tricky market to make money in -- tools, frameworks, IDEs, but in many cases there was a deferment involved. You might break even or even lose money on some of those areas in order to capitalize on the deployment side or even gain lock-in for those applications on a platform, and that's where you would have a very good business.

I think what we're seeing with cloud is something a bit different. When it comes to lock-in, and you have had experience of course in open source software, what are some of the good things and some of the more risky things when it comes to this desire, as we've seen in the past, to lock people in to either a platform, a service, a standard, or even a toolset?

Skok: You're on the money on a number of different fronts. First of all, as you say, people have historically very rarely made money out of tools. I don't think it will be any different in the cloud. The interesting piece in the cloud is you have the runtime potential to make money, but even then, it's an economy of scale game, so it's not a place that's easy for startups to play.

Platform lock-in

The second key point you're making is that people traditionally have looked at it as a means to get lock-in to a platform, and that is the exact thing that people are worried about in this cloud revolution too. The third biggest item of what's inhibiting cloud adoption in the survey is lock-in, and the fourth was interoperability. They were both very high on the ranking.

What people are worried about there is very simple. If we double-click on it, they're looking for three things to avoid lock-in. They want to avoid data lock-in, they want to avoid programmatic lock-in with application programming interfaces (APIs), and they want to avoid being locked into proprietary services or features that can't be transparently supported on other platforms.

That's a real challenge for the PaaS players at this point, because the giant here is Amazon, and they've got a series of de-facto standards. There are some companies like Eucalyptus who have been very smart and are reverse engineering or making sure they are compatible with those standards.

But those that are trying to compete on new grounds are certainly going to have to struggle with gaining critical mass and then answer the question about how they'll provide that interoperability on those three layers we just talked about, to get over that inhibitor of an adoption that people are worried about around lock-in.

People will have open access to the source to modify, adopt, and even change to create their own abstraction layers.

Gardner: So perhaps there's a de-facto standard around Amazon, but being challenged by OpenStack and CloudStack as well. Is there any inference in the survey as to whether the OpenStack and CloudStack approaches would mitigate a de-facto standard evolving rapidly, and how do you view that?

Skok: I'm going to slightly branch outside of the survey and mention that for several years, we've run an equivalent industry survey on open source. It's very widely adopted now, but when we started several years ago, it was early.

We've seen that cloud has very much become a part of open source, not just because a lot of cloud is built on open source, but because, as you say, people are looking at open source as a means to answer this lock-in. It answers one of the key areas, which is certainly programmatic, an API type lock-in.

People will have open access to the source to modify, adopt, and even change to create their own abstraction layers, but that will potentially enable this kind of interoperability.

Things like OpenStack, CloudStack, OpenShift, and other platforms are potentially an answer to that. The challenge there is that they're relatively young and early in their adoption. While they've got significant backing, you have yet to see broad deployment of them yet.

I'm hopeful that open source will provide some of the answer to vendor lock-in. It's certainly being proposed that way and it's being supported that way. If you talk to a certain segment of the user population, they would tell you that it's exactly what they're relying on, but in reality, we're too early to call that one.

Making good money

Gardner: One observation from me would be that the folks that are in a position to make good money on infrastructure, hardware or software facilities, and management, seem to be a natural affinity environment with the OpenStack, CloudStack approach, but those higher up the food chain in cloud that have more of a pure-services business model might be interested in having the de facto standard land in their particular data center. It will be interesting to see how that pans out.

Tell us once again, Michael, how people can get more information on your survey. Where could they go to get the nitty-gritty?

Skok: They can just go to mjskok.com under Industry, Future of Cloud Computing, and the full survey is available from that site on a slideshow for people to click through. Also, it's being covered in many different places by many of the vendors who have supported it. There's a lot of information being disseminated by the collaborators. You have full access to it.

Just to answer your question, because it's too good a question, who has what interest to go where? It's best exemplified by Oracle. Oracle took a long time to enter the cloud market. Of course, they have benefit all the way from hardware out to the applications because of the acquisition of Sun.

That's how they're pushing their cloud approach as a series of applications that are totally integrated from hardware, all the way through to software. That's certainly going to suit some class of buyers.

If you look at major waves like this, it's always a while before people can afford to have best of breed at various different layers.

But if you look at major waves like this, it's always a while before people can afford to have best of breed at various different layers. If you started building application, as we did in some of our investments like Demandware eight or nine years ago, there was no IaaS, there was no real depth of Amazon and no service-level agreements (SLAs) that you could have built a mission-critical eCommerce application on.

That is evolving, and the more stable and capable the IaaS and PaaS players become, new applications will be to take advantage of those, and new vendors will potentially be able to take advantage of best of breed. That's what's interesting about the surveys, but it's all about verifying and tagging the state of the industry to see where we are and benchmark how the future is going to play out.

Gardner: Perhaps what we're seeing is a flip from best of breed being a technology to best of breed being a service or ecosystem approach. And if you can perhaps sweeten the offer of moving your best of breed mentality in that direction by not locking people in, or at least giving them an option to have interoperability, or mobility of their services, then that might be an irresistible offer that the market can't refuse. We just don't know who is going to make it, right?

Skok: That's exactly right. That's perfectly said. A good example to highlight how this is still playing out is Zynga, who reverse burst to their own zCloud because the economies of scale made it worth their while to do that.

If you look forward, people are even talking about cloud brokerages. I think it's too early to do that. Forrester had some thoughts about that and was talking about cloud brokers like travel agents. I think we are a ways off from that.

But in the ultimate scenario, exactly as you were talking about it, you might see a place where you have best of breed, cloud services, and all kinds of cloud formations that we were talking about.

Best of breed

Applications will effectively be an amalgamation of the best-of-breed cloud services and cloud formations that will enable new classes of applications that have interoperability, or at the bare minimum of things like data that's passed up and down supply chains or along applications streams. The consumer is the ultimate benefactor, because they're getting those, not only at best of breed, but hopefully at the lowest cost and at highest value.

Gardner: Then, perhaps it would be embedded services across those best of breed processes that would include widespread analytics, mobility, and location services, so those become more sweeteners to the offer. There would be a race to who can put together the best banquets of services under the best interoperability terms and licensing terms. So again, it could be a very interesting next five years.

I assume that over the next several years, you're going to be continuing to do this survey each summer and therefore get the gravitas that we have seen with your open-source survey.

Skok: Indeed. There's been unbelievable response to it. In fact, just to give you a sense of it, the open-source survey took a number of years to gain the kind of momentum that it's now enjoying in its seventh year here.

This survey gained such incredible popularity that within the first couple of years, it already has as much support from the industry as the entire open-source survey does.

This survey gained such incredible popularity that within the first couple of years, it already has as much support from the industry as the entire open-source survey does. And we have got tremendous demand to continue doing it, from both vendors and customers alike.

We're continuing to use it to keep dialogue between vendors and customers and enhance the industry’s ability to respond to what they see as the future. So  with your support, we will continue to do it.

 By the way, the full results of the survey are available on our site at mjskok.com. Just look under the "Industry for Cloud," and you'll see "Future Cloud."

Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy. Access the survey.

You may also be interested in: 

• Thomas Duryea’s journey to the cloud: Part one
• CloudNOW Unveils its 2013 Cloud Computing Predictions
• For Dell’s Quest Software, BYOD puts users first -- and with IT’s blessing
• Businesses Starting to Use Cloud Without Knowing What They'll Do With It
• SOA provides needed support for enterprise architecture in cloud, mobile, big data, says Open Group panel
• IDC Cloud Computing Survey Shows Hybrid Model Increasingly Attractive
• Cloud Computing -- Why Waiting Might Make Sense
• Cost-Efficient Cloud Computing System Bridges Gap Between the Rich and the Poor