HP delivers NMC 9.1 as new demands on network management require secure, integrated, and automated response

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a transcript or download a copy. Learn more. Sponsor: HP.

The IT news headlines are full of incidents of major cloud instances brought down for days, and unfortunately often weeks, with some of the largest of these due to network issues in association with virtualization and storage sprawl. The price in the cloud era for such disruptions is very high and very public.

A big part of the solution to preventing such outages comes from comprehensive, automated, and increasingly integrated network management capabilities. The tasks before network managers have never been more daunting. There are far more devices, hybrid networks, hybrid compute resources, higher levels of virtualization, and there is a need to maintain security and compliance requirements throughout.

What’s more, the pressure to keep cost down and to seek lower cost alternatives for converged infrastructure remains a constant companion to business and IT architects, and therefore an ongoing network challenge.

Into this environment, HP this week delivered a wide-ranging update to its Network Management Center suite Version 9.1. The emphasis is on a comprehensive lifecycle approach to network management with deep data gathering, automated root cause analytics, and intelligent and proactive response features that enable consistently high performance and network reliability.

BriefingsDirect recently sat down with Ashish Kuthiala, Director of Product Marketing for HP Software’s Network Management Center, to dig into the new offerings and to better understand why previous fragmented approaches to network performance and stability just won’t hold up for most enterprises. The discussion is moderated by Dana Gardner, Principal Analyst at Interarbor Solutions.

Here are some excerpts:

Gardner: What it is about the new IT environment that is taxing the older ways of network management?

Kuthiala: When you're looking at the network today, it has become very complex and is increasingly becoming more complex. With new domains coming in, such as voice over IP (VOIP), webcasts, and video traffic, multiprotocol label switching (MPLS) services, unified communications, and cloud computing and virtualization, it just becomes a nightmare to manage your network for your business.

Then, you look at the volume of network devices coming online. Now, everyone wants to be in the instant-on enterprise mode. Everyone has to be connected. Everything has to be connected. Everyone expects immediate gratification and instant results. You have to respond to this opportunity continuously, and "any time, anywhere, any way" is the new tagline for anybody who is working.

Let’s look at the job of the director of network ops in a particular IT organization. Not only does he have to configure, manage, and standardize a network, he has to provision, he has to deliver, and he has to report on it. He has to do it very proactively and he has to do it very strategically at the lowest cost possible.

IT budgets are shrinking or remaining flat, whereas the demands on IT are really going up. It’s estimated that a customer can lose about $70,000 a minute during network outage, as I'm sure you’ve seen in the recent news. It's a big business inhibitor if the network goes down. It is what provides the experience to the end user for all the IT services that they experience.

Gardner: Why isn’t the previous mode of network management able to keep up?

Kuthiala: Today, if you were to look into a customer’s IT department managing a network environment, you would often see a war-room like approach to managing networks. ... They're very reactive. They have multiple tools, legacy approaches, and a lot of band-aids. The inability in tying together what used to be separate domains has become unacceptable.

If your shopping cart goes down doing the Christmas shopping season, and a customer tells you about it, that is just unacceptable.

The inability to cope up with the scale and complexity, the different teams hunched over their different monitors, is what I call the "swiveling chair syndrome." If there is a network outage, you have these 8 or 10 different operators looking at different aspects of the network. They are just swiveling in their chairs, talking to each other and looking for data that should really be on one screen for them to manage. The lack of scalability of such tools just adds to the problem.

Built-in intelligence

Gardner: How does an automated approach work better?

Kuthiala: To manage your network today, you really need to understand how your network is constructed from the bottom up, how it ties together, how it changes over time, and how it self-organizes. You need to build that kind of intelligence into your root-cause analysis.

The design of the tools has to be built ground up, based on these decisions. That’s how you need to construct the tools. That’s how they need to be integrated. For an operator, all these need to build upon each other.

It has to be in the right context. It cannot be siloed. It is a nightmare to manage. The desired nirvana for a network team is to reduce the numerous point tools to manage various aspects of network management. It has to be proactive, not reactive.

You have compliance management diagnostics and change issues that you need to take human error out of, and you need to automate that. You want to reduce the manual effort, the errors and increase control over your environment. You want to reduce the mean time to repair network outages, and maintain cost optimization as your network grows.

Today for customers, “performance is the new fault." So just because a network device is up and running, and you can ping it, doesn't mean it is providing the quality of service it should to the end user. It’s really the performance that the network is being measured against.

It’s all about efficiency, how you reduce your errors, and increase your speed through automation.

... Customers are looking for a solution that's efficient, automated, and secure for them. When they manage a network, they should be able to do things like fault, performance, change, configuration, compliance, trending and reporting, and this ties into their business services.

Long history

So, HP looked at this problem. As you know, we've had a long history of about 20 years with the HP OpenView product in network management. As we acquired other companies such as Opsware, they bought in additional tools with them. We looked at the tools and the evolving landscape of the network management domain and about five years ago, embarked on a re-architecture plan for these products from the ground up.

The approach wasn’t to make these products just work together by putting in connectors, but we wanted them to be integrated from bottom up, from the data level itself, where the data would build upon each other.

Now, as we look at the Network Management Center (NMC), it is a complete portfolio of solutions and tools that lets you do network management in an integrated and automated way.

This really builds upon the HP Network Node Manager i (NNMi), the related special plug-ins that handle complex services such as multicast traffic, VOIP, etc., as well as the network automation piece of it which really helps customers automate and manage their change, compliance, and configuration of network devices that they need to do on an ongoing basis.

The five-year journey of re-architecting our NMC portfolio completes with the 9.1 release that we are talking about today.

So, the earlier 9.0 release introduced a number of features including better user interfaces, the ability to scale to large environments, and tying our products together into better functioning solutions. With 9.1, we are building on that.

We've strengthened the ability of our customers to manage cloud services. The most critical capability that a customer must have is to manage the network the same way that they have managed traditional networks, and it doesn’t matter if they have to go across the cloud or are looking at private or public clouds.

Gaining visibility

Gaining visibility into the network elements, whether they are local, off-premise or the health and quality of the cloud services that's being delivered, is the most important step. Can I reach my device? Is it healthy? Is it performing to the expected levels of business needs?

And, of course, configuration compliance management of these devices across the cloud is very important, and corrective actions and rollbacks are very important. Our tools are able to do that across different environments.

The 9.1 release is also focused on the managed service provider’s (MSP's) market needs. There is a big trend of IT outsourcing to MSPs, and one of the things that customers want to outsource is network management services. So this is a big, growing market, and our MSPs need platforms to manage their customers' network environments in a way that that maximizes their profit.

They need to scale and grow with their customer in expanding network environments, reduce their hardware spend and their training costs, as well as grow their revenues and create new lines of business, as their own customers move to new and complex services.

For example, a customer might go from traditional phones to IP telephones, and at that point, the MSP has to manage that aspect of their customer’s environment as well, and they don’t want at this point to buy a new tool.

This helps them manage multiple customers, departments or sites per single software instance, driving down their cost and giving them a flexible architecture.

The size of the customer's network might increase, and you don’t want to buy another server, another set of tools and deploy another set of operators to manage that.

We have introduced multi-tenancy capability and security groups that allow our customers to separate their data and views into secure partitions. This helps them manage multiple customers, departments or sites per single software instance, driving down their cost and giving them a flexible architecture.

We’ve also done a lot of work on the performance-based, time-based thresholds for better alerting. What this means is that the performance data is in the context of the network topology providing a unique point of your fault monitoring. It helps them with proactive notification of performance degradation, fix it proactively and guarantee service delivery levels.

We've also increased the number of months that the data is retained. It's up to 13 months now which allows you to do forecasting and trending capabilities. This is a sufficient data retention period for compliance requirements for real-time and historical data, and allows a very efficient analysis.

Our user interface (UI) has been enhanced based on the feedback we’ve gotten from customers. The common look and feel UI across all the products and our solution set ensures lower training cost -- train once, leverage across all these tools.

Contextual information

The UIs show relevant contextual information on the nodes and incidents they're managing, giving them a lot of operational efficiency. The breadcrumb history and the easy navigation with right-click menus also allows the operators to get to the root cause more quickly, making them much more efficient and improving the time to resolution.

The analysis pane shows you a number of system component help enables you to get key information including availability and performance graph really quickly.

Gardner: In some of these high-profile outages that we've had recently, it seems that they were doing updates and that caused the cascading or spiraling effect and ultimately brought the network down. What is it about your suite and your comprehensive approach that could help ameliorate something like that?

Kuthiala: A network constantly needs updates, whether its configuration updates or being in compliance with a number of different policies -- Sarbanes-Oxley (SOX) or the Health Insurance Portability and Accountability Act (HIPAA), and government regulations.

When something goes wrong, you don't know what has gone wrong, and you are scrambling to fix it. Think about doing this across 50,000, 60,000, 70,000 devices in your network.

Typically, customers have a set of people who use multiple tools or manually log into a number of these devices and do these configuration changes manually. This is very dangerous. One, there is human error involved. Second, when something goes wrong, you don't know what has gone wrong, and you are scrambling to fix it. Think about doing this across 50,000, 60,000, 70,000 devices in your network.

Our network automation capabilities allow customers to automatically make these changes through our tools. As they implement these changes, it's takes minutes and hours, versus days, to keep these devices configured to the latest and greatest configurations and in compliance.

Think about when you are on the 59,000th device that you are updating and you realize there is an error. This was not the right thing to do, and you need to roll back. If you're doing this manually, you're spending many hours fixing the error while your business is suffering during that time. Our automation capabilities help customers; with a few clicks of buttons they are able to automate all of this.

Today, customers might be looking at a number of incidents -- 10,000, to 15,000 incidents. For example, if somebody yanks a LAN cord out and puts it back in, what really has happened is the interface has gone down and come back up. And now that is flagged as an incident or an event that the operator has to pay attention to.

With our root cause analysis engine, and the ability to map the topology dynamically in a spiral discovery fashion, the network topology is always up-to-date. The root cause analysis engine helps figure out whether this is an incident that needs to be paid attention to or not, auto-resolving some of that.

Meaningful incidents

The incidents that boil up to the operators are meaningful, and therefore are reduced in number to those that are actionable. We have had customers whose incidents have been reduced from 10,000-12,000 down to 400, and only about 100 of those have to be acted upon and escalated to the next level of management.

Automation really takes a lot of the work out of your hands and enables you to fix errors very proactively, and if there is a mistake, fix it right away with a few clicks.

... I'm talking very specifically about the configuration of network devices. The software that your network device comes with is the key differentiator in how they act, and the intelligence that they provide. So this has to be not only managed really well, but there are patches and upgrades, just as you have software patches and upgrades on your servers. These have to be managed. Sometimes, there are government regulations or company regulations that you want to propagate across these devices.

It's essential to understand what type of traffic is flowing on your network. This gives you the ability to optimize your network performance and network resiliency.

But tying to the business service management set of tools or the suite stems from the fact that, when you look at it from a business service availability aspect, it’s not just about the network. There are servers, there are applications, and they are all tied together. For example, if application business service is not working, do you know if it’s the server? Do you know if it’s the application? Do you know if it is the network?

Our Business Service Management offering ties in these aspects through our runtime service model. This ties your network, to your application, to your server and is able to give your business a look into how your business service is going to be affected by the failure of any one of these infrastructure elements.

Automated capabilities

Gardner: Now Network Management Center is a fairly significant set of different products, but most people already have something in place. So this is not a matter of starting greenfield. This is a matter of coexistence, migration, and transformation. How do you get started?

Kuthiala: Most customers today have in place something to monitor their networks, but a lot of customers have not automated their configuration, compliance, and diagnostic capabilities that we talked about.

We've seen a trend in our customer base where they buy smaller node packs to manage a small number of devices with our automation capabilities. Once they have put that in place, they start to see other efficiency use cases that they can achieve using our network automation capabilities.

We observe that these customers come back and buy more licenses for managing a greater number of network devices. So, that’s almost like a greenfield opportunity here.

But, when we look at the most customers looking at managing their networks and doing performance and monitoring, for example, if they have an instance of our software, it’s an in-place upgrade. We offer a dual entitlement and run a parallel program that allows customers is to seamlessly set up another parallel environment and bring the network up there, start to manage it, and seamlessly shift.

We’ve had an instance of a customer in the EMEA region, where they were testing our latest software and running it in parallel to see how it was functionally different and what effect of productivity it would have on their operators. A couple of weeks went by and their senior management started getting escalations for network problems.

Once they have put that in place, they start to see other efficiency use cases that they can achieve using our network automation capabilities.

Now, when senior management turned to the network operations team and asked, "We have all these incidents showing up. What is going on? Is something wrong?"

Almost sheepishly, the network operator team had to acknowledge that they were testing the new platform and had completely forgotten about the old tool which they needed to shut down because the new platform ignored the incidents that were not meaningful. They had “accidentally” migrated to the new platform to managing the network much more efficiently.

A lot of our customers use this approach to migrate to the new platform, and of course, our approach is modular. Start with the core product and add the special plug-ins to manage your IP telephony MPLS or multicast capabilities.

To see the HP Automated Network Management (ANM) Solution in action, you can watch a short overview and the ANM 9.10 Video Demo. This recording will explain the NMC components that make up the ANM solution and walk you through a use case to demonstrate the automated capabilities of HP Automated Network Management 9.10.

We also have an hp.com page, which is www.hp.com/go/nmc for downloading trial software, reading whitepapers, customer case studies, product capabilities and features. That’s a good starting point. We also blog about customer experiences and the stories they share with us as well.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a transcript or download a copy. Learn more. Sponsor: HP.

You may also be interested in:

• HP with FlexNetwork: You're going to have to update your network, so you might as well do it right
• HP's Robin Purohit unpacks Business Service Management 9 as way to address complexity in hybrid data centers
  
• Tag-Team of HP Workshops Provides Essential Path to IT Maturity Assessment and a Data Center Transformation
• HPs Kevin Bury on How Cloud and SaaS Will Help Pave the Way to Increased Efficiency in IT Budgets for 2011 and Beyond

Survey says: Open-source software making broad strides across the IT landscape

Once viewed in some quarters as a fringe movement and unreliable, open-source software is now a dominant force in the IT industry. It has been embraced by both the public and the private sectors and is being implemented across a wide variety of markets and applications such as social publishing and big data.

These are among the results of the fifth annual Future of Open Source Survey conducted by North Bridge Venture Partners in partnership with The 451 Group. More than 450 IT professionals took part in the survey with end users making up 60 percent of the respondents, who were asked about a wide range of issues that affect the open-source landscape. Most of those responding see an ever brighter future for open source.

Among the other findings:

• The open source community is now more focused on maturing technology concerns, including improved operational excellence around areas such as support, product management, feature functionality and return on investment, as opposed to earlier concerns around the legal implications of licensing and conforming to internal policies.

  More than half -- 56 percent -- of respondents believe that more than half of software purchases made in the next five years will be open source.

  
  
• Respondents identified software as a service (SaaS), cloud and mobile computing as the main areas that can have a dramatic impact on open source and a virtually untapped opportunity for growth.
  
• More than half -- 56 percent -- of respondents believe that more than half of software purchases made in the next five years will be open source.
  
• An overwhelming 95 percent of respondents noted that a turbulent economy continues to be “good” for open source software, though for the first year ever, lower cost has been overtaken by freedom from vendor lock-in as what makes open-source software more attractive.
  
• When asked about revenue generating strategies likely to create value for vendors, 56 percent of the respondents said that an annual, repeatable support and service agreement was the most likely.



The survey results were released during the opening panel at Computerworld’s OSBC conference which featured open source industry leaders: Jim Whitehurst, President and CEO, Red Hat; Mike Olson, CEO, Cloudera; Adrian Kunzle, Managing Director, Head of Firmwide Engineering and Architecture, JPMorgan Chase; Tom Erickson, CEO, Acquia; and was chaired by Michael Skok, General Partner, North Bridge Venture Partners.

You may also be interested in:

• Rapidly evolving IT trends make open source, agile application integration platforms more important than ever
• Open source solutions for SOA: Check your bias at the door
• WSO2 tailors open-source middleware platform for cloud-based applications, deployment models

HP with FlexNetwork: You're going to have to update your network, so you might as well do it right

As HP unveils it's new FlexNetwork Architecture (stories on ZDNet, InformationWeek and Network World), they seem to be making a bold statement about the future of corporate networks. And that is that changing requirements are inevitable, so why not build out a network that can support all the new cloud and mobile tricks you know about ... and the ones you don't?

Aside from the HP versus Cisco narrative that the press loves, there is a major need for a convergence on networks, but it's not just a convergence with the networks themselves, it's a convergence with the rest of the enterprise and the rest of the cloud and mobile requirements bubbling up fast. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

A future-proof network is more about management and security than hardware and platforms. And it's also about ecosystems: HP is partnering extensively with Citrix, Microsoft and Riverbed. No one vendor can or should be the sole network IP supplier (just like there should no be one cloud provider).

What's more, the virtualization trend that begets the cloud trend that supports the mobile trend all require intelligent networks that have security ingrained -- not gained after the fact. As workers depend more on cloud and hybrid computing services, the network is the cloud.

When I hear people complain about the risks associated with cloud, I dare them to look closely at their own mission critical networks. Often what they find are existing, in-place risks that dwarf what they fear most about the cloud. The fears about security, reliability, control, data and privacy: These risks already live on their disjointed networks.

Those networks, incidentally, are the weak link between their nice, safe, controlled data centers on premises and all the people and partners that actually need to use them. The boundary is not the enterprise, it is the ways in which their networks can adapt.

Fear your old network first

So if you fear cloud, you should probably fear your current network, for all the same reasons.

And that's because in this day and age all large-scale IT for enterprises is supported by WANs and how they play with the global stew of network service providers. This is for apps, data, communications, VOIP, media, VPN, branch, mobile ... you name it.

My modern network needs to be comptaible and secure for data center, campus, branch and WAN activities. And I need to stream and move large objects more than ever. It doesn't make sense for the CFO to ask workers to use the cloud (because it saves data center resources) when the network can't support cloud workloads and requirements, does it?

Whether you have to revisit your network architecture because of performance, costs, compliance, security or just new payloads like mobile and media, you might as well do it right. The network really should not be the weak link in the enterprise. Not any more. Not for any longer.

You may also be interested in:

• HP's Instant-On Enterprise Initiative Takes Aim at Shifting Needs of Business and Government
• Hastening Trends Around Cloud, Mobile Push Application Transformation as Priority, says Research
• Well-Planned Data Center Transformation Effort Delivers IT Efficiency Paybacks, Green IT Boost for Valero Energy
• Data Center Transformation Includes More than New Systems; There's Also Secure Data Removal, Recycling, Server Disposal

Talend delivers converged platform for better data services integration across multiple applications

Seeking to make it easier to synchronize data between applications and across enterprise boundaries, Talend on Monday delivered a unified platform for both data services and applications integration requirements.

The Talend ESB Standard Edition, built on an open source enterprise service bus (ESB), frees data services and data management processes from specific applications. By abstracting them as standards-based services that can be reused by other applications, the new Talend platform offers a common environment for users to manage an entire lifecycle of a data service, regardless of its origins.

Talend also announced the 4.2 version of its Data Integration, Data Quality and Master Data Management (MDM) solutions, which now work in combination with the new Talend ESB. [Disclosure: Talend is a sponsor of BriefingsDirect podcasts.]

Thanks to such trends as cloud, hybrid computing and massive data sets, the role and impact of integration has shifted. A more comprehensive and managed approach to integration is required -- one that spans data and applications services. Moreover, the tools that support enterprise integration need to useable by more types of workers, those that are involved at the business process and data analysis levels.

By seeking to reduce middleware complexity, Talend's combined offerings unify a platform with a common development, deployment and monitoring environment that spans both data management and application integration tasks and operations, said Pat Walsh, Vice President of Marketing for the Application Integration Division at Talend.

Many touch points

"There’s now the mandate that you can no longer isolate data from application, because the touch points are just so many. You now need to look at solutions that, from the get-go, consider both aspects of the integration problem -- the data aspect and the system and application integration aspect," said Walsh.

Talend ESB Standard Edition uses the Apache CXF services framework, Apache Camel integration framework and Apache ActiveMQ enterprise messaging capabilities. Talend's ESB Standard Edition also features Service Locator capabilities for automatic failover and load balancing through the Apache Zookeeper extension for dynamic endpoint registration and lookup. The Security Token Service (STS) framework supports SAML 2.0 (Security Assertion Markup Language 2.0), and Service Activity Monitoring fosters analysis of service activity.

"We've gone to great lengths to include security mechanisms into the solution," said Walsh, "so that we can have approaches whereby there are certain permissions for just individuals. Or, IT management can look at certain aspects while opening it up maybe to a broader audience, when it comes to development and use of the interfaces that are going to be developed on the data in application side."

Talend ESB Standard Edition source code is licensed under the Apache 2.0 License. A commercial edition is also available through the company.

You may also be interested in:

• Apache Camel Addresses Need for Discrete Infrastructure for Services Mediation and Routing
• Apache CXF: Where it's Been and What the Future Holds for Web Services Frameworks
• Workday Integration Cloud debut raises bar on integration as service, deeply ingrains integration as apps function
• Eclipse Swordfish OSGi ESB enters fray for SOA market acceptance, Sopera to add support
  

Open Group cloud survey shows a lack of IT preparedness, especially on measuring cloud's true costs and benefits

While more than 90 percent of companies are using -- or plan to use -- cloud computing, the cloud model is raising some concerns, as well as some paradoxes.

Unsurprisingly, security is the leading concern, while integration and governance issues are close behind. The paradox arises because industry executives know that cloud computing will impact their business, but are not yet prepared to handle that impact and, in many cases, don't yet know how to measure it.

These are some of the results from a recent survey by The Open Group, which polled 307 IT professionals who had purchasing or decision-making influence over cloud computing. The study earlier this year found that cloud computing required C-level approval at 55 percent of the companies, while 22 percent left it to IT directors and 8 percent to enterprise architects. [Disclosure: The Open Group is a sponsor of BriefingsDirect podcasts.]

Among the take-aways from the results: "You need to take the metrics and find out what your IT finances are to make a true business decision about cloud," said Dr. Chris Harding, Forum Director for The Open Group Cloud Computing Work Group. "And you need to track that as your move forward."

Among other findings from the survey:

• An impressive 92 percent of respondents said they are either currently using some cloud (49 percent) or had it on their IT road map (43 percent). The remaining 8 percent said they had no cloud plans.
  
  
• Only 17 percent of those polled said they use or would use public cloud, while the remainder were divided between private (29 percent) and hybrid (45 percent). Another 9 percent were unsure what they would use.

  The paradox arises because industry executives know that cloud computing will impact their business, but are not yet prepared to handle that impact and, in many cases, don't know how to measure it.

  
  
  
• The main drivers behind cloud adoption were cost (21 percent), timeliness and agility (19 percent), and resource optimization (17 percent).
  
  
• Data security is the biggest concern for companies (18 percent), followed by integration issues (15 percent), and governance (14 percent).
  
  
• A majority of participants (55 percent) said that cloud return on investment (ROI) would be easy to justify. However, only 35 percent of respondents said they had mechanisms in place to effectively measure ROI.
  
  
• An overwhelming 82 percent said they expected cloud to significantly impact one or more business processes, but only 28 percent are actually prepared for these changes.
  
  
• When asked if they were satisfied with cloud education and training available, 51 percent percent said they were satisfied or very satisfied, 34 percent said they were somewhat satisfied, and 15 percent said they were dissatisfied.

This survey builds on the ongoing work being orchestrated by The Open Group Cloud Computing Work Group, and the results will help guide its future development on the financial and business impact of cloud computing.

The results indicate that these enterprise architects and planners are primarily focused on private and hybrid computing, and not so much on so-called public cloud options, said Harding. The respondents expect that cloud will impact their business processes, but are not sure how.

Enterprises should start right away with measurement of shared services use and costs, said Harding. That will allow for any move to cloud with assurances that risks and rewards can be managed. Only by learning the internal costs can the right decision be made as to how external services affect the balance, he said.

If you are interested in getting involved with The Open Group Cloud Computing Work Group, visit http://www.opengroup.org/cloudcomputing/. For Cloud Computing resources published by The Open Group, visit: http://www.opengroup.org/cloud/whitepapers/.

Due out in mid-2011, The Open Group currently writing a book tentatively titled Cloud Computing for Business: The Open Group Guide, which will demonstrate a model for measuring costs, and for how to begin managing governance around costs and service-level agreements, said Harding.

You may also be interested in:

• Open Group cloud panel forecasts cloud as spurring useful transition phase for enterprise architecture
• Good insights from The Open Group Conference -- and its unconference brethren
• Cyber security top of mind for enterprise architects at Open Group conference
• Cloud brokering: Building a cloud of clouds

Rapidly evolving IT trends make open source, agile application integration platforms more important than ever

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or download a copy. Register for CamelOne. Sponsor: FuseSource.

Enterprise integration requirements are rapidly shifting to accommodate such trends as cloud computing, mobile devices' explosion, and increased demand for extended enterprise business processes.

Application-to-application integration inside an enterprise's four walls is well understood, but very quickly the demands placed on integration are spanning multiple enterprises, multiple types of applications, and varieties of service providers. As a result, software as a service (SaaS) and cloud computing are joining with legacy systems to form new and varied hybrid models that require whole new sets of integration needs and challenges.

Once these newer breeds of integrations are set up, can the old, brittle management and upkeep of them suffice -- or will agility and rapid upgrades and innovations require new tools to make integration a lifecycle function with ongoing management and more automated governance?

In the latest BriefingsDirect enterprise IT discussion, the panel examines how open-source integration projects like Apache Camel and lightweight integration implementations and graphical tools are making developers and architects more agile. At the same time, these open-source approaches are proving less vulnerable to the complexity, fragility, and cost that often plague aging commercial middleware integration products. [Learn more about the CamelOne conference May 24 in Washington, DC.]

Dana Gardner, Principal Analyst at Interarbor Solutions, recently sat down with Rob Davies, Chief Technology Officer at FuseSource, and Debbie Moynihan, Vice President of Marketing at FuseSource, to examine the need for innovative, new, open and agile integration capabilities.

Here are some excerpts:

Gardner: The need for integration is increasing. The things that need to be integrated are increasing, rapidly. Open source is well established. When you put these factors together this perhaps spells an historic shift. Has the ability to integrate openly become an essential ingredient of businesses?

Davies: Sometimes, it's difficult to see things happening like that, if you’re actually right inside in the middle of it. We probably are at that shift right now.

We’ve talked about cloud environment. Also there’s social networking, SaaS, and mobile devices, and you need to link all those together. It's coming to the point where organizations won’t have a choice other than to use open source as a way to try to keep up with a pace of change.

We're probably at a point now where we’re going to see that the traditional model of providing software is going to dwindle over time, probably pretty rapidly, as organizations realize that they need the flexibility and the ability to change what they’re doing very quickly.

Future-proofing applications

You have to start thinking about how you're going to future-proof your applications right from the beginning to adapt to changes in their environments. You have to architect in how you’re going to integrate and future-proof your applications, because it does get more costly if you do it as an afterthought.

Gardner: Many of the SaaS providers are doing multitenancy and providing applications as services on demand at a very attractive and aggressive price point. They're leveraging open source on the back end, I have to imagine. Do you have any insight into what the service providers themselves are building with?

Davies: Most applications now -- in particular in the cloud -- are using open source at the back end. We can't give you any specific details of vendors that are doing that, but I know they're using open-source projects, and not just the SaaS vendors, but some of the other existing product vendors use open-source as well to enable their products.

We certainly see open-source as definitely mainstream now, and we’ve seen it has been the first choice that people use for building any kind of application or service they’re providing. It's more a case of people asking the questions now of not should we be using open source, but why shouldn’t we use open source? It's starting to become a first choice for people to go to.

Gardner: Debbie, why do people need to rethink integration?

Moynihan: The business models are changing and people are being asked to do more with less. Teams and applications are more distributed than ever.

There are a lot of new technologies coming out that people are struggling to learn, and figuring out how to incorporate them into their infrastructure: cloud, mobile, the explosion of the huge amounts of data that enterprises are trying to understand and make sense out of. Not to mention the social media technologies that people are being asked about and wondering how to incorporate into their enterprise infrastructure.

There are a lot of different skills that people are looking to have that they've never been asked to have before. More and more people are being asked to perform IT tasks. It isn’t just highly skilled developers, but also business analysts and people who have never done integration before are being asked to do integration activities.

They're not sure how to keep up with all of these changes. Costs are a problem because essentially everyone has the same or smaller budget going forward and a lot of people have fewer people to do what they've been doing before.

At FuseSource, we've seen a lot people looking more and more to open source to solve some of these problems ... . There's a lot of flexibility. When the environment changes and new technologies come out, you need to integrate new things into your environment.

The community people, when they see a problem or new technology, just make it happen. They can add, expand, and modify what's involved in the various open-source integration projects without the overhead and bureaucracy of some of the traditional software development environments.

Gardner: In the past, when we had a shift in computing, we'd bring in a new set of applications, we'd update our platforms, and then think about integrating them. It was a sequential process and it could take three to five years to go through something like that.

We don’t really have that luxury anymore. Now things are happening in a simultaneous fashion. So integration really can't be an afterthought, but needs to be part-and-parcel with how you go about designing and implementing your applications.

Doesn’t open source, in a sense, allow for a compression of the time that we’ve traditionally taken with commercial products?

Moynihan: Absolutely. Open-source is a componentized, lightweight approach. As people develop their applications, they develop them in such a way that they can be broken apart in new and different ways down the road, and it's very transparent. It makes it easier over time to further integrate what you’ve built and to make changes as you need.

Gardner: One of the other aspects of this that I'm seeing in the market is that more people need to take part in integration. It can't just go through a bottleneck of "beard-and-sneaker guys" in the back room who can do coding. Integration needs to be part of process innovation. That means we need to elevate it out to a wider group of individuals, maybe as many as possible that are on the front lines of process innovation and analysis.

The addition of tooling is going to help broaden how many people can do integration, and we're real excited.

What's being done about the integration that we've been describing to make it more, well ... applicable?

Moynihan: On April 11, we announced the general availability of a new graphical tooling for Apache Camel. [Users can download a trial version of the plug-in, which includes some of the functionality of the fully paid version found on the subscription-based Fuse Mediation Router.]

The addition of graphical tooling makes it easier for more people to do integration development. They don't have to write code. They can use a drag-and-drop environment to select the integration patterns that they want to implement, and the software will implement them. They can test them and deploy them into production as well.

The addition of tooling is going to help broaden how many people can do integration, and we're real excited. We've been doing a beta program since the end of January with over 500 participants. Rob mentioned the breadth of all the components and how hot Apache Camel has been. We're not surprised that more and more people want to use it. So, the idea of having tooling on top of it is really attractive to users.

Gardner: So, what's the name and where do you go to find out more about them?

Moynihan: The Fuse IDE for Camel is the name. It plugs into an Eclipse environment and you can get it at fusesource.com.

Gardner: You know it strikes me that when we begin to talk about integration that I’d mentioned service-oriented architecture (SOA), but that was sort of yesterday’s buzzword. We're now into cloud, hybrid, and mobile. But, from an architectural perspective, you can't really scale and leverage these open components without that proper underpinning, typically an enterprise-service-bus (ESB) architecture.

Rob, help me understand why doing this correctly from an architecture -- not just an open-source -- perspective is really important as well.

Davies: You hit the core things about the SOA and the ESB architectures. We see where people are using, in particular, Apache Camel and some of our other open-source projects. They want flexibility there. So, they want to leverage a service bus, put things on, expose them as service, and expose them over the service bus, which uses different transports to enable that bus, be that messaging, HTTP, or whatever other means you want to use.

Application integration

At the same time, you also want to have the flexibility now to do it in application integration. You want to have that flexibility for some services and you very much need that enterprise service bus in place. But for other cases, you want to be able to do that more locally, where the integration points are.

The approach that we have is that we enable you to do both, because you can embed Apache Camel inside an application server, if you want it inside your application itself. If you want to use it in a more traditional sense, you can deploy it into ServiceMix. You can define your apps easily, deploy them into ServiceMix, and use it to manage the container.

Having that flexibility as well means that you can have the right architecture for your particular solution. If you look at how people would do the integration before, they’d have to get an ESB, and that would force the whole architecture of how they do things. When you’ve got more flexibility, it means that you can make the right architecture choices that you need, and you're not constrained to one particular style of integration.

Gardner: I'm facing a lot of questions more recently about how to architecturally cross the domains that we've mentioned -- SaaS, cloud, on-premises, traditional architecture, and private cloud architecture.

Does the service-bus approach and the open-source approach also give us some sort of a path or vision for how to go about this?

You can only really get that speed of innovation to keep up with the way the environment is changing by choosing open source.

Davies: Having open source enables you to have the insight into how the integration application works.

If you just look back just a couple of years, when people were starting to use the cloud, they weren’t even thinking about having hybrid clouds. Now, we're seeing more and more people, more of our customers, looking to hybrid clouds and have a private cloud for applications.

When they need the capacity, obviously they can get that capacity in a public cloud. But, to have all those PCs working together seamlessly, they need the agility that you get from an integration solution that can be deployed on a public cloud, locally, or a combination of both. That’s something that you can only get from software that has evolved at the same pace as the demands of the environment.

You can only really get that speed of innovation to keep up with the way the environment is changing by choosing open source, because the open-source community itself is driving the projects to keep up with the demands.

So, you have to try to move outside of a traditional release cycle that you would get from a traditional product company. You don’t really have any other alternatives, if you want to keep up, than to look at open-source projects, the Apache ones in particular. [Learn more about the CamelOne conference May 24 in Washington, DC.]

Apache projects certainly hit the right notes in that you've got both very business-friendly license from the Apache license and very active communities, and you’ve got diversity in that community. You know these projects are going to live beyond the lifetime of particular individuals on the projects.

Support and consultancy

You also have the benefit of having companies like FuseSource, which created the projects in the first place, and who are there and able to provide support and consultancy if you need it. You get the best of having a dynamic community, a dynamic project, and you also get the security of having professional company to back it up.

Gardner: How rapidly are the iterations within the Apache project, within Camel in particular, happening? How rapidly is innovation taking place?

Very fast pace

Davies: It’s happening at a very fast pace. When we do release these out of Apache, it's typically every three months, but in that three-month period there could be other components that have gone into the Apache Camel Framework. Because it's open source, people can actually look about, release their own components into an open-source environment, or develop them separately without necessarily releasing to Apache, just to get the functionality out.

That pace of change is very fast and it’s near real-time. When the need comes up, within a few days or a week, you would probably find someone who has already written that integration component that you need and it’s available. ... If you’ve got an open-source framework, you can actually have an insight into how the project works.

After we launched Apache Camel at the Apache Software Foundation, we provided a number of default integration components for Camel. But, as soon as they got out there and the community started to use them and saw the benefits of using them, we saw no end of contributions. People contributed adapters to weird and wonderful systems, and contributed them right back into the Apache project. [Learn more about the CamelOne conference May 24 in Washington, DC.]

We know from our customers that they’ve got specific needs. They’ve got legacy applications. Because we've gone to the effort of making sure that it's very easy to add a new component into Apache Camel, it's very straightforward for someone to add in extra functionality.

For example, if you want to write a component for legacy mainframe application, you could very easily do in a matter of hours. The old approach would take you weeks, months, maybe even years, especially if you don’t have access to the source code. So, you’ve got that added flexibility.

The fact that it's an open-source project at Apache means you can get feedback instantly, if you’ve got issues and problems. Of course, if you want professional help, there’s FuseSource as well. We have our own community at fusesource.com. So, all these things combined means that you have more flexibility and a much more agile way of doing integration.

Gardner: What's happening now in the community? I understand you have a conference that’s coming up May 24, a first of its kind. Why is this a good time to be pulling together the Camel Community?

The nice thing about Camel is that it provides a basic foundation and a terminology of well-defined patterns.

Moynihan: We’re really excited. We have an event coming up in May called CamelOne, and the reason why we focused on Camel with the name of the event is that it’s actually for open-source integration and messaging overall. It’s because Camel is a really great way for people to get started, and it brings together the entire community.

Camel is a great foundation and CamelOne is an event to bring together users of Camel and other open-source integration and messaging technologies to learn more about Camel, open-source messaging like ActiveMQ, and ESBs like Apache ServiceMix.

Camel provides a basic foundation and a terminology of well-defined patterns. The integration patterns themselves are very well-defined, but what's happening is all the different ways in which you connect and what you are connecting to have been changing and evolving over time.

Other people are going to be doing more in-depth management of many integration patterns and they may need to know all the nuances of an ESB platform. The focus of CamelOne is to bring people together to understand, learn about, and meet each other and to grow this community of open-source integration users.

Gardner: So, this is CamelOne, May 24, in the Washington D.C. area. Why Washington D.C.? Is there a lot of this going on in the public sector?

Central location

Moynihan: Actually, we do have a lot of users in the Washington D.C. area. We also thought that was a central location, where people could come from not only anywhere in the US but also from other regions of the world as well. There are a lot of direct flights to that location. But, we do have a lot of users in the area. For example, the Federal Aviation Administration (FAA) is going to be speaking and they have selected open-source integration for the next generation of their services infrastructure.

Since they connect with a lot of other agencies, there is a lot of interest in learning more specifically about that program and about the technologies that it's built upon, because a lot of other agencies need to connect.

Gardner: And how about more information on CamelOne? It’s simple, I suppose search on CamelOne will get you there.

Moynihan: Yes, camelone.com is the website as well.

Gardner: Now, you guys have been involved with a series of books and you have something new coming out in that series. Tell me about that.

Camel in Action

Moynihan: There are a couple of books that recently have come out. One is Camel in Action, which is fantastic for people who want to get going with Camel and learn how to use and deploy it. Rob is coauthor of the ActiveMQ in Action book, which has come out in print recently from Manning Publications.

Davies: ActiveMQ in Action is really a scripted book, which goes through all the different use cases of using ActiveMQ, right from getting started and what messaging is about. It walks you through different deployment options, all the way up through using clusters of ActiveMQ brokers, to using ActiveMQ as a wide area network, so you can connect geographically dispersed locations.

It shows you how to tune the performance of ActiveMQ and get the best out of it. So it's very comprehensive book about how to use ActiveMQ. It's somewhat complementary to Camel in Action as well, which goes through all the different patterns you can use.

It doesn't talk about using Camel. It talks about integration patterns as well and then describes how you can use those using Apache Camel, and you can use Apache Camel with ActiveMQ. ActiveMQ also can embed Apache Camel. So, you have routes running inside the broker from Camel. The two of them are very complementary.

On our website fusesource.com, we also have a lot of webinars, which are happening live on a regular basis. We have a lot of archived webinars, which actually walk you through technical tutorials on how to get started with these various open-source projects.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or download a copy. Register for CamelOne. Sponsor: FuseSource.

You may also be interested in:

• FuseSource Gains New Autonomy to Focus on OSS Infrastructure Model, Apache Community Innovation, Cloud Opportunities
• Apache Camel Addresses Need for Discrete Infrastructure for Services Mediation and Routing
• Apache CXF: Where it's Been and What the Future Holds for Web Services Frameworks
• Workday Integration Cloud debut raises bar on integration as service, deeply ingrains integration as apps function

Case Study: How Fairchild Semiconductor leverages the Workday Integration Cloud

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or download a copy. Learn more. Sponsor: Workday.

The latest BriefingsDirect podcast provides a case study on how new forms of cloud-based integration are helping a major high-tech company build new relationships among and between extended enterprise business processes.

We'll examine how Fairchild Semiconductor has been an early adopter of integration platform as a service (iPaaS). The venerable Silicon Valley company has been using graphical tools to build integrations among and between far-flung applications and services but with those integration platforms housed in a newly unveiled Workday Integration Cloud. [Disclosure: Workday is a sponsor of BriefingsDirect podcasts.]

We’ll learn here from the chief technology officer at Workday on what the integration cloud approach can do and how it points to a future in which broad integration capabilities are increasingly built into software-as-a-service (SaaS) applications.

This cloud-based integration model will prove far less vulnerable to the complexity, fragility and cost that plagues traditional on-premises middleware integration methods. It should also spur the evolution of services ecosystems among multiple business service providers and application providers.

Joining the conversation to dig into what makes integration as a service (IaaS) tick and what it means for the future is Paul Lones, Senior Vice President for Information Technology at Fairchild Semiconductor, and Stan Swete, the Chief Technology Officer at Workday. The discussion is moderated by BriefingsDirect's Dana Gardner, Principal Analyst at Interarbor Solutions.

Here are some excerpts:

Gardner: What's the problem now with integration? Why is this different than a few years ago? Why is it that we need to adopt a different take on integration?

Lones: We've just recently gone live with Workday, and several of their partners, and have completely transformed our human capital management landscape.

Fairchild Semiconductor has roughly 10,000 employees worldwide. We're a semiconductor manufacturing company. We have manufacturing facilities in the United States and throughout Asia. Our customer base is global, our employee base is global. Over 70 percent of our business is in Asia and 70 percent of our employees are in Asia. Having the capability to provide a core HR platform like this to that broad a set of colleagues around the world is really exciting for us, and to be able to support our internal customers and the HR group.

... Integrations are a new challenge, a broader challenge than they have been traditionally. ... In the HR arena, there has been no such thing as a standard integration. Every benefit provider, every payroll service provider that you want to work with requires a custom integration. That’s always been true, and having the set of tools that we now have at our disposal makes that a lot easier.

Companies like Fairchild are really trying to take advantage of some of the new capabilities that SaaS providers are offering. ... It's a critical enabler.

We look for two things. One, we want to find a supplier that thinks of this in a more holistic ecosystem-like way, and that has a series of application-level partners, that we can add to our overall architecture and overall application capability.

In addition to that, we look for good integration tools, because even beyond those partnerships, we still have to do a lot of integration work.

... For the Workday partners, those integrations are handled between Workday and their partners, which reduced our integration burden. We don't have to maintain those, as both of those applications continue to improve. In addition to that, we've built 28 application integrations ourselves, largely to benefit service providers and payroll service providers around the world.

We were fortunate enough that we were able to get some early access to the toolset that Workday is now making available to their broader customer and partner base.

I had a small team of IT staff that was completely unfamiliar with Workday when they first started, and we put them to work on these integrations. We were able to complete these 28 integrations in less than 120 days, which I think was pretty good performance. ... We do know that from an overall project implementation perspective that an on-premises application typically will take 2-3 times as long to execute, and I'd expect that the integration piece would have a similar scaling.

Gardner: Stan, what needed to change and when Workday looked at this issue of your online ecosystem and how it tied things together?

Swete: We still look at it as having the same requirements for enterprise integration. Especially for hub systems like human capital management, there are ton of other systems that you have to integrate with. So the requirements are daunting and are still there. It's been the same for a while at enterprise software.

What we see as being a cloud vendor, a SaaS vendor, is just new opportunities to leverage the SaaS model to do integration a little bit differently, have the application vendor take on more of the ownership of the integration issue, and use the fact that we've got all of our customers running on a single version of the product to tie some integration logic to that and bring more control and stability to that integration for our customers and our partners.

Gardner: Why is it then that the traditional systems, platforms, and middleware that are in place are not up to this task?

Swete: There's just a split today between the technologies and the platforms that are used to execute integration and convey data and then the application’s endpoints that are involved with and tied up in the logic of that integration.

It's not that no one is up to it, but it's just that that gap splits responsibilities where maybe they don’t have to be split. What we’re trying to do is marry it, use what we know about our applications to create integration logic, and then embed technology that hasn’t been embedded with applications before to help with the delivery of that.

That hasn't replaced every single kind of middleware technology that you need. You still need a middleware technology behind your firewall. You still need specialized middleware technology in the cloud to do things that it does best. But, for the application-centric part of integration, application vendors can do more.

... The Workday Integration Cloud is an extension of Workday's cloud that we use to host and process our on-demand applications and it has several really important components. One is a platform component. The tools that Paul mentioned that they used to build integrations, up until today, have been there for Workday developers. The announcement makes these tools fully available to Workday customers and to Workday partners.

In addition to the tools, there is a rich enterprise service bus (ESB) execution environment that runs the results of these developmental tools. We offer not only the tools to build integration systems but the execution environment for the integration systems. And then we've a set of scheduling and monitoring tools that our customers can use to directly schedule and monitor the execution of their integrations.

So those three things taken together form the platform, that's part of the integration cloud. The resulting integration systems we also consider a part of the cloud. Workday for some time has been building what we call Packaged Integrations and Connectors. We have a library of those that we can make available to our customers.

Fairchild has used some of these. These integrations are built with our tooling by us and for our customers. Packaged integrations really just look like another Workday product, but they handle both ends of the integration challenge.

We also have connectors that handle our end of it but build logic out. The main example is a payroll interface product that lets our customers, gives our customers a starting point for hooking up Workday human capital management to the variety of international payrolls many of our larger customers have.

This is very solid ESB technology, well thought of by the engineering talent that we now own.

Packaged integrations from Workday is another component of the Integration Cloud and the final one is just the body of integrations that our customers and partners create.

These are the intellectual property of our customers and our partners. Workday does facilitate sharing of those definitions if the customer and partners are interested, but there is that growing body of application as well. Those things taken together are the Workday Integration Cloud.

Gardner: And just to be clear, this is designed for your customers. This isn't just a general purpose integration service that you are opening up writ large. This is about your ecosystem and your customers, is that right?

Swete: The beauty of it is that it's based on middleware from a company formerly called Cape Clear that Workday acquired three years ago. I think that's very important to mention that. So it's not like we, an apps vendor, just did our take on an ESB. This is very solid ESB technology, well thought of by the engineering talent that we now own.

Built-in integration

We're taking this technology and integrating it into our applications, building integration into our applications as the way we refer to it, and then making the combined product available to both our customers and our partners. The partners are the equally important point. Systems integration partners from Workday can get access to these tools and this platform.

Gardner: And how about the pricing?

Swete: The Workday Integration Cloud platform is being made available at no additional cost to Workday customers and Workday partners. We make our money selling our application services.

Gardner: I'm intrigued by this notion of making integration part of the application. I think the history of this, Paul, has been that over the years, new applications and platforms, and even models of computing would come along. You would get great productivity from the application, you would buy and install and master the platform, and then you would be faced with an integration problem.

This is happened over and over again. We've seen it with mainframes to client-server and then into multi-tier and distributing computing and then ultimately with web and now cloud computing.

Companies like ours and many companies working on this are moving from a monolithic internal application orientation to one that's more of a hybrid model.

Given that integration has been a bolt-on, something that's been delivered after they shift in an application model, why now change? Why is integration and the application coming together now?

Lones: Part of it is that our approach to overall enterprise architecture is changing. Companies like ours and many companies working on this are moving from a monolithic internal application orientation to one that's more of a hybrid model, where we want to really take advantage of the new capabilities and the quicker pace of development and deployment of improvements that SaaS providers offer.

Therefore, integrations naturally become a critical part of that, because the number of applications that we use in our business increases somewhat with this sort of approach.

Swete: The challenge here is that the requirements in the large problem of integration haven't changed, and there have been a lot of tools developed to address the issue. Some results have been achieved, but I don't think anyone is satisfied with how maintainable enterprise integration is. And, we happened to think the answer is to build more robust integration where the integration definitions themselves are more informed by what exists and what's changing in the application.

Hub system

That's the opportunity that we were seeing. We came on to it by just being the provider of an application that is going to be the hub system and be hooked up to a lot of different systems.

We knew that integration was going to be front and center for us as a brand new SaaS vendor six years ago. One of the differences we wanted to make was to do more about the problem. So, we started with an investment of technology.

Where that has led us is really tying what can get done with integration technology to what applications know about, everything from their security model to, in our case, we leverage a lot the fact that we know about people and how they are organized. So, we're able to have integration definitions that can get routed around for the appropriate approvals before certain steps happen.

That’s unique, but it's breaking down the separation between integration that would be built by one side of the company and tying it back to who it's really serving, the other side of the company.

For payroll integration, the payroll admin can be hooked into the fact that a major feed of HR data is going out to a payroll system and they can get a check on that before it happens. That’s something we’ve built in and we’ll continue to look for those opportunities. I still think it's actually early days for what our integration tools can leverage inside the application.

You still have to have experts on integration middleware and we have that, but the real benefit we think comes from blurring the distinction and marrying these things together.

Gardner: So, the system of record for HR and the governance and policies about employees and their roles in the organization can now be applied pretty seamlessly to who gets to do integrations and/or how integrations as part of a business process would work. Am I reading that right?

Swete: Yeah, how they get executed, how they get approved is all built in to the same sort of system that you use to schedule a report or any other thing you’d do in your application. For us, it's just an extension of the application, rather than a hard line and then some integration technology that no one on the app side understands.

There still are differences. You still have to have experts on integration middleware and we have that, but the real benefit we think comes from blurring the distinction and marrying these things together.

... We’ve taken the approach of splitting the development tools into a framework that is more geared for developing simple integrations, as we call them. This is one-way data in or one-way data out of Workday to third-party systems, and we have a tool called the Enterprise Interface Builder (EIB) that is a non-programmer could use. You still need to know that you are sending something to a secure FTP location, but you don’t have to be a developer.

Sets of choices

We give you a graphical user interface, we give you a selected set of choices for how you can source data, a selected set of choices for how you can transform it, and a select set of choices for how you can deliver it. You can save that, and then you have a definition that you can then schedule on a recurring basis. That’s built for non-developers.

The other tool that we have has a completely different personality. It's what we call Workday Studio. This is the developer tool that we have used to build our integrations, and it is now available for our customers. But, on this one, you want to be a developer. You're not doing programming, but you are working in an Eclipse-based framework with detailed control over integration components and orchestration of how data flows. So, this is a technical development tool.

The thing it creates is the same thing that the EIB creates, an integration system that can then be executed in Workday, but the creation of it is much more technical.

Gardner: So it's interesting, Stan. You have a user like Fairchild, using these tools, building these integrations, moving more toward a multiparty ecosystem process-oriented benefit, but the responsibility on those integrations is with you.

It seems as if you're really giving an awful lot here. How can you do that with a strong sense of confidence? Isn't there a risk that if these integrations start breaking that you are in the catbird seat?

Levels of the game

Swete: Yeah, well, there are levels of the game for how you can leverage the support you get out of the core application that we keep moving forward. One level of the game is for us that's very important in the integrations we build and sell are ones that can just share the application definitions. So, we support those across all the updates and verify that the logic of those is going to work.

For the integrations tools, we can put smarts into the tools that share how the applications are constructed in that. It gives our customers a leg-up that they can start with these components. Then they can create integrations that are a little bit more impervious to being broken by changes in the applications, because they're sharing metadata back into the applications.

Lots of integrations are built on our application programming interface (API) and so we've got to be rigorous about versioning the API and having a contract to support back versions that gives us a certain amount of insurance. It's not like that with some of these opened in the tools that there couldn't be logic and coding errors that are put in and those are the ones that we would have to encounter together with our customers and we're not going to debug every single one of those.

So, for different levels of the game, more packaged, complete support, on up to the more open-ended integrations, you do what you can to try to make it so the integrations are a little bit more robust than what would have been built with a separate tool set.

... We also encourage people to want to share these integrations. We didn’t need to do more to automatically support that because our partners are going to be generating these things, as our customers, and in the SaaS community, there is just this great notion about sharing the things you do. So, we see supporting that and we can ultimately see that even leading to selling some of the things you do. All of those are potential features for this space.

Gardner: Paul, it sounds less like a buyer-seller relationship than a partnership. Do you view it that way?

Our experience to date is that companies like ours have more of a voice in the feature improvements of the application.

Lones: We do. Our experience to date, working with providers like Workday and some of the other SaaS providers that we are fortunate enough to do business with, is that companies like ours have more of a voice in the feature improvements of the application.

There tends to be, and certainly it's the case with Workday, a much more active community of clients, users, that are sharing information about everything from somewhat technical to very business process-oriented experiences that all of us have had. That's a very different experience.

In some ways, it's sort of ironic to me that we view it quite a bit more as a partnership. A lot of people perhaps think that it's a SaaS application and, if things don't work out, then when your contract is up, you just go find another SaaS provider.

It is true that there might be a little bit more flexibility, but what we’re finding so far in our experience, and it is early, is that the receptivity and the sense of making improvements together, I think it will actually stick longer than maybe some of the traditional software applications.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or download a copy. Learn more. Sponsor: Workday.

You may also be interested in:

• Delivering Data Analytics Through Workday SaaS ERP Applications Empowers Business Managers at Actual Decision Points
• Workday Integration Cloud debut raises bar on integration as service, deeply ingrains integration as apps function
  
• Executive Interview: Workday's Aneel Bhusri on Advancement of SaaS and Cloud Models for Improved ERP
• Architecture is Destiny: Why the Revolution in Business Interactions Can't Work on Conventional Databases
• HCM SaaS Provider Workday's Advanced Architecture Brings New Business Agility Benefits to Enterprises

Learning the right lessons from the Amazon cloud outage


This guest BriefingsDirect post comes courtesy of Jason Bloomberg, managing partner at ZapThink.

By Jason Bloomberg

Have you noticed that ZapThink’s crystal ball has been working overtime? We sounded the warnings about cyberwarfare mere days before the Stuxnet worm hit. Then we predicted the fall of enterprise architecture frameworks right before the Zachman organization imploded. Next, we heralded a secondary market for IP addresses as the IPv4 space ran out of them. Sure enough, that secondary market is now here. And last week, we warned against putting all your eggs in any one cloud provider’s basket. Sure enough, Amazon’s public cloud went belly up immediately afterward. All I can say is that if we make a prediction that will impact your business, you’d better take heed!

In all seriousness, there’s no supernatural clairvoyance at work here. What you’re seeing is the power of the ZapThink 2020 vision for Enterprise IT, which delineates the interrelationships among the numerous trends in the IT marketplace. Just as the best psychics are in reality masters at picking up subtle clues in human behavior, we’re tuning into the complex subtleties that the multiple forces of change in our midst present to us.

One of the primary insights of the ZapThink 2020 vision is that individual trends, let alone single events, should never be taken in isolation. This insight is particularly useful when a crisis like the Amazon crash presents itself.

At this point in time, we’re experiencing a backlash from this crash. People are reconsidering the wisdom of moving to the cloud, and in particular, public clouds. Perhaps the large infrastructure vendors who were warning their customers about the security and reliability issues with public clouds in order to sell more gear to build private clouds were right after all?

Not so fast. If we place the Amazon crash into its proper context, we are in a better position to learn the right lessons from this crisis, rather than reacting out of fear to an event taken out of that context. Here, then, are some essential lessons we should take away from the crash:

• There is no such thing as 100 percent reliability. In fact, there’s nothing 100 percent about any of IT—no code is 100 percent bug free, no system is 100 percent crashproof, and no security is 100 percent impenetrable. Just because Amazon came up snake eyes on this throw of the dice doesn’t mean that public clouds are any less reliable than they were before the crisis. Whether investing in the stock market or building a high availability IT infrastructure, the best way to lower risk is to diversify. You got eggs? The more baskets the better.
• This particular crisis is unlikely to happen ever again. We can safely assume that Amazon has some wicked smart cloud experts, and that they had already built a cloud architecture that could withstand most challenges. Suffice it to say, therefore, that the latest crisis had an unusual and complex set of causes. It also goes without saying that those experts are working feverishly to root out those causes, so that this particular set of circumstances won’t happen again.


  Just because Amazon came up snake eyes on this throw of the dice doesn’t mean that public clouds are any less reliable than they were before the crisis.

• The unknown unknowns are by definition inherently unpredictable. Even though the particular sequence of events that led to the current crisis is unlikely to happen again, the chance that other entirely unpredictable issues will arise in the future is relatively likely. But such issues might very well apply to private, hybrid, or community clouds just as much as they might impact the public cloud again. In other words, bailing on public clouds to take refuge in the supposedly safer private cloud arena is an exercise in futility.

• The most important lesson for Amazon to learn is more about visibility than reliability. The weakest part of Amazon’s cloud offerings is the lack of visibility they provide their customers. This “never mind the man behind the curtain” attitude is part of how Amazon supports the cloud abstraction I discussed in the previous ZapFlash. But now it’s working against them and their customers. For Amazon to build on its success, it must open the kimono a bit and provide its customers a level of management visibility into its internal infrastructure that it’s been uncomfortable delivering to this point.


The ZapThink take

Abstractions hide complexity from consumers of technology, but if you do too good a job hiding the underlying complexity, then the abstraction can backfire. But that doesn’t mean that abstractions are bad; rather, you need different abstractions for different audiences.

The latest crisis impacted a wide swath of small cloud-based vendors, from Foursquare to DigitalChalk to EDU 2.0. These firms’ customers simply wanted their tools to work, and were disappointed and inconvenienced when they stopped working. But the end-user customer may not have even been aware that Amazon’s cloud was behind their tool of choice. Clearly, those customers wouldn’t find better visibility into the cloud particularly useful.

No, it’s the technology departments at the small vendors that require better visibility. They are the people who require management tools that enable them to gain a greater level of control over the cloud environments they leverage in their own products. Once Amazon supports such management tools, then Amazon’s customers will be better able to provide the seamless abstraction to the cloud end user, who simply wants stuff to work properly. And there’s nothing supernatural about that!


This guest BriefingsDirect post comes courtesy of Jason Bloomberg, managing partner at ZapThink.

---

SPECIAL PARTNER OFFER

SOA and EA Training, Certification,
and Networking Events

In need of vendor-neutral, architect-level SOA and EA training? ZapThink's Licensed ZapThink Architect (LZA) SOA Boot Camps provide four days of intense, hands-on architect-level SOA training and certification.

Advanced SOA architects might want to enroll in ZapThink's SOA Governance and Security training and certification courses. Or, are you just looking to network with your peers, interact with experts and pundits, and schmooze on SOA after hours? Join us at an upcoming ZapForum event. Find out more and register for these events at http://www.zapthink.com/eventreg.html.

---

You may also be interested in:

• Amazon apologizes for last week's cloud crash
• Cloud brokering: Building a cloud of clouds
• Open Group cloud panel forecasts cloud as spurring useful transition for enterprise architecture
• Cloud computing drives need for open standards to define and describe a new enterprise environment