With Systinet 3.0, HP broadens SOA governance role to encompass services lifecycle, business processes, IT service management

Hewlett-Packard trotted out the new HP SOA Systinet 3.0 registry today, capping a year of announcements that create a services oriented architecture (SOA) lifecycle portfolio, and extends the governance function broadly -- a cradle to grave approach -- that spans from design time to run time and all the way up to project portfolio management (PPM).

The newest market leading Systinet UDDI registry forms the cockpit for managing not only services, but with the newly added Business Process Execution Language (BPEL) support, takes the helm for business processes, too. HP plans to further push the envelope on a master management value even further into IT operations and IT Service Management, as well as a PPM role with the registry.

HP SOA Systinet 3.0 is designed with broadening the use of SOA governance, and IT governance, in mind. HP Software sees SOA moving toward more enterprise-wide deployment. To get there, the role of the registry itself needs to expand. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

Brad Shimmin at Current Analysis has a comprehensive write-up on the earlier announcements and HP's direction for the product.

The new SOA infrastructure component captures more than UDDI information, it encompasses best practices, CMDB information, and sets the stage for a wider "culture of governance" to emerge in enterprises, said Kelly Emo, SOA product marketing manager at HP Software.

If you've been following HP's acquisitions and development efforts of the past five years, you'll see a distinct pattern of putting the pieces together for a total or master management capability. The goal is not simply putting all the management data in a common repository, but of elevating the visibility into management across more aspects of IT and business processes.

That visibility and the access to the right systems in the right business context then provides the means to further automate IT and SOA activities, to capture best practices and instantiate them back into how IP performs with repeatability and scale.

This latest product release caps a series of significant acquisitions by HP, from Systinet to Opsware. The cradle to grave story of comprehensive IT management and automation is not yet complete, but the strategy is clear. And the pivotal role of the registry is also clear.

The movement is to expand SOA governance, but perhaps more importantly, expand governance in general across more of what IT touches. Rules, roles, business context, policy, development-to-deployment lifecycles, operational efficiency, projects and services -- all need to be brought into a contextual whole. Not by a common product set, but via standards, technology provider inclusion, and with a methodological and cultural commonality emphasis. There really isn't another place to try and find this common framework for stitching together the disparate aspects of IT management -- the SOA registry is as good as we have nowadays.

Of course, the trends in the market make a move toward comprehensive IT service management via automation -- not reactive and disjointed manual stop-gaps -- imperative. As enterprises take up virtualization, cloud computing, SOA, master data management, and such IT shared services approaches typified by ITIL 3.0, then the scale, complexity and range of inter-dependent IT assets needs a better master.

HP is placing a large bet on the HP SOA Systinet 3.0 registry will fill the roles of eyes, ears, and execution coordinator for more of what makes IIT tick.

More information on the release.

WSO2 eases enterprise data availability for SOA access, consumption

A new data services offering from WSO2 allows a database administrator (DBA) or anyone with a knowledge of SQL to access enterprise data and expose it to services and operations through a Web services application-programming interface (API).

WSO2 Data Services, the latest open-source product from the Mountain View, Calif. company, helps DBAs and programmers contribute to a company’s service-oriented architecture (SOA) by creating WS-* style Web services and REST-style Web resources based on enterprise data. [Disclosure: WSO2 is a sponsor of BriefingsDirect podcasts.]

Users can enter queries and map them into services and operations. Once the query or stored procedure has been exposed as a service, it can be accessed across the network as a service or Web resource.

In its initial release, the application supports access to data stored in relational databases such as Oracle, MySQL and IBM DB2, as well as the comma-separated values (CSV) file format, and Excel spreadsheets. It allows users to authenticate, encrypt and sign services using the WS-Security and HTTP security standards. In addition, support for the WS-ReliableMessaging standard provides enterprise-level reliability.

Current Analysis's Brad Shimmin has some good points on the release (log in required).

Features of WSO2 Data Services 1.0 include:

• Data aggregation, which allows administrators to create services that aggregate data from multiple data sources.

• Wizards for easy configuration.

• XML configuration file format.

• A "try-it" tool that lets users test the data services they have created within the Data Services console.

• Dual REST and WS-* support. REST resources access data using a unique URL for each record; WS-* services use typical Web service access to expose data.

• Built-in caching to eliminate the system overhead of repeatedly returning the same XML response to clients.

The new data services product will be available for download beginning today from the WSO2 Web site. As an open-source product there are no licensing or subscription fees, although service and support options are available.

BriefingsDirect Insights analysts examine HP-Oracle Exadata, 'extreme' BI, virtualization and cloud computing news

Listen to the podcast. Download the podcast. Find it on iTunes/iPod. Learn more. Sponsors: Active Endpoints, Hewlett-Packard.

Read a full transcript of the discussion.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at www.activevos.com/insight.

Welcome to the latest BriefingsDirect Insights Edition, Vol. 30, a periodic discussion and dissection of software, services, SOA and compute cloud-related news and events, with a panel of IT analysts.

In this episode, recorded Sept. 26, 2008, our experts examine the HP-Oracle announcements at Oracle OpenWorld, cloud computing and "on-premises" clouds, and recent virtualization news from VMware, HP, Red Hat and Citrix.

Please join noted IT industry analysts and experts Joe McKendrick, an independent analyst and ZDNet blogger; Brad Shimmin, principal analyst of Current Analysis; Jim Kobielus, senior analyst at Forrester Research, and Dave Linthicum, independent SOA consultant. Our discussion is hosted and moderated by your's truly.

Here are some excerpts:

Oracle announced the release, in partnership with HP, of a very high-end data warehousing appliance. They may not use the word "appliance," but that's in fact what it is. It's called the HP Oracle Database Machine. It encompasses and includes the Oracle Exadata Storage Server, which is a grid storage level server.

What Oracle and HP have essentially done is take a page from the Netezza book, because that is, of course, the feature of the Netezza performance system. What they did essentially is they also shot across Teradata's bow, because this is Oracle's petabytes-scale, data warehousing-solution platform.

I was shocked, shocked, absolutely simply shocked. This is because historically Oracle has strayed so far away from the appliance market. I am glad to see this happening. ... Now if only they would release parts of their middleware as an appliance, I would be very happy.

Larry Ellison indicated that they seem to have some plans for that. They really resisted details -- but they seem to have some plans to "appliance-ize," if that's the word, more and more the Oracle Fusion Middleware stack.

It almost seems now that Oracle has anointed HP at some level as a preferred hardware supplier on storage, if not also other aspects of hardware. What does that mean for EMC and some of the other storage hardware providers?

I think that all of those relationships will come under strain from this. There is no question about that. I think there are going be a lot of far-ranging ripples from this relationship that will change the way the market functions.

It certainly moves the business intelligence (BI) arena forward. ... Now there is a trend emerging. I am sure Oracle has an eye on this as well. It's toward open source. We are seeing more open source in data warehouses too. This is open source at the warehouse level itself, at the database level itself. [Sun Microsystem's] MySQL for example has been pointing in this direction, PostgreSQL as well. [And there's Ingres.]

Now with Oracle and HP cooperating, why shouldn't we expect Sun to come out with something quite similar, but with MySQL as the database, and their [Sparc/UltraSparc] processing, and their rack, cooling and InfiniBand, and of course, their storage? If Sun does that then IBM will certainly come out with something around DB2.

There is an emphasis on simplifying data warehousing, making data warehousing simple for the masses. Microsoft, love them or hate them, has been doing a lot of work in this area by increasing the simplicity of its data warehouse and making it available at more of a commodity level for the small to medium size business space.

One of the other important outcomes from my point of view this week at Oracle OpenWorld -- was the fact that Oracle, now in conjunction with Amazon's Elastic Compute Cloud, has an Oracle cloud -- the existing Amazon cloud can take Oracle database licenses. ... Using tools that Oracle is providing they can move their data to back it up or move databases entirely to be persistent in the cloud, in Amazon's S3 service.

I think that this is one step in the direction that, in essence, we're going back in time a bit, moving back into the time-sharing space. A lot of things are going to be pushed back out into the universe through economy's scale, and also through the value of communities. It just can be a much more cheap and cost effective way of doing it. I think it's going to be a huge push in the next two years.

I think that Oracle is going to have a cloud offering, IBM is going to have a cloud offering, Sun is going to have a cloud offering, and it's going to be the big talk in the big industry over the next two or three years. I think they are just going to get out there and fight it out.

I think you are going to have number of start-ups, too. They are going to have huge cloud offerings as well. They are going to compete with the big guys. ... Quite frankly, I think, maybe the more agile, smaller companies may win that war.

These vendors are basically tripping over themselves and rushing out to the market, way before these private clouds have even established themselves. Yet the vendors are declaring that they have the infrastructure and the approach to do it. It sort of reminds me of a platform, or even operating system, land grab -- that getting there first and establishing some of the effective standards and coming up with industry-common implementations gives them an opportunity to at some level or format create the de facto portability means.

As we go forward, I think that's the destination. If you look at how everything is going, I think everything is going to be pushed up into the cloud. People are basically going to have virtual platforms in the cloud, and that's how they are going to drive it. Just from a cost standpoint, everything we just discussed, the advantages are going to be for those who get there first.

I think that very much like the early adopters of the Web, back in the 1990s, this is going to be the same kind of a land grab, and the same kind of land rush that's going to occur. Ultimately you are going to find 60 percent to 80 percent of the business processes over the next 10 years are going to be outsourced.

Listen to the podcast. Download the podcast. Find it on iTunes/iPod. Learn more. Sponsors: Active Endpoints, Hewlett-Packard.

Read a full transcript of the discussion.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at www.activevos.com/insight.

Improved insights and analysis from IT systems logs helps reduce complexity risks from virtualization

Listen to the podcast. Download the podcast. Find it on iTunes/iPod. Learn more. Sponsor: LogLogic.

Read complete transcript of the discussion.

Virtualization has certainly taken off, but less attention gets directed to how to better manage virtualization, to gain better security using virtualization techniques, and also to find methods for compliance and regulation of virtualized environments -- but without the pitfalls of complexity and confusion.

We seem to be at a tipping point in terms of everyone doing virtualization, or wanting to do it, or wanting to do even more. IT managers experimenting with virtualization are seeking to reduce costs, to improve the efficiency in use of their assets, or for using virtualization to address the issues they might have with energy cost, energy capacity or sometimes even space capacity in the data center. But the paybacks from virtualization can be lost or mitigated when management and complexity are not matched. Poorly run or mismanaged virtualized environments are a huge missed opportunity.

Now's the time when virtualization best practices are being formed. The ways to control and fully exploit virtualization are in demand, along with the tools to gain analysis and insights into how systems are performing in a dynamic, virtualized state.

To help learn about new ways that systems log tools and analysis are aiding the ramp-up to virtualization use, I recently spoke with Charu Chaubal, senior architect for technical marketing, at VMware; Chris Hoff, chief security architect at Unisys, and Dr. Anton Chuvakin, chief logging evangelist and a security expert at LogLogic.

Here are some excerpts:

The reasons people are virtualizing are cost, cost savings and then cost avoidance, which is usually seconded by agility and flexibility. It’s also about being able to, as an IT organization, service your constituent customers in a manner that is more in line with the way business functions, which is, in many cases, quite a fast pace -- with the need to be flexible.

Adding virtualization to the technology that people use in such a massive way as it's occurring now brings up the challenges of how do we know what happens in those environments. Is there anybody trying to abuse them, just use them, or use them inappropriately? Is there a lack of auditability and control in those environments? Logs are definitely one of the ways, or I would say a primary way, of gaining that visibility for most IT compliance, and virtualization is no exception.

As a result, as people deploy VMware and applications in a couple of virtual platforms, the challenge is knowing what actually happens on those platforms, what happens in those virtual machines (VMs), and what happens with the applications. Logging and LogLogic play a very critical role in not only collecting those bits and pieces, but also creating a big picture or a view of that activity across other organizations.

Virtualization definitely solves some of the problems, but at the same time, it brings in and brings out new things, which people really aren't used to dealing with. For example, it used to be that if you monitor a server, you know where the server is, you then know how to monitor it, you know what applications run there.

In virtual environments, that certainly is true, but at the same time it adds another layer of this server going somewhere else, and you monitor where it was moved, where it is now, and basically perform monitoring as servers come up and down, disappear, get moved, and that type of stuff.

The benefits of virtualization today ... is even more exciting and interesting. That's going to fundamentally continue to cause us to change what we do and how we do it, as we move forward. Visibility is very important, but understanding the organizational and operational impacts that real-time infrastructure and virtualization bring, is really going to be an interesting challenge for folks to get their hands around.

When you migrate from a physical to a virtual infrastructure, you certainly still have servers and applications running in those servers and you have people managing those servers. That leaves you with the need to monitor the same audit and the same security technologies that you use. You shouldn't stop. You shouldn't throw away your firewalls. You shouldn't throw away your log analysis tool, because you still have servers and applications.

They might be easier to monitor in virtual environments. It might sometimes be harder, but you shouldn't change things that are working for you in the physical environment, because virtualization does change a few things. At the same time, the fact that you have applications, servers, and they serve you for business purposes, shouldn't stop you from doing useful things you're doing now.

Now, an additional layer on top of what you already have adds the new things that come with virtualization. The fact that this server might be there one day, but be gone tomorrow -- or not be not there one day and be built up and used for a while and then removed -- definitely brings the new challenges to security monitoring, security auditing in figuring out who did what where.

The customers understood that they have to collect the logs from the virtual platforms, and that LogLogic has an ability to collect any type of a log. They first started from a log collection effort, so that they could always go back and say, "We've got this data somewhere, and you can go and investigate it."

We also built up a package of contents to analyze the logs as they were starting their collection efforts to have logs ready for users. At LogLogic, we built and set up reports and searches to help them go through the data. So, it was really going in parallel with that, building up some analytic content to make sense of the data, if a customer already has a collection effort, which included logs from the virtual platform.

All the benefits that we get out of virtualization today are just the beginning and kind of the springboard for what we are going to see in terms of automation, which is great. But we are right at the same problem set, as we kind of pogo along this continuum, which is trying really hard to unite this notion of governance and making sure that just because you can, doesn't mean you should. In certain instances the business processes and policies might prescribe that you don't do some things that would otherwise be harmful in your perspective.

It's that delicate balance of security versus operational agility that we need to get much better at, and much more intelligent about, as we use our virtualization as an enabler. That's going to bring some really interesting and challenging things to the forefront in the way in which IT operates -- benefits and then differences.

Read complete transcript of the discussion.

Listen to the podcast. Download the podcast. Find it on iTunes/iPod. Learn more. Sponsor: LogLogic.

Oracle and HP explain history, role and future for new Exadata Server and Database Machine

Listen to the podcast. Download the podcast. Find it on iTunes/iPod. Learn more. Sponsor: Hewlett-Packard.

Read complete transcript of the discussion.

The sidewalks were still jammed around San Francisco's Moscone Center and the wonderment of an Oracle hardware announcement was still palpable across the IT infrastructure universe late last week. I sat down with two executives, from Hewlett-Packard and Oracle, to get the early deep-dive briefing on the duo's Exadata appliance shocker.

Oracle Chairman and CEO Larry Ellison caught the Oracle OpenWorld conference audience by surprise the day before by rolling out the Exadata line of two hardware-software configurations. The integrated servers re-architect the relationship between Oracle's 11g database and high-performance storage. Exadata, in essence, gives new meaning to "attached" storage for Oracle databases. It mimics the close pairing of data and logic execution that such cloud providers as Google use with MapReduce technologies. Ellison referred to the storage servers as "programmable."

Exadata also re-architects the HP-Oracle relationship, making HP an Oracle storage partner extraordinaire -- thereby upsetting the status quo of the world's of IT storage, databases and data warehouses markets.

Furthermore, Exadata leverages parallelism and high-performance industry standard hardware to bring "extreme business intelligence" to more enterprises, all in a neat standalone package that's forklift-ready. Beyond 10 terabytes and into the petabyte range was how HP and Oracle designers describe the scale and 10x to 72x typical performance gains from the high-end Exadata "Machine."

The unveiling clearly deserves more detail, more understanding. Listen then as I interview Rich Palmer, director of technology and strategy for the industry standard servers group at HP, along with Willie Hardie, vice president Oracle database product marketing, on the inside story on Exadata.

The interview comes as part of a series of sponsored discussions with IT executives I've done from the Oracle OpenWorld conference. See the full list of podcasts and interviews.

Read complete transcript of the discussion.

Listen to the podcast. Download the podcast. Find it on iTunes/iPod. Learn more. Sponsor: Hewlett-Packard.

Greenplum pushes envelope with MapReduce and parallelism enhancements to its extreme-scale data offering

Greenplum has delivered on its promise to wrap MapReduce into the newest version of its data solutions. The announcement from the data warehousing and analytics supplier comes to a fast-changing landscape, given last week's HP-Oracle Exadata announcements.

It seems that data infrastructure vendors are rushing to the realization that older database architectures have hit a wall in terms of scale and performance. The general solution favors exploiting parallelism to the hilt and aligning database and logic functions in close proximity, while also exploiting MapReduce approaches to provide super-scale data delivery and analytics performance.

Greenplum's Database 3.2 takes on all three, but makes signigficant headway in embedding the MapReduce parallel-processing data-analysis technique pioneered by Google. The capability is accompanied by new tooling to extend the reach of using the technology. The result is Web-scale analytics and performance for enterprises and carriers -- or cloud compute data models for the masses. [Disclosure: Greenplum is a sponsor of BriefingsDirect podcasts.]

The newest offering from the San Mateo, Calif.-based Greenplum provides users new capabilities for analytics, as well as in-database compression, and programmable parallel analytic tools.

With the new functionality, users can combine SQL queries and MapReduce programs into unified tasks executed in parallel across thousands of cores. The in-database compression, Greenplum says, can increase performance and reduce storage requirements dramatically.

The programmable analytics allow mathematicians and statisticians to use the statistical language R or build custom functions using linear algebra and machine learning primitives and run them in parallel directly against the database.

Greenplum's massively parallel, shared-nothing architecture fully utilizes each core, with linear scalability to thousands of processors. This means that Greenplum's open source-powered database software can scale to support the demands of petabyte data warehousing. The company's standards-based approach enables customers to build high-performance data warehousing systems on low-cost commodity hardware.

Database 3.2 offers a new GUI and infrastructure for monitoring database performance and usage. These seamlessly gather, store, and present comprehensive details about database usage and current and historical queries internals, down to the iterator level, making this ideal for profiling queries and managing system utilization.

Now that HP and Oracle have taken the plunge and integrated hardware and software, we can expect that other hardware makers will be seeking software partners. Obviously IBM has DB2, Sun Microsystems has MySQL, but Dell, Hitachi, EDS and a slew of other hardware and storage providers may need to respond to the HP-Oracle challenge.

On Greenplum's blog, Ben Werther, director, Professional Services & Product Management at Greenplum, says: "Oracle has been getting beat badly in the high-end warehousing space ... Once you cut through the marketing, this is really about swapping out EMC storage for HP commodity gear, taking money from EMC's pocket and putting it in Oracle's."

It will also be interesting to watch as bedfellows and evaluated from Microsoft/DatAllegro, what happens with Ingres, whether Sun with MySQL can enter this higher end data performance echelon. This could mean that players like Greenplum and Aster Data Systems get some calling cards from a variety of suitors. The Sun-Greenplum match-up makes sense at a variety of levels.

Stay tuned. This market is clearly heating up.

Interview: From OpenWorld, HP's John Santaferraro on latest BI Modernization strategies

Listen to the podcast. Download the podcast. Find it on iTunes/iPod. Learn more. Sponsor: Hewlett-Packard.

Read a full transcript of the discussion.

Leading up to HP and Oracle's blockbuster announcement Sept. 24 of record-breaking data warehouse appliance performance, the business value of these infrastructure breakthroughs was the topic of a BriefingsDirect interview with John Santaferraro, director of marketing for HP's Business Intelligence Portfolio.

Now that the optimized hardware and software are available to produce the means to analyze and query huge data sets in near real-time, the focus moves to how to best leverage these capabilities. Soon, business executives will have among the most powerful IT tools ever developed at their disposal to deeply and widely analyze vast seas of data and content in near real time to help them run their business better, and to steer clear of risks.

Think of it as business intelligence (BI) on steroids.

At the Oracle OpenWorld unveiling, HP Chairman and CEO Mark Hurd called the new HP Oracle Database Machine a “data warehouse appliance.” It leverages the architecture improvements in the Exadata Programmable Storage Server, but at the much larger scale and with other optimization benefits.

The reason for the 10x to 72x performance improvements cited by Oracle Chairman and CEO Larry Ellison have do to bringing the “intelligence” closer to the data, that is bringing the Exadata Programmable Storage Server appliance into close proximity to the Oracle database servers, and then connecting them through InfiniBand connections. In essence, this architecture mimics some of the performance value created by cloud computing environments like Google, with its MapReduce technology.

To better understand how such technologies fit into the Oracle-HP alliance, with an emphasis on professional services and methodologies, I asked HP's Santaferraro about how BI is changing and how enterprises can best take advantage of such new and productive concepts as "operational BI" and "BI Modernization."

The Santaferraro interview, moderated by your’s truly from San Francisco, comes as part of a series of discussions with IT executives I’ll be doing this week from the Oracle OpenWorld conference. See the full list of podcasts and interviews.

Read a full transcript of the discussion.

Listen to the podcast. Download the podcast. Find it on iTunes/iPod. Learn more. Sponsor: Hewlett-Packard.

HP and Oracle team up on 'data warehouse appliances' that re-architect database-storage landscape

Oracle CEO Larry Ellison today introduced the company's first hardware products, a joint effort with Hewlett-Packard, to re-architect large database and storage configurations and gain whopping data warehouse and business intelligence performance improvements from the largest data sets.

The Exadata Programmable Storage Server appliance and the HP Oracle Database Machine, a black and red refrigerator-size full database, storage and network data center on wheels, made their debt at the Oracle OpenWorld conference in San Francisco. Ellison called the Machine the fastest database in the world.

HP Chairman and CEO Mark Hurd called the HP Oracle Database Machine a "data warehouse appliance." It leverages the architecture improvements in the Exadata Programmable Storage Server, but at the much larger scale and with other optimization benefits. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

The hardware-software tag team also means Oracle is shifting its relationships with storage array vendors, including EMC, Netezza, NetApp and Terradata. The disk array market has been hot, but the HP-Oracle appliance may upset the high end of the market, and then bring the price-performance story down market, across more platforms.

I think we can safely say that HP is a preferred Oracle storage partner, and that Oracle wants, along with HP, some of those high-growth storage market profits for their own. There's no reason to not expect a wider portfolio of Exadata appliances and more configurations like the HP Oracle Database Machine to suit a variety of market segments.

"We needed radical new thinking to deliver high performance," said Ellison of the new hardware configurations, comparing the effort to the innovative design for his controversial America's Cup boat. "We need much more performance out of databases than what we get."

This barnburner announcement may also mark a market shift to combined and optimized forklift data warehouses, forcing the other storage suppliers to find database partners. IBM will no doubt have to respond as well.

The reason for the 10x to 72x performance improvements cited by Ellison are do to bringing the "intelligence" closer to the data, that is bringing the Exadata Programmable Storage Server appliance into close proximity to the Oracle database servers, and then connecting them through InfiniBand connections. In essence, this architecture mimics some of the performance value created by cloud computing environments like Google, with its MapReduce technology.

Ellison said that rather than large data sets moving between storage and database servers, which can slow up performance at 1TB and larger databases, the new Exadata-driven configuration moves only the query information across the networks. The current versions of these optimized boxes use Intel dual-core technology, but they will soon also be fired up by six-way Intel multi-core processors.

Talk about speeds and feeds .... But the market driver in these moves is massive data sets that need to be producing near real-time analytics paybacks. We're seeing more and more data, and varyinf kinds of data, brought into data warehouses and being banged on by queries of applications and BI servers from a variety of business users across the enterprise.

HP and Oracle share some 150,000 joint customers worldwide, said HP Executive Vice President, Technology Solutions Group Ann Livermore. That means that these database boxes will have an army of sales and support personnel. HP will support the Machine hardware, Oracle the software. Both will sell it.