Part 1 of 4: Web data services extend business intelligence depth and breadth across social, mobile, web domains

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download the transcript. Learn more. Sponsor: Kapow Technologies.

See popular event speaker Howard Dresner's latest book, Profiles in Performance: Business Intelligence Journeys and the Roadmap for Change, or visit his website.

The latest BriefingsDirect podcast discussion on the future of business intelligence (BI) -- and on bringing more information from more sources into an analytic process, and thereby getting more actionable intelligence out.

The explosion of information from across the Web, from mobile devices, inside of social networks, and from the extended business processes that organizations are now employing all provide an opportunity, but they also provide a challenge.

This information can play a critical role in allowing organizations to gather and refine analytics into new market strategies, better buying decisions, and to be the first into new business development opportunities. The challenge is in getting at these Web data services and bringing them into play with existing BI tools and traditional data sets.

This is the first in a series of podcasts, looking at the future of BI and how Web data services can be brought to bear on better business outcomes.

So, what are Web data services and how can they be acquired? Furthermore, what is the future of BI when these extended data sources are made into strong components of the forecasts and analytics that enterprises need to survive the recession and also to best exploit the growth that follows?

Here to help us explain the benefits of Web data services and BI is Howard Dresner, president and founder of Dresner Advisory Services, and Ron Yu, vice president of marketing at Kapow Technologies. The discussion is moderated by me, Dana Gardner, principal analyst at Interarbor Solutions.

Here are some excerpts:

Dresner: BI is really about empowering end users, as well as their respective organizations, with insight, the ability to develop perspective. In a downturn, what better time is there to have some understanding of some of the forces that are driving the business?

Of course, it's always useful to have the benefit of insight and perspective, even in good times. But, it tends to go from being more outward-focused during good times, focused on markets and acquiring customers and so forth, to being more introspective or internally focused during the bad times, understanding efficiencies and how one can be more productive.

So, BI always has merit and in a downturn it's even more relevant, because we are really less tolerant of being able to make mistakes. We have to execute with even greater precision, and that's really what BI helps us do.

... The future is about focusing on the information and those insights that can empower the individuals, their respective departments, and the enterprise to stay aligned with the mission of that organization.

... If you're trying to develop [such] perspective, bringing as much relevant data or information to bear is a valuable thing to do. A lot of organizations focus just on lots of information. I think that you need to focus on the right information to help the organization and individuals carry out the mission of that organization.

There are lots of information sources. When I first started covering this beat 20 years ago, the available information was largely just internal stores, corporate stores, or databases of information. Now, a lot of the information that ought to be used, and in many cases, is being used, is not just internal information, but is external as well.

There are syndicated sources, but also the entire World Wide Web, where we can learn about our customers and our competitors, as well as a whole host of sources that ought to considered, if we want to be effective in pursuing new markets or even serving our existing customers.

Yu: I fully agree with Howard. It's all about the right data and, given the current global and market conditions, enterprises have cut really deep -- from the line of business, but also into the IT organizations. However, they're still challenged with ways to drive more efficiencies, while also trying to innovate.

The challenges that are being presented are monumental where traditional BI methods and tools are really providing powerful analytical capabilities. At the same time, they're increasingly constrained by limited access to not only relevant data, but how to get timely access to data.

What we see are pockets of departmental use cases, where marketing departments and product managers are starting to look outside in public data sources to bring in valuable information, so they can find out how the products and services are doing in the market.

... Inclusive BI essentially includes new and external data sources for departmental applications, but that's only the beginning. Inclusive BI is a completely new mindset. For every application that IT or line of business develops, it just creates another data silo and another information silo. You have another place that information is disconnected from others.

... There is effectively a new class of BI applications as we have been discussing, that depends on a completely different set of data sources. Web data services is about this agile access and delivery of the right data at the right time.

With different business pressures that are surfacing everyday, this leads to a continuous need for more and more data sources.

... Critical decision-making requires, as Howard was saying earlier, that all business information is easily leveraged whenever it's needed. But today, each application is separate and not joined. This makes the line of business and decision- making very difficult, and it's not in real time.

An easier way

As this dynamic business environment continues to grow, it’s completely infeasible for IT to update their existing data warehouses or to build a new data mart. That can't be the solution. There has to be an easier way to access and extract data exactly where it resides, without having to move data back and forth from data bases, data marts, and data warehouses, which effectively becomes snapshot.

... Web data services provides immediate access to the delivery of this critical data into the business user's BI environment, so that the right timely decisions can be made. It effectively takes these dashboards, reporting, and analytics to the next level for critical decision-making. So when we look deeper into this and how is this actually playing out, it's all about early and precise predictions.

Dresner: ... Some IT organizations have become pretty inflexible. They are focused myopically on some internal sources and are not being responsive to the end user.

To the extent that they can find new tools like Web data services to help them be more effective and more efficient, they are totally open to giving line of business self-service capabilities.

You need to be careful not to suffer from what I call BI myopia, where we are focused just on our internal corporate systems or our financial systems. We need to be responsive. We need to be inclusive of information that can respond to the user's needs as quickly as possible, and sometimes the competency center is the right approach.

I have instances where the users do wrest control and, in my latest book, I have four very interesting case studies. Some are focused on organizations, where it was more IT driven. In other instances, it was business operations or finance driven.

Yu: ... For example, in leading financial services companies, what they're looking for is on this theme of early and precise predictions. How can you leverage information sources that are publicly available, like weather information, to be able to assess the precipitation and rainfall and even the water levels of lakes that directly contribute to hydroelectricity?

If we can gather all that information, and develop a BI system that can aggregate all this information and provide the analytical capabilities, then you can make very important decisions about trading on energy commodities and investment decisions.

Web data services effectively automates this access and extraction of the data and metadata so that IT doesn't have to.

Web data services effectively automates this access and extraction of the data and metadata and things of that nature, so that IT doesn't have to go and build a brand new separate BI system every time line of business comes up with a new business scenario.

... It's about the preciseness of the data source that the line of business already understands. They want to access it, because they're working with that data, they're viewing that data, and they're seeing it through their own applications every single day.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download the transcript. Learn more. Sponsor: Kapow Technologies.

See popular event speaker Howard Dresner's latest book, Profiles in Performance: Business Intelligence Journeys and the Roadmap for Change, or visit his website.

Process isomorphism: The critical link between SOA and BPM

This guest post comes courtesy of Jason Bloomberg, managing partner at ZapThink.

Take the BriefingsDirect middleware/ESB survey now.

By Jason Bloomberg

ZapThink has long championed the close relationship between business process management (BPM) projects and service-oriented architecture (SOA) initiatives. As anyone who has been through our Licensed ZapThink Architect Bootcamp can attest, we have a process-centric view of SOA, where the point to building loosely coupled business services is to support metadata-driven compositions that implement business processes, what we call service-oriented business applications, or SOBAs, for want of a better term.

Nevertheless, there is still confusion on this point, among enterprise practitioners who see BPM as a business effort and SOA as technology-centric, among vendors who see them as separate products in separate markets, and even among pundits who see Services as supporting business functions but not business processes.

On the other hand, there are plenty of enterprise architects who do see the connection between these two initiatives, and who have pulled them together into "BPM enabled by SOA" efforts. This synergy, however, is not automatic, and requires some hard work both among the people focusing on optimizing business processes to better meet changing business needs as well as the team looking to build composable business services that support the business agility and business empowerment drivers for their SOA initiatives.

ZapThink has worked with many such organizations, and over time a distinct best practice pattern has emerged, one that is both fundamental as well as subtle, and as a result, has fallen through the cracks of compendia of SOA patterns: the Process Isomorphism pattern. Understanding this pattern and how to apply it can help organizations pull their BPM and SOA efforts together, and even more importantly, improve the alignment of their SOA initiatives with core business drivers.

What is Process Isomorphism?

An isomorphism is a mathematical concept that expresses a relationship between two concepts that are structurally identical but may differ in their respective implementations. A very simple example would be two tic-tac-toe games, one with the traditional X's and O's, and the other with, say, red dots and blue dots. The game board and the rules are the same, in spite of the difference in symbols the players use to play the games. If two particular games follow the same sequence of moves, they would be isomorphic.

The term process isomorphism usually refers to two processes that are structurally identical, typically between two companies in the same industry. For example, if the order-to-cash process is structurally identical between companies A and B, that is, the same steps in the same order with the same process logic, those processes would be isomorphic, even if the two companies had differences in their underlying technical implementations of the respective processes.

We're using the term differently here, however. Process isomorphism in the SOA context is an isomorphism between a process on the one hand, and the SOBA that implements it on the other. In other words, if you were to model a business process, and as a separate exercise, model the composition of services that implements that process, where those two models have the same structure, then they would be isomorphic.

One conversation that helped crystallize this notion was with John Zachman, who was explaining some of the changes he has recently made to his seminal Zachman Framework. He has renamed Row 3, which had been the System Model row, to the System Logic row. People were confusing the System Model with the physical representation of the system, which resides one row down. Our discussion of process isomorphism is essentially a design practice that relates these two rows of Column 2, the How column. In essence, the process logic model is one level above the service composition model that implements the process logic. The process isomorphism pattern states that these two models should be isomorphic.

Process Isomorphism in Practice

We've been using a wonderful example of Process Isomorphism on our LZA Course for a few years now, courtesy of British Petroleum (BP), who presented at our Practical SOA event in February, 2008 (more about our upcoming Practical SOA event). The presentation focused on how process decomposition is the common language between business and IT efforts, and one of the examples focused on the Well Work Performance process, one of thousands of processes in their oil drilling line of business:

BP's Well Work Performance Process

The Description column in the chart above reflects the four main subprocesses that make up this process. The Sub-Task columns represent individual sub-tasks, or steps in the process. Finally, the Supporting Service Name column indicates the Business Service that implements the corresponding sub-task. The fact that there is a one-to-one correspondence between sub-tasks and supporting Services, combined with the implied correspondence between the process logic and the composition logic, illustrates Process Isomorphism. In this simple example, the process logic is a simple linear sequence, but if the logic were more complex, say with branching and error conditions, then the process would exhibit isomorphism if the composition logic continued to reflect the process logic.

It is important to point out that the one-to-one correspondence between process sub-tasks and supporting Services is by no means a sure thing, and in practice, many organizations fail to design their compositions with such a correspondence. Frequently, the issue is that the SOA effort is excessively bottom-up, where architects specify services based upon existing capabilities. Such bottom-up approaches typically yield services that don't match up with process requirements. Equally common are BPM efforts that are excessively top-down, in that they seek to optimize processes without considering the right level of detail for those processes to enable services to implement steps in the processes. Only by taking an iterative approach where each iteration combines top-down and bottom-up design is an organization likely to achieve Process Isomorphism.

The Process Isomorphism Value Proposition

The essential benefit of Process Isomorphism is being able to use the process representation to represent the composition and vice-versa. While these concepts are fundamentally different, in that they live on different rows of the Zachman Framework, the isomorphism relationship allows us the luxury of considering them to be the same thing. In other words, we can discuss the composition as though it were the process, and the process as though it were the composition.

This informal equivalence gives us a variety of benefits. For example, if process steps correspond directly to services, then service reuse is more straightforward to achieve than when the correspondence between steps and services is less clear. Service reuse discussions can be cast in the context of process overlaps. If two processes share a sub-task, then the SOBAs that implement those processes will share the supporting service. In addition, the metadata representation of the composition logic, for example, a BPEL file, will represent the process logic itself. Without process isomorphism, the process logic the BPM team comes up with won't correspond directly to the BPEL logic for the supporting composition. This disconnect can lead directly to misalignment between IT capabilities and business requirements, and also limits business agility, because a lack of clarity into the relationship between process and supporting composition can lead to unintentional tight coupling between the two.

The ZapThink Take

Perhaps the greatest benefit of Process Isomorphism, however, is that it helps to establish a common language between business and IT. The business folks can be talking about processes, and the IT folks can be talking about SOBAs, and at a certain level, they're talking about the same thing. The architect knows they're different concepts, of course, but conversations across the business/IT aisle no longer have to dwell on the differences.

The end result should be a better understanding of the synergies between BPM and SOA. If process specialists want to think of business services as process sub-tasks, then they can go right ahead. Similarly, if technical implementers prefer to think of business processes as being compositions of services, that's fine too. And best of all, when the BPM team draws the process specification on one white board and the SOA team draws the composition specification on another, the two diagrams will look exactly alike. If that's not business/IT alignment, then what is?

This guest post comes courtesy of Jason Bloomberg, managing partner at ZapThink.

Take the BriefingsDirect middleware/ESB survey now.

---

SPECIAL PARTNER OFFER

SOA and EA Training, Certification,
and Networking Events

In need of vendor-neutral, architect-level SOA and EA training? ZapThink's Licensed ZapThink Architect (LZA) SOA Boot Camps provide four days of intense, hands-on architect-level SOA training and certification.

Advanced SOA architects might want to enroll in ZapThink's SOA Governance and Security training and certification courses. Or, are you just looking to network with your peers, interact with experts and pundits, and schmooze on SOA after hours? Join us at an upcoming ZapForum event. Find out more and register for these events at http://www.zapthink.com/eventreg.html.

---