Integrien deepens analytics, betters interoperability and userability in Q4's Alive 6.0 release

Read a full transcript of the discussion. Listen to the podcast. Sponsor: Integrien Corp.

The movement of IT and systems management to the end-to-end business service value level has been a long time in coming. Yet the need has never been higher. Enterprises and on-demand application providers alike need to predict how systems will behave under a variety of conditions.

Rather than losing control to ever-increasing complexity -- and gaining less and less insight into the root causes of problematic applications and services -- operators must gain the ability to predict and prevent threats to the performance of their applications and services. Firefighting against applications performance degradation in a dynamic service-oriented architecture (SOA) just won't cut it.

By adding real-time analytics to their systems management practices, IT operators can determine the normal state of how systems should be performing. Then, by measuring the characteristics of systems under many conditions over time, administrators can gain predictive insights into their entire operations, based on a business services-level of performance and demand. They can stay ahead of complexity, and therefore contain the costs of ongoing high-performance applications delivery.

I recently had a podcast discussion with Mazda Marvasti, the CTO of Integrien Corp., on managing complexity by leveraging probabilistic systems management and remediation. I learned that Integrien's Alive suite uses probabilistic analysis to predict IT systems problems before costly applications outages. Furthermore, I received some details on the next Alive 6.0 release in Q4 of this year.

Here are some excerpts:

Can you give us some sense of the direction that the major new offerings within the Alive product set will take?

Basically, we have three pillars that the product is based on. First is usability. That's a particular pet peeve of mine. I didn't find any of the applications out there very usable. We have spent a lot of time working with customers and working with different operations groups. ... The second piece is interoperability. The majority of the organizations that we go to already have a whole bunch of systems, whether it be data collection systems, event management systems, or configuration management databases, etc.

Our product absolutely needs to leverage those investments -- and they are leveragable. But even those investments in their silos don’t produce as much benefit to the customer as a product like ours going in there and utilizing all of that data that they have in there, and bringing out the information that’s locked within it.

The third piece is analytics. What we have in the product coming out is scalability to 100,000 servers. We've kind of gone wild on the scalability side, because we are designing for the future. Nobody that I know of right now has that kind of a scale, except maybe Google, but theirs' is basically the same thing replicated thousands of times over, which is different than the enterprises we deal with, like banks or health-care organizations.

A single four-processor Xeon box, with Alive installed on it, can run real-time analytics for up to 100,000 devices. That’s the level of scale we're talking about. In terms of analytics, we've got three new pieces coming out, and basically every event we send out is a predictive event. It’s going to tell you this event occurred, and then this other set of events have a certain probability within a certain timeframe to occur.

Not only that, but then we can match it to what we call our "finger printing." Our finger printing is a pattern-matching technology that allows us to look at patterns of events and formulate a particular problem. It indicates particular problems and those become the predictive alerts to other problems.

Now, with SOA and virtualization moving into application-development and data-center automation, there is a tremendous amount of complexity in the operations arena. You can’t have the people who used to have the "tribal knowledge" in their head determining where the problems are coming from or what the issues are.

The problems and the complexity have gone beyond the capability of people just sitting there in front of screens of data, trying to make sense out of it. So, as we gained efficiency from application development, we need consistency of performance and availability, but all of this added to the complexity of managing the data center.

That’s how the evolution of the data center went from being totally deterministic, meaning that you knew every variable, could measure it, and had very specific rules telling you if certain things happened, and what they were and what they meant -- all the way to a non-deterministic era, which we are in right now.

Now, you can't possibly know all the variables, and the rules that you come up with today may be invalid tomorrow, all just because of change that has gone on in your environment. So, you cannot use the same techniques that you used 10 or 15 years ago to manage your operations today. Yet that’s what the current tools are doing. They are just more of the same, and that’s not meeting the requirements of the operations center anymore.

I’ve been working on these types of problems for the past 18 years. Since graduate school, I’ve been analyzing data extraction of information from disparate data. I went to work for Ford and General Motors -- really large environments. Back then, it was client-servers and how those environments were being managed. I could see the impending complexity, because I saw the level of pressure that there was on application developers to develop more reusable code and to develop faster with higher quality.

The run book is missing that information. The run book only has the information on how to clean it up after an accident happens.

That’s the missing piece in the operations arena. Part of the challenge for our company is getting the operations folks to start thinking in a different fashion. You can do it a little at a time. It doesn’t have to be a complete shift in one fell swoop, but it does require that change in mentality. Now that I am actually forewarned about something, how do I prevent it, as opposed to cleaning up after it happens.

Read a full transcript of the discussion. Listen to the podcast. Sponsor: Integrien Corp.

Before we know who owns the SOA business case, how about simple business processes?

There's a good article on "owning" the business case for SOA on SearchWebServices.com. Some of my most respected analysts are quoted.

But is the question posed a relevant one? While making the business case for SOA is and will be a fascinating topic for some time, we may be jumping the gun.

From where I sit, just about everyone that has a strategic role in IT and business decisions at an enterprise has an "ownership" stake in SOA. It's that pervasive. The COO may be the best person under many current organizational charts to see all the moving SOA parts.

Yet buy-in and inclusiveness -- both wide and deep -- for SOA are essential, so it can't really fall to any one person. Assigning "business value" ownership is too abstract, really, for real-world companies to begin using it and embracing SOA. SOA is ubiquitous in its effects. The positioning of SOA as an abstraction is holding back its embrace and adoption.

So let's look at more practical questions on SOA and business value, before we go shooting for the moon. Sadly, in even the most progressive enterprises, the ownership of a single business process is ambiguous. Organizations have been ceated for decades based on the notion of decentralization -- which is just another way of breaking up complexity into small chunks and assigning responsibility for the chunks, often at the expense of minding the whole. Very few individuals or teams are defined or incentivized to manage an entire business process. Yet this an essential stepping stone to SOA, and to eventually making the business case for SOA.

We see attempts to proffer SOA from the top down, with even less emphasis on adoption from the bottom up. What I'm saying is also, and perhaps predominantly, build it from the middle out. Create the new middle for SOA at the business process level, and then evangelize it in any which way.

In effect, SOA and its foundational core, business processes, are fighting back against the long-term tide of decentralization and IT specialization. SOA says you can now make the chucks of discrete IT resources relate far better, so why not begin to look at an entire process and work to make it more efficient, and more flexible? Why not extract the best of specialization and improve, refine and reuse the parts best in the context of general business -requirements whole? See the forest and the trees. Make better business decisions -- operationally and strategically -- as a result.

As Dr. Paul Brown points out in a book I recently helped review via a sponsored podcast, Succeeding with SOA: Realizing Business Value Through Total Architecture (Addison-Wesley, April 2007), the business process is the right level to assign "ownership." Now.

When an analyst or architect -- as well as their teams -- begin to see themselves as managing and evangelizing on a business process level, then SOA can begin to make strides as a concept and methodology more broadly. To try and inject SOA into a company broadly, then discretely is putting the cart in front of the horse. Better yet to re-arrange all the horses and carts based on the right trips for the right loads, making it easier to change horses and carts as needed.

I like the idea of cross-functional teams (horses, carts, drivers, and caravans) created that serve a business process lifecycle. These would be pods (perhaps virtual in nature) of tightly-coordinated people with the right mix of skills and experience -- specific and general, technical and business-oriented, able to communicate as a team on many levels.

Like the Ray Bradbury book, Fahrenheit 451, where individuals learn and carry on whole books in their memories as a way to preserve the books and their knowledge, business process pods would retain and refine the essence of a business process and care for it and extol its virtues throughout an enterprise. They would cross all the chasms across the constituent services but at the higher business value level.

We've heard talk of a "T" person from SOA evangelists at IBM, whereby the horizontal bar in the "T" represents business acumen, and the vertical bar represents technical depth. But I like the idea of the cross-functional pod better -- a team of, by, and for the business process.

The ownership of a business process (never mind SOA) is too much for one person. A multi-talented team can provide the wetware and organizational dynamism to get SOA started on a practical, middle level -- that of a business process as a productivity entity. This step is what's needed before we start assigning ownership for the business case for SOA.

Developing enterprise applications for mobile devices remains way too hard

A logjam exists between developers and their ability to productively deliver enterprise applications and data to mobile devices, such as cell phones, PDAs, and so-called converged devices like the Apple iPhone.

The logjam is complexity and too many obnoxious variables. To develop applications that reach even a small number of major handset environments means big-time custom plumbing, from the various data sources, to the mixture of networks, to the choices on synchronization, to the various security needs, to the many user interfaces and mobile client operating systems. Managing all these variables requires a high degree of skill across many different skill sets. There are not many developers that fit this bill in your average enterprise.

And this all means a lot of time and money is required to bring just a few basic applications to just a few basic mobile clients. No wonder enterprise mobility stubbornly remains below the radar for IT leadership. Mobile remains relegated to the crowded back burner of IT imperatives.

And given all the variables and high degree of required customization, few ISVs have emerged to try and make a living at producing mass-market mobile applications. The subsequent lack of killer applications, other than standards-based email and text messaging, reduces the appetite to take on the infrastructure complexity for taking the corporate datacenter out to the mobile client across commercial mobile networks.

Also impacting the complexity is the diversity in how mobile devices work from geographic and regulatory market to market. It's nearly impossible to envision a global approach to mobile wireless computing, as we've seen for desktop and web computing. Designing for one mobile market does not give you much of a leg-up in reaching many others.

The "inclusive platform" approach of dictating the exact device and/or runtime environment up and down the stack that mobile applications play in is itself stifling given the inability to take advantage of the low-cost devices and services available via commercial mobile service carriers. What's more, aligning the back-end and front-end infrastructure does not necessarily align with how the mobile telecommunications carriers and handset operators, well ... operate. They don't like the idea of losing control of what their clients do on their networks.

Clearly, wireless handheld delivery of enterprise data and applications has yet to reach its untapped and vast potential.

Microsoft for years has been grappling with these issues, with many fits and starts. There have been some impressive successes with Windows Mobile, but Microsoft has by no means sewn up the field of mobile enterprise applications design and delivery. Microsoft with its Windows Mobile approach has not yet achieved a critical mass for how business applications and data can be driven out to a field-based workforce. And it's likely that today's widely heterogeneous environment for end points and devices will be with us -- at least in the U.S. -- for a long time to come.

These seemingly intractable roadblocks to wider mobile business use are one reason we're seeing what amounts to appliances for the client devices. The Apple iPhone, which launched with great fanfare in late June, is a prime example. Apple has fused hardware, software and applications -- as well as a few critical APIs -- and has picked one opening-inning carrier, AT&T, as well as Wi-Fi generally, for connectivity. And that combination of attributes, along with what should soon be more APIs, makes the converged device/appliance approach not just a consumer affair -- it begins to help reduce the complexity for enterprise mobility too.

Microsoft in the summer said that it likes the idea of such fused mobile clients -- the appliance on the client -- so much that it has hinted it will produce such integrated devices too. Rumors persists that Google also has its sights on a mobile handheld appliance of some sort. Things are clearly heating up.

These nifty clients should go quite a way to making enterprise mobility far easier by allowing developers to exploit them with fewer interface and connectivity variables to manage. But the converged client still needs a back-end or middleware counterpart to help coordinate an enterprise's data, logic, and security needs. Sybase is hard at work on what may prove to be a game changer for such enterprise mobility middleware -- especially when coupled with a converged device such as the iPhone.

Sybase has bet its future growth on mobility. And while the Dublin, Calif. company has not yet announced the details of its full stack, the Sybase vision makes a lot of sense. Expect in the next year to see what amounts to a distributed middleware system approach to enterprise mobility from Sybase that may break the current logjam in enterprise mobile development and deployment. If Sybase can shake up the enterprise mobility infrastructure industry -- and partner effectively with the likes of Apple's iPhone -- then Microsoft's response will need to be swift and significant.

Dare I say it? What the heck: Look to 2008 to be the year that enterprise mobility finally gets some legs.

Survey uncovers heightening reliance on search across business purchasing

Listen to the podcast. Or read a full transcript. Sponsor: ZoomInfo.

It seems that businesses, whether they're small or global 2000 concerns, are buying more supplies using search at some point in the B2B procurement process. Some people begin and end a procurement journey with search. They actually buy the products through a strictly search-dependent process.



Yet many still use a combination of word-of-mouth, search, and traditional information gathering to guide them to the best deals on the most goods.

To find out just how much B2B buying behaviors are shifting, Enquiro Search Solutions conducted a survey earlier in 2007. They found that online search was consistently employed throughout the entire buying process, from awareness right through to purchase.

There’s still a lot of back and forth: Offline factors influence online activity, and vice-versa, for a merging of the online and the offline worlds. In an audio podcast discussion, as well as the accompanying BriefingsDirect multi-media video-podcast, I helped plumb the depths of Enquiro's findings and then vetted them through the experiences of B2B search engine ZoomInfo.

Join Gord Hotchkiss, President and CEO of Enquiro, and Bryan Burdick, COO of ZoomInfo, with moderation by myself, Dana Gardner, for a deep dive on B2B search trends and analysis.

Here are some excerpts:

We did the original survey in 2004 and, at the time, there wasn't a lot of research out there about search in general, even on the consumer side. There was virtually nothing on the B2B side. The first survey ... certainly proved that search was important. We found that online activity, in particular that connected with search activity, was consistent in a large percentage of purchases. In 2007, we added more insight to the methodology. We wanted to understand the different roles that are typical in B2B purchases -- economic buyers versus technical buyers versus user buyers. We also wanted to get more understanding of the different phases of the buying cycle.

As far as the main takeaways from the study, obviously online activity is more important than ever. In fact, we asked respondents to indicate from a list of over 30 influencers what was most important to them in making the purchase decision. Online factors, such as interaction with the vendor Website and interaction with the search engine were right up there with the traditional winner, word of mouth. What we see is a real link between those and looking for objective information and specific detail.

We did notice an evolution of behavior as you move through the funnel, and the nature of the interactions with the different online resources changes how you navigate to them and how you go to different sites for information. But, online research was consistent through the entire process, from awareness right through to purchase. There’s a lot of back and forth. ... We saw a merging of the online and the offline worlds in making these decisions and trying to come to what’s the right decision for your company or what’s the right product or service.

We just found increased reliance on online to do that research. When we say "increased reliance," we're probably talking 10 percentage points up over the three years. So, if 65 percent of the people were doing it in 2004, 75 percent of the people are doing it now. That’s primarily where we saw the trends going.

When we looked at the different phases of the buying cycle, it starts with awareness. You become aware that you need something. There was a high percentage of people -- in the high 60-percent range -- who said, "Once I become aware that I need something, the first place I'm going to go is the search engine to start looking for it." A lot of that traffic is going to end up on Google. It was the overwhelming choice among general search engines for B2B buyers.

But, as you move through the process, you start doing what we call a "landscape search." The first search is to get the lay of the land to figure out the information sites that have the information you are looking for. Who are the main vendors playing in this space? Where are the best bets to go and get more information to help make this purchase decision?

So, those first searches tend to be fairly generic -- shorter key phrases -- just to get the lay of the land to figure out where to go. As you progress, search tends to become more of a navigational shortcut, and we’ve seen this activity increase over the last two to three years. Increasingly, we're using search engines to get us from point A to point B online.

We also wanted to get a retroactive view of a successful transaction. So, in the second part of the survey, we asked them to recall a transaction they had made in the past 12 months. We wanted to see whether that initial search led to a successful purchase down the road, and, at the end of the road, how the different factors influenced them. So, we actually approached them from a couple of different angles.

Now, 85 percent of these people say they're using online search for some aspect of this purchasing process. It strikes me that this involves trillions of dollars worth of goods. These are big companies and, in some cases, buying lots of goods at over a hundred thousand dollars a whack. Do you concur that we're talking about trillions of dollars of B2B goods now being impacted significantly by the search process?

Absolutely. The importance of this is maybe the most mind-numbing fact to contemplate. Traditionally, the B2B space has been a little slow to move into the search arena. Traditionally, in the search arena, the big advertisers tend to be travel or financial products. B2B is just starting to understand how integral search is to all this activity. When you think of the nature of the B2B purchase, risk avoidance is a huge issue. You want to make sure that whatever decisions you make are well-researched and well-considered purchases. That naturally leads to a lot of online interaction.

The business information search is a primary factor driving [ZoomInfo's] growth. Our company right now is growing on two fronts. One is our traditional paid-search model, where we have subscription services focused on people information that is targeted at salespeople and recruiters as a source for candidates and prospects.

The more rapidly growing piece of our business is the advertising-driven business information search engine, which I think is a really interesting trend related to the concept you guys were just talking about. Not only does the B2B advertiser spend lots of money today trying to reach out, but the B2B searcher has new tools, services, and capabilities that provide a richer, better, more efficient search than they’ve had through the traditional search engines.

Everybody needs to be focused on search. I can’t see an exception. You mentioned the percentage that said they would go online. We segmented out the group that didn’t indicate they go online to see what was unique about them. The only thing unique about them was their age. They tended to be older buyers and tended to be with smaller organizations, where the CEO was more actively involved in the purchase decision. That was really the only variants we saw. If it’s a generational thing, then obviously that percentage is going to get smaller every year.

... From a vertical business information search perspective, that we’re really in the first inning here. A lot of interesting trends and enhancements are going to be coming down the road. One in particular that may have an influence in the next year or two is the community aspect within the search. ... I think that you’ll start to see a marriage of, not only B2B search, but also online community and a factoring into that whole process. Then, who knows where we’ll go from there? ... The word of community.

Listen to the podcast. Or read a full transcript. Sponsor: ZoomInfo.

Book review discussion: 'Total Architecture' elevates SOA to its business benefits potential

Listen to the podcast. Or read a full transcript of the discussion. Sponsor: TIBCO Software.

The impact that Services Oriented Architecture (SOA) has on an organization is deep and wide. The changes required to bring SOA to fruition and make it as productive as possible affect the way both IT leaders and business managers think and operate.

To provide a solid foundation on how SOA impacts a business, Dr. Paul Brown, a principal software architect at TIBCO Software, recently wrote "Succeeding with SOA: Realizing Business Value Through Total Architecture." The book emphasizes a "Total Architecture" perspective for SOA and advocates how business processes are the real focus for enterprises to prosper.

Many activities that used to be done manually either completely or partially now ingrained in IT systems. SOA helps elevate those activities into loosely coupled services with standard interfaces so they can be used and reused flexibly within business processes, while leveraging the underlying IT assets and automation benefits.

Brown calls on businesses to define, fully grasp, control, manage and adjust the processes as rudders that steer overall business agility, powered by SOA methods. However, enterprise business processes and enterprise IT systems are so intertwined that you can’t really talk about designing one without designing the other -- hence the need for "Total Architecture."

In this podcast, Dr. Brown is interviewed about his book and the concepts of Total Architecture by enterprise architect Todd Biske, with moderation by myself, Dana Gardner, principal analyst at Interarbor Solutions. I do hope that people take a look at this book. I think it has a lot to offer.

Here are some excerpts:

The structure and organization of IT is very often not completely aligned with the business. The care and feeding of the IT infrastructure has become a focal point of a lot of the IT investment. What we’ve lost is the connection between what’s going on in IT and what’s going on in the business.

The structure of our IT solutions are difficult to map onto the structure of the business processes. It’s that realignment that we’re trying to address with SOA. Ideally, the services that we’re building -- that have technical implementations -- are building blocks of business processes.

The real business value comes out of having a portfolio of business functionality in the form of services that you can quickly reorganize and re-orchestrate to build new business processes or to modify your existing processes. That’s where the big business win comes in.

Dr. Brown points out that a lot of times IT may be misaligned and not be able to see the business processes that are spanning the silos. It may also be the case that the business isn’t either. So, you’re really setting up challenges, when you’re trying to view things from more of an enterprise perspective.

... The pressures of globalization make viewing your entire enterprise and your business as a whole even more important. It’s hard to continue to operate in a niche and continue to sustain that organization for many years, whether it’s from the IT aspect or the business aspect.

It’s distressing, when I go into a company, how infrequently I find anybody who can articulate what the end-to-end business process is that produces the key results or can tell me what the entire order-to-cash cycle looks like. There are lots of experts for fragments of it. The IT focus has traditionally been on fragments of the processes.

The pressure these days is to improve the overall business process, the response time to the customers and partners, and the overall quality of what’s going through. So we need this focus on end-to-end business process and the focus on the end-to-end system interaction that helps bring that business process to life.

What we need to introduce into the picture is more thinking in terms of how these different silos play together to make the business work. That has to happen on both the business side, looking at business processes, and the technology side, looking at systems.

... We need a proactive enterprise architecture group that’s willing to roll up its sleeves and get its hands dirty, touching the individual projects that are going on. They really need to coordinate their work, so that we don’t end up with the chaos that we’re in today, and so that we have pieces that elegantly fit together and can be rearranged to achieve new business goals.

When you design a business process, you’re making assumptions about what systems are going to do and what people are going to do. You can’t validate the assumptions until you actually start doing the technical design. You need to have a systems architect involved when you are planning business processes.

Conversely, you can’t really make modifications in the system without inadvertently changing the business processes. So it doesn’t make sense to separate these activities. You really have to treat them as if you're architecting the business, which consists of both business processes and systems.

One of the things that was impactful for me was the notion that SOA can be a catalyst to thinking about things differently and bridging this business-IT chasm. And also that SOA can be a precursor to a whole new conversation about how to think about businesses from this perspective of "Total Architecture."

... Your business processes and your systems are intertwined, and the design of one affects the other. That’s reality. You can choose to ignore your business processes and let them evolve by accident, as Todd was talking about earlier. That’s how we got in the bind we’re in right now. To a large extent, we’ve been focused on individual profit centers and driving down cost in individual functions. We’ve lost sight of the end-to-end business processes.

The resurgence of interest in customer loyalty, customer service, and all of that is really a reflection that, as a business, we need to stand back and look at what we’re doing to our customers and our partners. The investments we’re making and the individual activities in the business processes are fine, but in order to remain competitive we need to stand back and take the holistic view. The holistic view simply means you need to understand what that business process looks like before you can improve it.

I’ve seen horror stories of people who have tried to improve a business process by examining a small portion and making some changes, without realizing that the changes that they’re making are having an adverse impact on the bigger business process. So while they thought they were improving things -- they actually made them worse. You can’t live with that.

Listen to the podcast. Or read a full transcript of the discussion. Sponsor: TIBCO Software.