Progress gives CEP a performance boost with multi-core support on Apama

Progress Software this week announced the release of an enhanced Parallel Correlator for its Apama Complex Event Processing (CEP) platform so it can take advantage of multi-core, multi-processor hardware.

Progress claims a seven-fold increase in CEP performance on an eight-core server in the company’s internal benchmark testing of this version of the Parallel Correlator in what the company described as “real-world customer scenarios.” [Disclosure: Progress is a sponsor of BriefingsDirect podcasts.]

John Bates, who founded Apama in 1999 to build out technology based on his research at Cambridge University in the UK, believes CEP is an easier technology sell to business users than service oriented architecture (SOA) because a clear case can be made for the ability of a product like Apama to execute high-speed transactions based on the identification of millisecond movements in the business enivronment. It can also provide business managers and executives with split-second snapshots of how they are doing in their markets.

I guess we can think of CEP as SOA for high octane business intelligence (BI) for transactional and real-time insights and inferences from tremendously complex and often massive streams of services (and more). Mostly traditional BI comes from read-only agglomerations of fairly static SQL data, some of which needs a lot of handholding before it gives up its analytics gems.

Incidentally, I also spoke this week with Cory Isacsson at CodeFutures, busy at the MySQL show, who has a lot to say these days about database sharding and how applying it to OSS databases like MySQL gets, among other things, more BI love from transactional read/write SQL data. More here.

Back to CEP ... This is being newly perceived by some as much more tangible to business users than the more nerdy benefits of SOA, such as reuse of services for more agile programming of new applications. Talk about the benefits of CEP and business users eye light up. Talk about the benefits of SOA and even business process management (BPM) and their eyes can glaze over.

I should point out that my buddy at ActiveEndpoints Alex Neihaus (another disclosure on their sponsorship of BriefingsDirect podcasts) would argue that CEP and SOA are the real somnolence inducers, and that BPM and visual orchestration form the far better point on the business value arrow around service swarms. Talk among yourselves ...

In making the latest Apama announcement, Progress touts an IDC report on CEP (excerpts) that included evaluation of the 2008 version of the Apama platform. IDC gave the Progress product high ratings in the categories of “Low Latency,” the speed of event processing, “Business User Control,” how it works for the business people, and “Deterministic Behavior,” the predictability and repeatability of the event processing programs.

Lo, and although it is not mentioned in the Progress announcement, Apama did not get such high scores in the two other IDC categories, “Data Management,” and “Complex Event Detection.”

IDC does non-metaphysical squares, rather than Magic Quadrants, we should gather.

In the real world, the major market for CEP appears the beleaguered financial services industry and the government watchdog agencies that are overseeing them. This appears reflected in the Apama customers listed in this week’s announcement, including JP Morgan, Deutsche Bank, and FSA (Financial Services Authority) of the UK.

Written in the midst of this recession, the IDC report worries: "Because Apama is so closely identified with the financial markets, the current downturn is likely to negatively impact Apama's opportunity and growth prospects in the near term. Therefore, it is incumbent that Progress figure out how to cost effectively apply the technology to new markets with better short-term growth prospects."

At the beginning of last fall’s financial system meltdown, Bates told a reporter that there may be a silver lining for CEP even in the midst of a banking crisis. He foresees potential for greater use of CEP by both government regulators as well as the financial institutions that need to supply more and more detailed data to show how they are complying with new regulations now being formulated, as well as old regulations now being more rigorously enforced.

Too bad they can't apply it to card counting or my wife's algorithmic-rich shuffling of copius coupons for generating a simple groceries list. Just start with the old one, I keep telling her.

Other industries that both Progress and IDC agree might provide new markets for CEP include transportation and inventory control systems based on RFID, and ERP systems for manufacturing. I continue to be intrigued too by mobile commerce (Google Voice, anyone?), laced with locations services and other varibles like weather.

CEP is going to advance the competitive capabilities for a lot of companies. What's less clear is how they will manage that along with their BI, SOA, cloud, and other must do somedays on the IT groceries list.

Rich Seeley provided research and editorial assistance to BriefingsDirect on this blog. He can be reached at Writer4Hire.