Splunk adds change-management and Windows support to IT search software

IT search company Splunk today added to its arsenal of tools for IT managers with the launch of Splunk for Change Management, an application to audit and detect configuration and changes, and Splunk for Windows, which indexes all data generated by Windows servers and applications.

The San Francisco company provides a platform for large-scale, high-speed indexing and search technology geared toward IT infrastructures. The software, which comes in both free and enterprise versions, allows a company to search and navigate data from any application, server, or network device in real time. [Disclosure: Splunk is a sponsor of BriefingsDirect podcasts.]

Splunk for Change Management, which requires an enterprise license, continuously audits all configurations and changes, detects unauthorized changes, validates change deployment, and discovers service-impacting changes during incident response.

The new application leverages the existing Splunk Platform, allowing users to combine change audit events, configuration data, activity and error logs, and actual system and user behavior. This differentiates it from the traditional approach, which is often disconnected from incident response and cut off from other sources of IT data.

Among the features of the new product are:

• Out-of-the box dashboards with over 40 reports showing changes across all datacenter components including applications, servers and network devices.
  

• Predefined alerts that detect unauthorized change based on configuration variances and correlation with service desk systems.
  

• Predefined searches to help identify service-impacting changes
  

• Integration with service desk systems that validates the effect of change on system behavior.

Splunk for Windows, a free application, integrates Splunk's IT search with Microsoft's System Center Operations Manager's command and control view of the Windows infrastructure.

Splunk indexes event logs, registry keys, performance metrics, and applications log files, making all the data searchable from a single place.

Reports and dashboards included in the application provide a bird's eye view of service levels and problems across a large number of servers and applications, and predefined alerts can warn of cross-component problems.

Splunk has a variety of solutions for IT managers and developers who need some visibility into their various systems and components. Just a few weeks ago, I wrote about the Splunk Platform.

"The Splunk Platform and associated ecosystem should quickly grow the means to bridge the need for transparency between runtime actualities and design-time requirements. When developers can easily know more about what applications and systems do in the real world in real time, they can make better decisions and choices in the design and test phases. This obviously has huge time- and money-saving implications."

And, more than two years ago, I did a podcast about Splunk, when it launched the Splunk Base, an open Creative Commons-licensed repository of Wikis that with volume adoption to give systems troubleshooters a searchable library of knowledge about what ails IT components and how to swiftly remedy those ills. You can listen to the podcast here.

Splunk for Change Management pricing starts at $4,000 and requires an enterprise license. A 30-day free trial is available.

Splunk for Windows is free and is now available on the Splunk Base site.

HP's security management model brings comprehensive approach to corporate risk reduction

Listen to the podcast. Read a full transcript. Sponsor: Hewlett-Packard.

We live in an age where there is so much exposure to risk and information security pitfalls that when data gets out -- it gets out in a big way. Devastating security breaches are becoming routine in the media, and those are only the ones we hear about. There have never been more ways for sensitive data and corporate assets to be poorly managed.

So how do large, complex companies and governments better protect themselves? How do they manage new compliance regulations that spout up and change constantly? How can people and processes be better organized to thwart bad practices before they lead to potentially catastrophic losses?

Surprisingly, the answer has more to do with management methodology than security technology. In this sponsored podcast discussion learn from HP security expert Tari Schreider how a comprehensive new security management approach, called Information Security Service Management (ISSM) and its reference model, offers companies a comprehensive framework with which to finally come to grips with myriads corporate risks and daunting compliance requirements.

Here are some excerpts:

When we read about a breach of security -- the proverbial tape rolling off the back of the truck with all of the Social Security numbers -- we find that, when you look at the morphology of that security breach, it’s not necessarily that a product failed. It’s not necessarily that an individual failed. It’s that the process failed. There was no end-to-end workflow and nobody understood where the break points were in the process.

It’s not unusual for us to present back to a client that they have three or four different identity management systems that they never knew about. They might have four or five disparate identity stores spread throughout the organization. If you don’t know it and if you can’t see it, you can’t manage it.

HP's ISSM ... positions security as a driver for IT business-process improvement. It reduces the amount of operational risk, which ensures a higher degree of continuity of business operations. It’s instrumental in uncovering inadequate or failing internal processes that stave off security breaches. It also turns security into a highly leveraged, high-value process within your organization. ... It allows you to actually make security sticky to other business processes.

When I sit down with CFOs or CIOs or business-unit stakeholders, I can ask one question that will be a telltale sign of whether they have a well-managed, continuously improving information security program. That question is, "How much did you spend on security last year?" Then I just shut up. ... They don't have any answer. If you don’t know what you are spending on security, then you actually don’t know what you are doing for security. It starts from there.

We show them that they actually have 40, 50, or 60 [security products], because they're spread throughout the organization, and there's a tremendous amount of duplication. ... Today, security controls are buried in some spreadsheet or Word document, and there is really no way to manage the behavior of those controls.

We want to work with that individual and position the ISSM Reference Model as the middle layer, which is typically missing, to pull together all the pieces of their disparate security programs, tools, policies, and processes in an end-to-end system.

Historically, businesses throughout the world have lacked the discipline to self-regulate. So there is no question that the more onerous types of regulations are going to continue. That's what happened in the subprime [mortgage] arena, and the emphasis toward [mitigating] operational risk is going to continue and require organizations to have a greater level of due diligence and control over their businesses.

It seems that you are weaving ISSM together so that you get a number of checks and balances, backstops and redundancies -- so that there aren’t unforeseen holes through which these risky practices might fall.

The beauty of ISSM is that it's very nimble and very malleable. We can assign responsibilities at an attribute level for control, which allows people to contribute, and then it allows them to have a sharing-of-power strategy, if you will, for security.

It's that cohesion that we bring to the table. How they intersect with one another, and how we have common workflows developed for the process in an organization gives the client a sense that we are paying attention to the entire continuum of continuity of business.

Businesses are run on technology, and technologies require security and continuity of operations. So, we understand that this is a moving target.

Listen to the podcast. Read a full transcript. Sponsor: Hewlett-Packard.

BriefingsDirect Insights podcast examines WOA-SOA continuum with keen eye on cloud computing

Listen to the podcast. Download the podcast. Read a full transcript.

There's been welling interest and discussion lately around so-called Web Oriented Architecture (WOA) and established Services Oriented Architecture (SOA), and how the two relate. And then there's the whole cloud computing trend, and well ... how does that relate, too?

So I gathered a panel of noted IT analysts for a BriefingsDirect podcast discussion, moderated by myself, to delve into the topic even more deeply. We came up with some gems, and perhaps moved the needle forward on understanding these fascinating issues.

But let's back up a bit. The recent chapter of the WOA story began with some blogs and research that concluded that SOA was not a barnstorming trend, and that perhaps WOA was more of interest to many service developers and line of business entrepreneurs inside and outside of enterprises.

That lead to more discussion on WOA as a superset of SOA, and how SOA may need WOA to accelerate its adoption. And, of course, there's been Google App Engine, Microsoft Live Mesh, and the Salesforce.com-Google Apps synergy to chew over.

Then last week, StrikeIron CEO Dave Linthicum presented a podcast on some of the powerful points of the discussion, and Dion Hinchcliffe, founder and CTO of Hinchcliffe & Co., has been posting, micro-blogging and lecturing on the subject for much of the past two weeks. Those discussion points brings us up to the latest BriefingsDirect Insights Edition podcast, Vol. 28.

In this episode, recorded April 24, 2008, we're joined by Jim Kobielus, senior analyst at Forrester Research; Joe McKendrick, an independent analyst and ZDNet blogger; Tony Baer, principal at OnStrategies and blogger; Brad Shimmin, principal analyst at Current Analysis, and Phil Wainewright, independent analyst, director at Procullux Ventures and ZDNet SaaS blogger.

I'll be delivering a transcript on the chat as well, but the topic is fresh enough to run with the audio-only content now. Let us know, did you learn anything or develop any keener understanding about WOA and SOA from this podcast?

Listen to the podcast. Download the podcast. Read a full transcript.

Tidal Software launches intelligent reporting for Enterprise Scheduler

Tidal Software has announced a reporting product that gives enterprises insight into the functioning of their job scheduling environment, enabling IT and line-of-business personal to make quick decisions in both IT and business environments.

Tidal Intelligent Reporting, designed to support Tidal Enterprise Scheduler, aggregates all the scheduler's metrics into a data warehouse and can combine information from multiple job scheduling environments, giving users an enterprise-wide view. This allows personnel to view performance across multiple sites and have access to comparisons between development, test, and production environments.

The product includes ready-to-run reports on production day status, job history, user activity, and audit reports. Users can customize these and create new reports to meet differing needs.

Automatic scheduling allows reports to run automatically, and users can view them in a browser or choose to deliver them to a PDF document, an Excel spreadsheet, or a Word document. Users also have access to a report editor to modify and customize report views.

Security features include the ability to enact fine-grained authorization, segregating specific reports and views depending on an individual user's need to access certain information.

Last fall, I had the pleasure of participating in a live discussion on IT and SOA management at the Harvard Club of Boston with Jason Bloomberg, managing partner at analyst firm ZapThink. Moderating the discussion was Martin Milani, chief technology officer at Tidal Software, which sponsored the luncheon event. Jason and I explored how IT management will evolve in the world of service-based applications. [Disclosure: Tidal Software is a sponsor of BriefingsDirect podcasts.]

The discussion delved into issues of new standards, how SOA demands that performance management and change management should augment and elevate the role of systems management, and on how the integrity of services delivery requires a deep and wide approach to "management in total" across a service's lifecycle. You can listen to the podcast here and view a complete transcript here.

Intelligent Reporting is currently available and supports version 5.3 and higher of Tidal Enterprise Scheduler.

'Enough with WOA, stick to SOA,' say IT architects -- I say drop WOA and SOA

Mike Meehan at SearchSOA.com has done some homework on the use of Web Oriented Architecture (WOA), and the IT folks in the field are fed up. Enough with the labels, they seem to be saying.

And they raise excellent points. I for one am by no means wed to the "WOA" nomenclature. Several other industry analysts recently told me as much -- "WOA is not the right term" -- during a dinner at the IBM Impact event earlier this month.

So what really counts is the concept of not waiting for legacy-abstracted, middleware-driven, investments-heavy SOA before seeking wider berth for more easily available and ecumenical services-based productivity. WOA is about lightweight and externally and internally originating standards-based services and independent data being used now, not after an internal SOA infrastructure is ready (and for some that's five years).

You know the drill: Build it and the services will come, so ramp up on that registry/repository, BPEL engine, scalable middleware beyond EAI, SOAP and XML appliances, additional performance management tier, ESB, federated ESB, data services tier (and another 15 acronyms there), SCA/SDO support, Windows Communication Foundation hooks, and so on.

All of these can be powerful and necessary, but there are multiple tracks to services and business processes flexibility. And some of them are ready now, are cheap and even free, and they are driving a lot of innovation in the field. And some do not require all that much input from IT.

So, true, WOA, isn't an architecture, it's a webby style of apps and integration, of mashups and open APIs, of using REST and RIA clients, all from a variety of Internet sources. It's integration as a service, too. These can all be composited, accessed and managed by an enterprise's internal SOA, or not. The services can come from a cloud, public or private.

These webby assets could just as well come together as portals, standalone Web apps, SaaS, or RIA front ends for composited ecology services that support extended enterprise processes. The point is there's no need to wait.

So WOA as a term does help break out of the box in terms of thinking about SOA as more than "the long journey" that can pay off in years after taking years to develop. Some vendors would have you believe that SOA only happens after a PO is issued for their products.

I also think there's more grassroots political support for webby apps/services inside of sales, marketing, procurement, and line of business departments in many enterprises. They don't know they want SOA, but they may know they want what they see on the Web, and from startups, and from their personal use. They want to use tools they can understand, that help them reach customers and suppliers, by gaining productivity by doing a Web search and signing up to build or access a useful service.

We are now, and this week in particular at the Web 2.0 Expo, seeing rapid ramp-up of services hybrids -- of public/private clouds, services ecologies, internal and external hosting, social enterprise media tools, mashups in myriad forms, integration of services regardless of origins or types of aggregation.

You can today begin a business online and scale it without an IT department, or an on-premises datacenter. You just can.

These concepts are different from what most think of SOA. And if all of this is SOA, then SOA loses it's meaning. By meaning too much, SOA means nothing. And SOA as a term has never been easy for a lot of people to get comfortable with, in the first place.

The fact is that the definitions of and distinctions between applications, platforms, services, tools, clouds, portals, integration, middleware are -- all up for grabs. IT as a concept is up for grabs. The shifts in the software arena at that disruptive. It's why Microsoft is seeking to buy Yahoo, and not Oracle.

I'll bet if Mike Meehan interviewed some sales executives, marketing managers, business analysts, entrepreneurs, and human resources directors -- they might say they cotton to WOA and what it means, more than to SOA and what they don't yet understand it to mean.

This is my point: SOA as nomenclature is not cutting it outside of the IT department. And perhaps some other phrases and/or value propositions would better describe than WOA the innovation now taking place.

Perhaps we need to drop any reference to architecture, and reference the payoffs -- better online work done quickly and cheaply. Perhaps we should call is SWA -- services without architecture, and be done with enterprise architecture all together (as Dave Linthicum boldly suggested recently).

Perhaps it's best not to call what's going on anything at all, and just do it. And that includes dumping "SOA" as a name. So I'm for dropping WOA, but let's be really honest and drop "SOA" too.

Kapow's Web-to-spreadsheet data service helps enterprises exploit cloud-based mashups

Kapow Technologies at the Web 2.0 Expo this week will aim to solve one of the biggest problems facing enterprises as they seek to solve external-internal data chaos by leveraging cloud-based data management services.

With Kapow OnDemand, a cloud-based service that uses the company's Mashup Server, Kapow will provide the ability to create data-rich mashups in minutes and then make that Web data ready for delivery into ubiquitous internal Microsoft Excel spreadsheets, or other enterprise applications and integration infrastructure.

Kapow OnDemand offers users access to a visual scripting environment for building the services and feeds that automates the access and delivery of web-based intelligence and data -- then delivers it the desktop or application of choice. According to Kapow, even Web-savvy, non-technical users will be able to build "robots" in a matter of minutes that can extract, transform, and output Web data.

The hosted service may provide the fastest way to deliver real-time data from the Web into Excel spreadsheets, and therefore into the hands of business analysts, business processes and for internal publishing feeds and streams. This will circumvent the old cut-and-paste logjam and allow analysts to rapidly collect market data on such things as competitive pricing, product mix analysis, or financial metrics, for example.

Despite a huge and growing amount of "webby" online data and content, capturing and defining that data and then making it available to users and processes has proven difficult, due to differing formats and data structures. The usual recourse is manual intervention, and oftentimes cut-and-paste chores. IT departments are not too keen on such chores.

But Kapow's OnDemand approach provides access to the underlying data sources and services to be mashed up and uses a Robot Designer to construct custom Web harvesting feeds and services in a flexible role-based execution runtime. Additionally, associated tools allow for monitoring and managing a portfolio of services and feeds, all as a service.

Deployed on a commercial-grade grid computing environment, OnDemand offers tight security, load balancing, high availability, failover, and automated backup and restore. Pricing for the service will begin at $3,400 per month.

Kapow this week will also announce its Connector for Excel, which allows spreadsheet users to find and execute Web services. By using Kapow OnDemand or the Kapow Mashup Server Web 2.0 Edition along with Connector for Excel, these users can bring XML content and Web services directly into their spreadsheets.

Kapow will offer a product preview Webinar on April 29, covering both OnDemand and the Excel Connector.

Last January, I sat down for a sponsored podcast with Kapow CTO Stefan Andreasan. He explained how much of the potentially useful data on the Internet exists in a form that is designed to be easily read by humans, and not by enterprise applications. [Disclosure: Kapow is a sponsor of BriefingsDirect podcasts.]

There's is a third group, which I call intelligence data. That's hard to find, but gives you that extra insight, extra intelligence, to let you draw a conclusion which is different from -- and hopefully better than -- your competitors. That’s data that’s probably not accessible in any standard way, but will be accessible on the Web in a browser. This is exactly what our product does. It allows you to turn any Web-based data into standard format, so you can access what I call intelligence data in a standard fashion.

Joe Keller, Kapow's chief marketing officer, explained to Computerworld the significance of the new OnDemand service:

By connecting [Web mashups] to Excel, users can have real-time data inside their spreadsheets along with their corporate data to get that 360-degree view of the data they are analyzing. If users can build spreadsheets, if they can do the programming of those spreadsheets, the plug-in makes [mashups] a native element inside of Excel.

Mashups provide that layer we need to really let the business do a lot of the work themselves. It still governs the services and creates the services, but it allows the business start doing business themselves.

Last month, Kapow raised another $11.6 million from investors, including Steamboat Ventures, Kennet Partners, and NorthCap Partners.

This service and the means to sidestep IT (in a good way) so that line of business decision-makers can avail themselves of all the data they can, regardless of its origins, begins the path toward solving the data management mess most enterprises are in. I expect to see many variations on this theme, with data access growing richer and varied -- but also with access and security controls.

As enterprises grasp the productivity that comes with public cloud data management, it may well spur them to bring more of their own data into the services layer where it can be delivered to where it brings the most value.

Open source SOA infrastructure project CXF elevated to full Apache status

After community incubation and development for nearly two years, the Apache CXF open-source SOA and middleware interoperability framework evolved last week into a full project of the Apache Software Foundation.

CXF, with some 60,000 downloads since July 2007, takes its place alongside 60 other Apache projects. The framework began its life as Celtix, which was supported by IONA Technolgies in the ObjectWeb community, and then merged with XFire from Codehaus. It was later moved to the Apache incubator process.

CXF's graduation from incubator to project status involved widespread developer collaboration, taking it through six releases. CXF is now ranked among the top 10 Java software projects, receiving support from the Mule and JBoss communities.

It also serves as the foundation for IONA' FUSE Services Framework. Dan Kulp, IONA's principal engineer has been designated as the CXF project management committee chair. [Disclosure: IONA is a sponsor of BriefingsDirect podcasts.]

Nearly a year ago, I sat down with Kulp for a podcast on Apache and CXF. Here's what he had to say:

CXF is really designed for high performance, kind of like a request-response style of interaction for one way, asynchronous messaging, and things like that. But it’s really designed for taking data in from a variety of transports and message formats, such as SOAP or just raw XML. If you bring in the Apache Yoko project, we have CORBA objects coming in off the wire. It basically processes them through the system as quickly as possible with very little memory and processing overhead. We can get it to the final destination of where that data is supposed to be, whether it’s off to another service or a user-developed code, whether it’s in JavaScript or JAX-WS/JAXB code.

That’s the goal of what the CXF runtime is -- just get that data into the form that the service needs, no matter where it came from and what format it came from in, and do that as quickly as possible.

You can listen to the podcast here and read a full transcript here. IONA recently told fellow ZDNet blogger Paula Rooney that it intends to continue to invest in and support open source activities. And IONA is increasing its role in Apache.

As we now explore the fascinating intersection of SOA and WOA -- with on-premises services and cloud-based resources (including data) supporting ecologies of extended enterprises business processes -- I expect open source projects such as CXF to play a major role.Creating federated relationships between private and public clouds and their services and resources requires more than just industry standards. It requires visibility and access, the type that comes from open source communities and open use licenses.

I expect that open source code-based services and infrastructure will be the preferred choice for building the layers of an extended enterprise service ecology that binds organizations while protecting their assets and interests -- and which allows for trust and cooperation.

In a sense, open source SOA software is ready-made for extra-cloud oriented business processes and relationships. Perhaps one of the supporters of these projects will become a cloud host for integration as a service services?

TIBCO's ActiveMatrix earns 'Product of Year' nod from SearchSOA.com judges

TIBCO Software came home this week with a gold star when SearchSOA.com named its ActiveMatrix SOA platform a "Product of the Year" in the assembly and integration category in SearchSOA.com's annual awards.

The award was sweeter for the Palo Alto, Calif., company because it was based on products released before TIBCO has announced a beefed-up version of ActiveMatrix 2.0, adding new functions and a new enterprise service bus (ESB).

ActiveMatrix provides a single platform for developing, deploying and managing heterogeneous SOA. It includes capabilities for service integration, composite application development and governance. Expect more in the service performance management space from TIBCO soon.

TIBCO claims that customers using version 2.0 can achieve additional business process productivity gains, and can lower total cost of ownership compared to alternative approaches.

One member of the SearchSOA judging panel explained why ActiveMatrix got a top spot:

TIBCO has pushed the envelope with grid architecture here. It definitely helps in terms of achieving technology independence and it gives users a service platform that should be easier to scale. Most times you see "platforms" that lack any central organizing technology. This has it and it should enable users to deploy the functionality they need only when they need it.

I've been following TIBCO's upward trajectory for some time and, more than a year ago, produced a BriefingsDirect SOA Insights Edition podcast devoted largely to TIBCO and ActiveMatrix. You can listen to the podcast here.

Last year, TIBCO architect Paul Brown joined me for a sponsored book review podcast on the concept of Total Architecture, which ActiveMatrix 2.0 undergirds. Read a full transcript of the discussion.

[Disclosure: TIBCO has been a sponsor of my BriefingsDirect podcasts. I have also been a reviewing judge for SearchSOA.com product rankings.]