A logistics and shipping carol: How online retailers Alibris and QVC ramp up for holiday peak delivery

Listen to the podcast. Or read a full transcript. Sponsor: UPS.

Santa used to get months to check his list and prepare for peak season, but for online and television retailers such as Alibris and QVC they need to take orders the make deliveries in a matter of days. The volume and complexity of online shipping and logistics continues to skyrocket, even as customer expectations grow more exacting. Shoppers routinely place gift orders on Dec. 21, and expect the goods on the stoop two days later.

For global shopping network QVC, the task amounts to a record peak of 870,000 orders taken in a single day -- more than three times typical volume. For rare book and media seller Alibris, the task requires working precisely across ecologies of sellers and distributors. For partners like UPS, the logistics feat requires a huge undertaking that spans the globe and requires a technology integration capability with little room for error at record-breaking paces.

Listen as innovative retailers Alibris and QVC explain how they deal with huge demands on their systems and processes to meet the holiday peak season fulfillment. One wonders how they do it without elves, reindeer, or magic.

Join Mark Nason, vice president of operations at Alibris, and Andy Quay, vice president of outbound transportation at QVC, as we hear how the online peak season comes together in this sponsored podcast moderated by Dana Gardner, president and principal analyst at Interarbor Solutions.

Here are some excerpts:

What we strive for [at Alibris] is a consistent customer experience. Through the online order process, shoppers have come to expect a routine that is reliable, accurate, timely, and customer-centric. For us to do that internally it means that we prepare for this season throughout the year. The same challenges that we have are just intensified during this holiday time-period.

Alibris has books you thought you would never find. These are books, music, movies, things in the secondary market with much more variety, and that aren’t necessarily found in your local new bookseller or local media store.

We aggregate -- through the use of technology -- the selection of thousands of sellers worldwide. That allows sellers to list things and standardize what they have in their store through the use of a central catalogue, and allows customers to find what they're looking for when it comes to a book or title on some subject that isn’t readily available through their local new books store or media seller.

You hit on the term we use a lot -- and that is "managing" the complexity of the arrangement. We have to be sure there is bandwidth available. It’s not just staffing and workstations per se. The technology behind it has to handle the workload on the website, and through to our service partners, which we call our B2B partners. Their volume increases as well.

So all the file sizes, if you will, during the transfer processes are larger, and there is just more for everybody to do. That bandwidth has to be available, and it has to be fully functional at the smaller size, in order for it to function in its larger form. ... These are all issues we are sensitive to, when it comes to informing our carriers and other suppliers that we rely on, by giving them estimates of what we expect our volume to be. It gives them the lead-time they need to have capacity there for us.

Integration is the key, and by that I mean the features of service that they provide. It’s not simply transportation, it’s the trackability, it’s scaling; both on the volume side, but also in allowing us to give the customer information about the order, when it will be there, or any exceptions. They're an extension of Alibris in terms of what the customer sees for the end-to-end transaction.

[For QVC] peak season 20 some years ago was nothing compared to what we are dealing with now. This has been an evolutionary process as our business has grown and become accepted by consumers across the country. More recently we’ve been able to develop with our website as well, which really augments our live television shows.

... In our first year in business, in December, 1986 -- and I still have the actual report, believe it or not -- we shipped 14,600 some-odd packages. We are currently shipping probably 350,000 to 450,000 packages a day at this point. We've come a long way. We actually set a record this year by taking more than 870,000 orders in a 24-hour period on Nov. 11. This led to our typical busy season through the Thanksgiving holiday to the December Christmas season. We'll be shipping right up to Friday, Dec. 21 for delivery on Christmas.

We’ve been seeing customer expectations get higher every year. More people are becoming familiar with this form of ordering, whether through the web or over the telephone. It's as close to a [just-in-time supply chain for retail] as you can get it. As I sometimes say, it's "just-out-of-time"! We do certainly try for a quick turnaround.

The planning for this allows the supply chain to be very quick. We are like television broadcasts. We literally are scripting the show 24-hours in advance. So we can be very opportunistic. If we have a hot product, we can get it on the air very quickly and not have to worry about necessarily supplying 300 brick-and-mortar stores. Our turnaround time can be blindingly quick, depending upon how fast we can get the inventory into one of our distribution centers.

We carefully plan leading up to the peak season we're in now. We literally begin planning this in June for what takes place during the holidays -- right up to Christmas Day. We work very closely with UPS and their network planners, both ground and air, to ensure cost-efficient delivery to the customer. We actually sort packages for air shipments, during critical business periods, to optimize the UPS network.

Listen to the podcast. Or read a full transcript. Sponsor: UPS.

ELC Technologies and FiveRuns join forces for Rails development

ELC Technologies and FiveRuns Corp. are joining forces in a strategic partnership designed to offer enterprises a broad scope of resources for Ruby on Rails, the Web application framework.

Both ELC Technologies, Santa Barbara, CA, and FiveRuns, Austin, TX, already have a strong presence in the Rails market. We're also glad to see RESTful Ruby.

ELC Technologies specializes in Rails-based business applications and agile software development practices. It counts such companies as Buy.com, Cisco, Live Nation, MediaTrust and TuneCore among its client base.

FiveRuns delivers tools that allow IT managers to monitor the performance of Rails applications and their underlying infrastructure in production environments.

The two companies are collaborating on enterprise Rails deployments, which will be announced at a later date.

Jonathan Siegel, founder and president of ELC Technologies, explained the rationale for the new partnership.

We have repeatedly demonstrated the value of Rails for business-critical applications to the global companies we have as clients. However, one of the greatest challenges our clients face is monitoring and maintaining Rails within large-scale enterprise environments. Working with FiveRuns will allow our clients to easily manage their Rails deployments using FiveRuns' tools--and to demonstrate for themselves that Rails can deliver enterprise performance as well as shortening time to deployment.

I'm seeing a lot of enthusiasm for Ruby on Rails in the enterprise, and it's beginning to pull out of the exotic niche category into more mainstream RAD use, as fellow ZDNet bogger Joe McKendrick points out.

Wind River's John Bruggeman on Google Android and the advent of mobile internet devices

Listen to the podcast. Or read a full transcript.

The Android open source mobile platform made a splash in October when Google announced it, along with the Open Handset Alliance (OHA). An Android software development kit (SDK) came on Nov. 12, and the first Android-based open source platform mobile phones are expected in mid-2008.

The impact of such a platform on mobile phones and carriers has been roundly debated, yet the implications for an entirely new class of mobile internet devices has received less attention.

In this podcast, John Bruggeman, chief marketing officer of Linux software provider Wind River Systems, digs in to the technical, business model and open source implications of Android and OHA -- but he goes a step further.

Android will lead, he says, to a new class of potentially free mobile internet devices (MIDs) that do everything a PC does, only smaller, cheaper and in tune with global mobile markets that favor phones over PCs for web connectivity. [Disclosure: Wind River has been a sponsor of BriefingsDirect podcasts.]

Listen as I interview Bruggeman on the long-term disruptions that may emerge from the advent of Android.

Here are some excerpts from our discussion:

What’s new [in Android] is the business models that open up, and the new opportunities. That’s going to fundamentally change the underlying fabric of the mobile phone space and it’s going to challenge the traditional operators' or carriers' positions in the market. It’s going to force them, as the supply chain, to address this. ... Carriers potentially are going to have to embrace completely new revenue and service models in order to survive or prosper.

Clearly, the great promise of the Google phone platform is aimed more at an ISP mentality, where they make money on how we provision or enable new services or applications. ... the traditional carrier is a more connection-based business model. You pay for connection. This model will clearly evolve to be some sort of internet model, which today is typically an ad revenue-share model. That’s how I see OHA will play out over time. We’re going to have to adopt or embrace an ad revenue-share model.

There might be revenue that’s derived through connectivity, but increasingly we're seeing the big money around the monetization of advertising attached to search, advertising attached to specific content, and advertising attached increasingly to mobile location and presence.

I don’t think that the extreme is that improbable, that the actual connection price would go down to zero. I could have a mobile phone and pay a $0 monthly fee. ... The ad revenue is where the real dollars are here, as well as all the location-based value that you can do. This is the true delivery on the promise of the one-to-one marketer's dream. You’ve got your phone. I know exactly where you are.

It would be naïve to say the technology issues are completely solved, but I think a lot of the hard problems are understood, and there is a path to solution. Those will play out over the next 12 months. I see a clear road to success on the technology side. It will be easier for the technologists to overcome the obstacles than it will be for the business people to overcome the new models in an open source world.

There’s going to be a lot of pressure to drive down that connectivity price really quickly. I say that because I think you can’t ignore the overtones of Google being willing to buy their own bandwidth and become their own carrier. That threat is out there. As a carrier, I've either got to embrace or fight -- and embrace seems most logical to me.

These devices, the converged mobile device in particular -- something like an iPhone -- strikes me as a stepping stone between a traditional PC, as we know it, and some of these mobile devices.

If I can get a lot of what I get through the PC free or low-cost through one of these mobile devices, the only real difference is the size of the monitor, keyboard, and mouse. Isn’t there an opportunity in two, three, or four years that I might say, “I don’t need that PC and all that complexity, cost and so forth. I might just use my mobile device for almost all of the things I do online?"

PC manufacturers and those that are the traditional part of that supply chain are threatened by that every day now. You've hit it on the head. There’s an emerging market. Maybe the most important technology market to observe right now is the mobile Internet device (MID).

Many analysts are starting to pick up on it, and it could be viewed as the next generation of the mobile phone. But I think that’s underselling the real opportunity. If you look on the dashboard of your automobile, the back of your airplane seat, everywhere you go and everything you touch, it is a potential resting place for a MID with a 4x6 screen or a 3x5 screen, or all different kinds of form factors. That kind og use gives you the experience that is the eventual promise of the Android platform.

We all should start thinking about and talking about the MID market pretty quickly. ... The pie that we're defining isn’t really just mobile internet or voice, presence, and mobile commerce. It’s really the whole internet.

The first thing is we need to get some Android-based phones out there. Some time next year, you're going to see the first phones and that’s when we're actually going to have to see the operators who offer those phones address all the business model issues that you and I've have been talking about today.

So the next big step is that it’s got to move from the talk about to the reality of "here are the phones," and now we're going to have to resolve all these issues that are out there. That's not years away -- that’s next year.

Listen to the podcast. Or read a full transcript.

Red Hat unveils JBoss Developer Studio -- is it destined for an Amazon or IBM cloud?

Red Hat, Inc., Raleigh, NC, has finally released JBoss Developer Studio, an open-source Eclipse -based integrated development environment (IDE) that combines tooling with runtime.

Red Hat released the beta version for free download on JBoss.org last August and said that the final subscription version would be available "later this summer." Since the beta was made available, according to Red Hat, there have been over 50,000 downloads.

Designed to allow enterprises to be more agile and to respond more quickly to changing business requirements, Developer Studio eliminates the need to assemble IDEs. It's built on the Eclipse-based developer tools contributed to Red Hat by Exadel in March and introduced under open source in June, The Exadel products contributed to the project included Exadel Studio Pro, RichFaces, and Ajax4jsf.

JBoss Developer Studio incorporates Eclipse tooling, integrated JBoss Entrperise Application Platform, Red Hat Enterprise Linux for development use and full access to Red Hat Network. Also included are tooling for technologies, including JavaEE, JBoss Seam, Ajax, Hibernate, Persistence, JBoss jPBM, Struts, and Spring IDE.

Developer studio is available by subscription for $99.

On a modestly related note ...

You'll recall that Red Hat also made news when it announced its runtime-as-a-service on the Amazon EC2 cloud. Now ... I wonder, perhaps these tools could emerge as an IDE as a service placed up on Amazon, to deploy to RHEL runtime instances. Wow, could make a very cool combo.

Folks like Coghead and Bungee Labs are already making waves with development and deployment as a service. And Amazon ought to bringing tools -- not just platform -- through it's pay-as-you-drink hosting offerings sometime soon. Genuitec certainly has its eyes on this model, and is well-placed for it with MyEclipse.

So who will be the one to move their tools environment to the Amazon cloud first? Perhaps Amazon will offer several tools options, such as one for web apps and mashups, and another (or two) for Java development? You know you want to, Jeff.

I'd like to see a way for .NET developers to get such a Visual Studio-as-a-service (or open source Mono thing equivalent) up on Amazon EC2, to then force Microsoft Live to follow suit. Who needs tools licenses anymore then?

How about the IBM cloud? What tools might go well there? Can Sun Microsystems resist following suit with NetBeans-as-a-service? A New Hope.

Tools in the clouds. Luke, it is your destiny!

Federated ESBs come to fore as natural outcome of guerrilla SOA practices

Some IONA Technologies announcements today point up the growing practice of multiple ESBs within enterprises, often associated in a federated manner, and sometimes using ESBs tasked with specific types of integration duties.

IONA is taking a "hybrid" approach to ESB offerings, with a coordinated open source and commercial strategy. [Disclosure: IONA has been a sponsor of BriefingsDirect podcasts.] IONA Technical Director Jim Strachan addresses some of the open source issues here. And IONA has also upgraded its Artix ESB, and has partnered to bring a management dashboard benefit to the mix.

These moves reflect how enterprises and service providers are using ESBs in innovative ways, in effect creating distributed ESBs to support SOA, SaaS and guerrilla SOA -- while building a path to holistic SOA that follows a crawl, walk run ramp-up.

Indeed, some new use traits are emerging on how ESBs are actually being used in the market. One is that multiple ESBs are often used, or come into play, rather than one honking ESB swallowing everything up. Sometimes such varied ESB use comes from different SOA projects that evolved using different means to access and integrate resources. Sometimes it's from separate organizations or departments that merged or became partners. SaaS also encourages ESBs at the edge and internally, so there's likely a mix there too.

I'm also seeing instances of ESBs that are tuned or dedicated to specific types of integration, or integration that is "pointed," if you will, in a specific direction. By that I mean integration for data services, unstructured content, or integration for management feeds, or integration from outside partners of supply chains.

An ESB for various flavors of integration makes a ton of sense to me. Deciding later whether to consolidate ESBs, while learning what works best on a more granular level, follows how IT often evolves. It certainly aligns with open source infrastructure use adoption patterns.

Given these scenarios, rather than force an architect to pick or choose one ESB and make it dominant, we just as often now see federation of several ESBs. Due to their nature, this makes sense -- many integration points. So hybrid ESB use makes sense and is reflective of what's going on in actual use. Another aspect is that ESBs are not just federated on equal footing. An ESB can be used in master and slave configurations, where various architectural topologies are likely given the many possible ways that these SOAs emerge. Old and new can play well based on many types of integration means. Think of it as distributed integration.

In this environment, IONA is offering a logical hybrid solution set. On one hand, FUSE allows for the benefits of open source and community development to make ESBs inclusive and standards-based. And the community provides a great way for many connectors and modules to well-up to bring even more assets and resources into play with the ESB. This makes it far easier for esoteric applications and content to play in an SOA, and those connectors are made openly available.

In this open source role for an ESB, Metcalfe's Law (value of network grows with number of participants on it) applies too. The value of the ESB increases with the number and diversity of assets and resources that can attach to it. FUSE aims to exploit this, as well as provide a low-cost and simpler way for developers to enter into ESB use.

On the other hand, in legacy-rich and CORBA environments, the IONA Artix offering binds and integrates core and more traditional messaging and ORB-based assets and resources. So you have a backward-facing and legacy-compatible ESB offering, one that scales to large transactional demands in Artix. And you have the new kids on the block, Web services and SOA greenfield services that can be accessed and organized via FUSE and the Apache community.

Putting FUSE and Artix 5.1 into a federated yet managed configuration then offers the best of many worlds, and gives organizations variety of choices on how to enter and manage the expansion of SOAs, based on their specific situations. And this also mitigates future risk by making unknown scenarios -- including more SaaS use -- easier to meld into the architecture.

IONA's partnering with Hyperic for FUSE HQ broadens the management into mature consoles-based delivery, while also expanding the scope of what is managed. So that makes sense. All in all, an approach to the market that makes market adoption and inclusion more in tune with guerrilla SOA than master plan SOA based on one vendor or one product set.

In other SOA news today -- again with an open source angle -- WSO2 announced an open-source approach to help users pass consistent identity data across networks, while protecting them from such things as phishing and other identity attacks in Web applications.

Web services middleware provider WSO2, based in Colombo, Sri Lanka and Mountain View, Calif., recognizes that as SOA becomes more prevalent and more complex -- linking data, applications and service both inside and outside the enterprise -- that security and authentication become prime concerns. [Disclosure: WsO2 has been a sponsor of BriefingsDirect podcasts.]

The WSO2 Identity Solution (IS) is based on Microsoft's CardSpace technology, which is built on the open standards Security Assertion Markup Language (SAML) and WS-Trust. WSO2 IS will operate with CardSpace components from multiple vendors.

It also works with current enterprise identity directories, such as those based on the Lightweight Directory Access Protocol (LDAP) and Microsoft Active Directory.

WSO2 IS has two primary components: Identity Provider and Relying Party Component Set. Identity Provider allows users to issue "cards," both managed information cards and self-issued cards that allow users to log into any Web application that supports CardSpace. Web sites that rely on this authentication don't need to store any passwords or other personal details.

Key features of Identity Provider include:

• User store support for the most common directories that offer standard LDAP or Java Database Connectivity interfaces. It also includes a built-in user store for smaller companies.
• Claim support for standard and custom claims, so users can keep full control over what personal information is shared.
• Statistics, reporting, and an audit trail, which let administrators monitor user accounts and issuances of information cards.
• Revoking mechanism, which allows administrators to revoke user cards and block them from being used for authentication.

The Relying Party Component, build around the Apache HTTPD module (mod_cspace), plugs into the most common Website servers to add support for CardSpace authentication requests. The module is independent of any server-side Web framework, and it can set up CardSpace authentication with any Web framework running on Apache 2, including PHP, Python, and Perl applications.

Key features of the Relying Party Component include:

• Access control for static content in Apache HTTPD
• An integration interface for developers
• Support for leading content management frameworks, such as Drupal and MediaWiki
• A Java servlet filter to provide an integration point for J2EE-based Web applications.

WSO2 also recently upgraded its open source ESB.

ZoomInfo offers more evidence of a 'controlled circulation' advertising benefit quickly emerging on the web

Get ready for new "controlled circulation" models on the web, ones that target you based not on your preferences for music or soft drinks -- but on what you consume in your occupation. Think of it as B2B social networking.

First, some set-up ... One of the great media inventions of the mid-20th century was the notion of affinity-based, controlled circulation publishing. Those creating magazine titles that catered to defined groups -- rather than mass media volume plays like network television -- went granular.

By focusing on concentrated audiences, these publishers walled up "universes" of buyers that passionately sought specific information as defined by discrete hobbies or occupations. Bill Ziff Jr. honed in on the hobbies, and grew a media empire on titles that linked up dedicated buyers -- of things like electronics kits, models, automobiles (and the jackpot, personal computers) -- to the sellers of the actual goods behind the passion. The ads inside these special interest pubs generated high premiums, based on the tight match between engaged (and well monied) buyers and drooling sellers.

Norm Cahners took the model in the direction of industrial business niches. He provided free monthly magazines based on slices of industrial minutiae that delivered useful albeit dry information to those specifiers of myriad corporate goods and services. You order gizmos for your buggy whips? You probably spend millions of dollars on procurement per each kind of good per year. Let me introduce you to some sellers of those goods who want to make you a deal.

The Cahners Publishing magazines -- on things like plastics use, integrated circuits developments, materials handling and design engineering -- were free to readers, as long as those readers identified themselves as corporate decision makers with budget to spend. Again, high ad premiums could be charged by linking engaged readers (with huge annual budgets) with advertisers who needed reach hard-to-find and shifting groups of corporate buyers.

Soon the burgeoning lists of these readers, sliced and diced by buying needs, and sanctified by audit bureaus as valid (mostly), became very, very valuable. As a controlled circulation publisher, if you had the top one of two monthly magazine titles that generated the definitive list of those buying all the industrial values, say, in North America -- you were sitting pretty. You controlled the circulation, defined and refined the audience, and so told the sellers how much they needed to pay you to reach those buyers. You priced high, but still less than these sellers would need to spend to send a warm body carrying a bad into each and every account (on commission).

In effect, the controlled circulation publishers collected straight commissions on billions of dollars in commercial and special interest goods being bought and sold. They were a virtual sales team for all kinds of sellers. Editorial was cheap. Life was good.

And then 10 years ago the Web came along and pretty much began to blow the whole thing apart. Engaged users started using Web search, and explored their vendors' web sites on their own. Vendors could reach users directly, and used their websites as virtual sales forces too. Soon there were wikis that listed all the sellers of goods in certain arenas of goods and services. Those seeking business or hobby information could side-step the editorial middleman and go direct to the buying information on goods and services they wanted. We're only into the opening innings on this, by the way.

But the same disruption that plagues newspapers like the San Jose Mercury News and The Boston Globe -- both of which should be doing great based on their demographic reach -- is undermining the trade media too. It's the web. It's search. It's sidestepping the traditional media as a means to bind buyers and sellers. The web allows the sellers to find the buyers, and the buyers to find the sellers with less friction, less guessing, less cost. Fewer middlemen.

And this means the end of controlled circulation has we have know it. ... Or does it?

Just as the web made has made it a lot harder for media companies to charge a premium for advertisers to reach a defined universe of some sort, the web could also allow for a need breed of controlled circulation, one that generates "universes" on the fly based on special interest search, not based on special interest magazines.

The current web ad model has evolved to be based on blind volume display ads, with the hope of odd click-throughs, usually of less the 0.5 percent of the total banner ads displayed. Advertisers know exactly what their ad dollar gets them, and it's not enough. Even when seekers click on ads, they usually get sent to a home page that was just as easily reached through keyword searches from a web search provide (for free), based on their real interests. Enter Google. And you know the rest.

Why the history lesson? Because we're now beginning to see some new variations on the controlled circulation theme on the web that create additional models. Controlled circulation could be back. It that could mean much bigger ad bucks than web display ads or even key-word-based ads can generate. It's what has Microsoft gaga over Facebook. And News Corp. gaga over MySpace. And Viacom beside itself because it has no such functional base yet.

Controlled circulation is coming to the web on one level via social networks, mostly for consumer goods and services -- sort of what Bill Ziff did for hobbyists in the 1950s and 1960s. Social networks like Facebook and MySpace endear their member users to cough up details about themselves -- just like controlled circulation publishers used to require for readers to get free magazines on specific topics. Based on the need to expose yourself on a social network to get, well ... social ... you therefore provide a lot of demographic details that can then be carved up into the equivalent of controlled circulation universes. Based on your declared consumer wants, fad preferences, age and location, you give advertisers a means to target you.

This model is only just now being probed for its potential, as the Beacon trial-and-error process as Facebook these days attests. Soon, however, an accepted model will emerge for binding consumers and sellers of goods and services, a model better than banner ads, one that can go granular on user preferences (but not too granular, lest privacy bugaboos rear their paranoid heads). When this model is refined, everyone from Microsoft to Yahoo to Google and Time Warner will need to emulate it in some fashion. It will be the third leg on the web ads tool: display, search-based, and now reader-profile constructed controlled circulation.

Which brings me to ZoomInfo. (Disclosure: ZoomInfo has been a sponsor of some BriefingsDirect B2B podcasts and videocasts that I have produced). What's so far missing in all of the Facebook hysteria is the Norm Cahners part, of how to take the emerging controlled circulation web model and apply it to multi-trillion dollar B2B global markets. How to slice and dice all the companies out there with goods and services you -- as a business buyer -- need to know about? Instead of the users giving up profile information on themselves as a way of providing profile-constructed controlled circulation, why not let the companies provide the profiles that the users can access via defined searches based on their actual needs?

Wade Rouch over at Xconomy gives us a glimpse of this model based on what ZoomInfo is now doing with "business demographics" or what Zoom calls Bizographics. This is the B2B side of what social networks are doing on the consumer side, but with a twist. By generating the lists of businesses that provide goods and services sough via a search, and even more lists of the goods themselves, users can educate themselves and the bond between B2B buyers and sellers is made and enriched. All's that's needed is the right kinds of searches that define the universe of providers that users can then explore and engage with.

ZoomInfo is but scratching the surface of what can be an auspicious third (but robust) leg on the B2B web knowledge access stool. By satisfying both seekers and providers of B2B information on business needs, ZoomInfo can generate web page real estate that is sold at the high premiums we used to see in the magazine controlled circulation days. Occupational-based searches for goods, information, insights and ongoing buying activities is creating the new B2B controlled circulation model.

What's more, these defined B2B universes on the fly based on occupations and buying needs amounts to giving more power to the users via what Doc Searls correctly calls Vendor Relationship Management. It's a fascinating concept we'll be seeing a lot more of: Matching buyers and sellers on the web based on their mutual best interests. Mr. buyer, please find Mr. Seller -- on your terms, based on your needs.

More hints that IT systems analysis and on-demand models are coming together

The hot (albeit not necessarily sexy) segment of IT operations -- the analysis and intelligence-gathering from logs and performance management data -- is showing increasing signs of an on-demand future.

First, Paglo came out last month (in beta) with a free and open source (GPL) crawler service that scours the reams of log files and other electronic records users point it at inside of data centers and server farms. With a subsequent index, IT operators can view on and search for needed analysis and metrics of IT use and performance data as an online service via browsers.

Paglo provides IT administrators and operators the free crawler service to gain information or meta data on all sorts of assets on their networks, including across VPNs to remote offices. As an open source crawler, folks are free to write scripts to search into various modules and whatever else they want to gather data from on their networks. Other users can then benefit from these scripts via the community. Pretty quickly the Paglo community ought to be able to index just about anything of import on their networks. No cost incurred for users but their time involved.

The meta data then -- they assure me, safely -- is sent to an index instance in the cloud managed by Paglo. The managers of the crawler and hosted data can then securely search the logs using all sorts of queries, charts, views, and dashboards to gather quantitative and qualitative business intelligence on their IT systems use and use patterns.

The analysis can initially help with such chores as determining how many Microsoft Office suites are actually in use, or how to do quick audits of this or that element on a network. This can help with audits, to identify straggler application installations and to track down when users have installed things they should not. But later, the service could spawn premium services for operations analytics and troubleshooting.

Furthermore, by aggregating and (one hopes) anonymizing the data from many IT sites, Paglo could create definitive market research on just what constitutes IT use and context based on just the facts, ma-am. Rather than rely on quasi-annual surveys by IT analyst firms (always on the vanguard of objective results), a broad Paglo audit of large swaths of IT use and habits -- based on valid and scientific samplings (if not actual empirical censuses) -- could take the guess work out of what IT is actually being used in certain types of companies, and regions. That would be some mighty fine data, and could hold the IT vendors' feet to the fire on their real penetration and use patterns.

But I can see where this can go much further. Views and queries can show exactly what is beng used and in many respects how. Also, Paglo can then aggregate that across many user sites and types of users to draw empirical, statistically relevant determinations of what is being used in the field. Compare and contrast between verticals, SMBs and enterprises, regions and/or geographies.

If Paglo gets any kind of volume adoption and the data is good and comprehensive, we could end up with a comScore for IT components and infrastructure bits. Perhaps Paglo will make its money from selling the use patterns and market share data, while giving away the means to the tactical analysis for each company. So far they are mum on where their remuneration will come from.

Suffice to say, such a service will generate a lot of page views that only an IT systems administrators could love. That in itself could spell advertising gold for those selling to IT shops.

And, hey, free insight into IT ops -- as long as you feel okay about someone else's crawler sniffing around your network and servers -- could be an offer some cheapo outfits can't refuse. If the CIO won't pay for analytics products, what else could an operations manager do to prevent those awful Monday mornings.

On another IT analysis front, LogLogic announced today that longtime IT infrastructure thought leader Pat Sueltz has joined as CEO. Pat has been marching upward in title (while perhaps sliding a bit in employer size) over the past seven years. You may recall Pat as the gal who managed the Java relationship for IBM, back when Sun Microsystems and IBM saw eye to eye, at least on a common foe: Microsoft.

Then Pat went to Sun -- after making a lot of noise at IBM on why Java ought to be overseen by a standards body (if not open source). And this back in mid-1990s. After a stint at Sun in charge of software (not great timing it turns out) and then Sun services, she did a well-timed stint at Salesforce.com. And there lies the rub on the intersection of LogLogic and SaaS and on-demand models.

She won't commit, of course, this being her first week on how on-demand and LogLogic come together. But I'll wager a new chapter of growth potential for LogLogic lies in some of the interesting things Paglo has been trying, not to mention following the Salesforce ecology thing. There's also Splunk and what it has done with an online open repository of analytics data, know as SplunkBase. [Disclosure: Splunk has been a sponsor of BriefingsDirect podcasts.]

Pat comes to LogLogic from SurfControl, where she was CEO. I'll be keeping an eye on Pat, with keen interest on how research, trends, data and online business models come into play with the perhaps no longer esoteric log file management arena. I'm also looking for real business intelligence as applied to IT, culled from this log data. Between those values and the compliance imperatives, this is a high-growth area.

In other words, there's gold in them thar logs.

We'll see more acquisitions that meld telcos with IT vendors

News from London that Deutsche Telekom may make a bid to buy IT services giant EDS. This is only the opening volley in a forthcoming period of acquisitions that meld telcos with IT vendors.

I recently suggested that BEA Systems, as it spurns Oracle's initial bid, may also be a good fit for a large teleco such as AT&T. Much of the same logic I applied to a BEA-telco mashup works for a Deutsche Telekom and EDS marriage.

The fact is that IT vendors -- be they code/systems providers or systems integrators (or both) -- are becoming more like service providers. We see evidence of this with IBM's recent Blue Cloud announcement, the go-to-market match-up between Red Hat and Amazon, and also the way that many new startups are entering the field -- as services -- such as Paglo this week. (Look for a separate blog on Paglo soon.)

The fact also remains that telcos and mobile services providers are increasingly becoming IT providers, either directly or as integrators or aggregators of IT functions that they then deliver to their customers -- both B2B and B2C. Enterprises will enjoy efficiencies in buying business services from a single entity when that organization can combine the IT, network, integration, communications services, outsourcing and software. Who or what best combines these features for the best business-cost benefit, is the $100 billion question.

The value-add to enterprises on IT increasingly comes from the integration, services provisioning and services ecology partnerships, not from the code base or hardware differentiation. Virtualization, open source, and SaaS will hasten this irreversible course. And when everything is a TCP/IP-driven function or asset, why not merge, mash, and package it all up with bright red bow and lock it into a big multi-year services contract?

And, of course, we're now also on the downward slope of a massive IT supplier consolidation era (most notably among software vendors). Some even call it the end of best-of-breed. I'm not sure it's the end of best-of-breed, there will always be standalone functions and/or applications and services that come to markets to meet new needs.

As Peter Zotto, CEO of IONA Technologies, recently told me:

As 'middleware' vendor consolidation continues, big propriety stacks will get bigger, more expensive and more complex-and the speed of innovation will decline. This is the exact opposite of the potential of SOA. "Anti-stack" vendors, like IONA, that deliver industry-standard middleware technology for performance-demanding SOA environments are
already benefiting customers looking for lower-cost and easier-to-deploy software. This is just the beginning of a new innovation cycle kick-started by industry consolidation. (Disclosure, IONA has been a sponsor of my BriefingsDirect podcasts.)

But clearly the larger vendors -- Oracle, IBM, SAP, HP, Microsoft, et al -- have gotten even larger via consolidation, and are closer to providing a full set of IT offerings, with varying degrees of actual deep and meaningful integration. As they become more like service providers these bulked-up vendors actually drive ecologies of ISVs and providers, and -- just like a telco -- manage the customers on one end, and the supply chain participants on the other.

So when you associate and explore the consequences of these trends, it points to more types of mergers along the lines of Deutsche Telekom and EDS, or even BEA and AT&T.

The telcos had better not wait too long as the are buying or being bought. They will eventually be competing with a class of consolidated vendor/suppliers that have traditionally moved more quickly and better than the telcos in their best days. There will only be a handful of these behemoths bestriding the globe (until and if decentralization again appears?).

Indeed, if the telecos wait too long, or make the wrong acquisitions, they might lose that customer relationship altogether. And where would they be then, especially as new networks based on new wireless technologies appear?

One aside: Watch how Cisco Systems moves on this. I predict some interesting mergers involving Cisco and large network/services providers in 2008.