Yesterday, I finished compiling and publishing the data portion of our in depth evaluations of online communities on leading IT Websites. The ten companies under our microscope are IBM, HP, Dell, Symantec, SAP, Intel, Oracle, Sun, EMC and Citrix.

One of the things I love to do during this process is ‘short list’ the benchmark to identify the common elements across sites – and then ‘long list’ the benchmark to tease out which set of companies are pushing on the envelope by providing content, features and capabilities that other companies don’t deliver.  Once I completed this process for communities, a billion dollar question rolled around in my head: Do these companies actually have a “build plan” for their communities – or are they just winging it?  I’m voting for the winging it option.  Here’s why.

When the 490 criteria on the online communities benchmark is ‘short listed’ to include only those items that are common across 60% of the sites, only 55 capabilities (13% of the total) end up on the list. That’s not to say that the online communities we evaluated only do 55 things. Far from it.  The “long list”, which represents features and capabilities provided on 30% or fewer communities, is a whopping 326 items.

These communities dance to the tune of their own drummers

Next, add in the fact that, with the exception of Sun, every community we studied meets less than a third of the criteria on the benchmark, and you have your story. Every community is dancing to the beat of its own drummer.  Instead of using a comprehensive plan, these communities are being built around 55 basics – and they are ‘winging it’ after that. Given this, it’s no wonder that so many of their members appear to be lost and confused.

This brings me to the second thing I learned during my “short” and “long” listing exercises. The amazing power of our communities benchmark.  This benchmark provides a bird’s eye view of what ten leading IT communities actually deliver to their members – and pinpoints the holes in every strategy. As important, it provides a perfect ‘build plan’ for any company that wants to jump into the communities waters. You may not want to do everything on the list – but it’s nice to know what the rules of the game are before you start.