The title of the classic 60’s Broadway play “A funny thing happened on the way to the forum” kept popping into my head as the evaluation results smacked me in the head and led me in an entirely new direction.

We’ve just finished phase one of our in depth evaluations of leading online communities in the IT industry. Phase two launches this week, which focuses on providing insights and recommendations to siteIQ clients.

As I started the insights and strategy portion of this research, the title of the classic 60’s Broadway play “A funny thing happened on the way to the forum” kept popping into my head as the evaluation results smacked me in the head and led me in an entirely new direction.

The Rosetta Stone turned out to be a close look at metrics that show how leading communities are structured and organized, and how effectively they address the needs of four key audiences: non-technical managers & purchase influencers, partners, support users, and developers. When I looked at these results, it became clear that companies are going in two directions.

Some are building (or expanding existing) communities to target classic technical audiences, and are, by and large, looking for payoffs in lower support costs. But others, like IBM, are trying to break new ground through a new generation of brand and social media marketing communities. A third group—and arguably the largest—reminds me of the blind men and the elephant. They have a firm grasp on the tail but they sense there’s some really big ears somewhere. Their inability to see the big picture is creating communities that behave somewhere between a practicing schizophrenic and a “frankensite.”

For my money, the brand and social media marketing activities and strategies are the most interesting for a couple of reasons. First, these communities focus on creating and nurturing brand loyalty (product, strategy, vision, company) — and generating new revenues by using the voice of the corporation & the voice of the customer to woo and win over purchase influencers and prospects. Second, defining and documenting the payoffs is going to be tricky – and failing to do so could be a career stopper for many current advocates.

Oh, and there’s a third one. You can build brand and social media marketing communities all day long – but manning them and making them meaningful requires a HUGE shift in the company’s culture. From what I’ve seen thus far, I don’t think that any of the companies in our study get this (yet).

What’s your opinion about new generation  brand and social media marketing communities? Sound off in the comments below.

When we decided to take a close look at leading IT communities, the conventional wisdom went like this.

Sun.com, Dell.com, and Oracle.com would rank at the top of the charts, HP.com would fall somewhere in the middle, and most newcomers (like Symantec Connect, EMC, Citrix and the like) would fall to the bottom of the list.

We weren’t quite sure where IBM.com’s newer communities would fall, but knowing IBM, we knew there would be a twist in the story.

From our perspective, the thinking was perfectly logical.

• Sun and Dell pioneered communities on the IT Web, and they’ve had years to flesh out their content & capabilities —and learn from early mistakes.
• Oracle has long been a leader in developing community award programs and its Mix community has been an interesting trendsetter for the past 18 months.
• HP.com’s massive industry footprint and strong consumer roots would surely deliver at least a moderately effective catalog of communities.
• IBM always measures twice and cuts once before making these kinds of bold online commitments – and then throws in a ringer to create a bit of competitive chaos.

As for the rest, we figured that most of these communities would be crafted out of murky objectives or be “Franken-communities” lashed together out of pieces and parts observed on other sites.

When the results were in, conventional wisdom turned out to be dead wrong.

• SAP.com and Intel.com — both built on totally different platform approaches — tie for first place, proving that architecture isn’t the key to building and operating world-class communities.

• Sun.com ranks second overall based on its massive size, but delivers truly dreadful usability results.

• Symantec.com and Dell.com round out the top five with better than average performances and interesting lessons worth learning.

And what about the industry’s largest players — IBM.com and HP.com? Well color us amazed. When 429 criteria were evaluated and 35 usability issues were considered, IBM.com’s over-engineered offering lands in seventh place – but delivers an important paradigm shift that shouldn’t be missed (the twist) — and HP.com’s current community chaos ranks dead last.