Overall: Oracle.com
Usability & Effectiveness: IBM Software Group
Content, Features & Capabilities: Oracle.com

With two companies performing complete site overhauls–and one nudging its site into place over the past year—rankings were bound to change in 2011. So which sites were the major movers and shakers over the past 12 months? Which ones are the sites to watch through 2012? Here’s the rundown from first place to last:

1. This year Oracle.com retained its title as the content “big foot” on the siteIQ eBusiness Index (1)—and this footprint worked to deliver it the first place ranking. On the other hand, a broader look at the enterprise software industry shows that Oracle.com doesn’t have any room to rest on its laurels. Now it has CA.com gunning to be an enterprise software Goliath and IBM Software is upping the usability bar. To compete in the future, Oracle.com needs to up its game.
2. Incremental change was the name of the game at CA.com. That’s not to say there were any major splashes or “ta da’s”. Just a series of re-launches and endless tweaking that ended up creating a noticeable transformation. On the other hand, one look at CA.com’s scores and rankings shows that change isn’t always a good thing. CA.com lost its #1 usability ranking (dropped to third)—and is the only site that registered a net decrease in its usability & effectiveness score.
3. IBM Software’s latest site redesign is part of IBM.com’s Centennial revamp that is trying to change all of the rules. Here, The IBM Software team’s hard work—and CA.com’s tumble—delivered it the first place usability & effectiveness ranking and the only Good Practice rating in the group. The site’s renovation also delivered more content & features—but the real story is the site’s impressive string of 13 “Good Practice” usability ratings. The fact that its online communities are now visible on the site didn’t hurt either.
4. We didn’t add Symantec.com to the enterprise software mix this year because it’s selling large, complex software solutions. We added it because the enterprise systems “big wigs” are continuing to move down market with SMB and SaaS solutions that rely on the kinds of marketing and eCommerce capabilities that have long been a Symantec.com forte. So how did Symantec.com fare when compared to its mega software brethren? On the one hand, there’s no doubt that Symantec’s content & features are the size of a newt (relatively speaking). On the other hand, Symantec.com proves that small can be mighty. It usability & effectiveness debuted at second place—which proves that it knows how to capitalize on every single asset.
5. In April the German giant SAP.com unveiled a ginormous make-over that is still a work in progress. But if early scores are any indication, this new site is likely to give other sites a run for their money. Within a month of launch, this partial implementation was already moving the site’s content and usability & effectiveness scoring needles in the right direction. It ranks fifth to be sure—but its strong showing tells us that SAP.com will be a serious competitor within the next 12 months.
6. SAS.com’s scores paint a picture of a site that is running hard to stay in place. Here, substantial corporate marketing-class additions and improved communities visibility delivered the largest increase in overall content, features & capabilities scores—but burning the content midnight oil didn’t move its usability & effectiveness scores one whit. It ends up in 6th place (that’s dead last) with a 50.96% score and two stars (below average).

There’s plenty more rankings & ratings – and our take on the evaluation results – in our new Enterprise Software Report.

(1) Note: Oracle.com’s hardware product areas (a.k.a. Sun Microsystems hardware & systems) were not reviewed for this industry segment.

Subscribers Login Here.
Non-subscribers can learn more about The Library here.

Did you hear that the blue jean giant The Gap announced a new logo? Yeah, it didn’t create a blip on my radar either. I work in the tech biz, and I don’t shop at The Gap.

But what did get my attention is how it’s re-branding effort bombed (and was subsequently pulled) solely based on the public outcry on social media outlets.

There are many blog posts and news outlets weighing in on The Gap’s misguided design decision, and the stunning influence social media can have on corporate decisions. Some blogs even present the intriguing (if not a bit far-fetched) hypothesis that The Gap’s social media strategy was actually an ingeniously crafted publicity stunt formulated to increase holiday sales and bolster waning stock performance.

So was the The Gap’s decision to announce its new logo using Facebook and Twitter a public gaffe or social coup? President of Gap Brand North America, Marka Hansen believes they “did not go about this the right way.” Sorry Ms. Hansen, I disagree. To my mind, it is proof positive that when a company employs social media properly—intentionally or accidentally—it strikes gold.

What did The Gap get right?

1. They published the announcement on two different social media outlets instead of one. This lets them “manage the message” by combining Facebook’s robust and flexible platform, with Twitter’s “cut to the chase” schema and vast reach. In social media, two forums are always better than one.
2. They have the opportunity to glean far more information than focus groups and studies can ever deliver. Their study sample is The Gap’s most loyal customers and company watchers. Its demographic is worldwide. The responses are genuine and unvarnished. Best of all—it’s virtually free. If The Gap’s number crunchers are smart, they’ll catalog responses that go beyond opinions about the logo. There is gold in them there hills.
3. They quickly responded to public feedback by pulling the new logo and publishing a press release focusing on how their customers influenced in their decision. The company has even offered customers to submit their own designs going forward. Nothing fosters customer loyalty more than showing they are valued.

What did they do wrong? From the tenor of Ms. Hansen’s latest press release, it would seem that The Gap and its branding agency weren’t emotionally ready for such overwhelmingly negative feedback. That’s about all.

Maybe the whole logo idea wasn’t all it was cracked up to be. Perhaps the overwhelmingly negative feedback was more than the folks at The Gap expected. But, their social media strategy was a resounding success. It was done with the right intentions—if not a bit fearlessly. It gave them the feedback they needed. They responded with all the right messages. And most importantly—in the face of negative feedback they didn’t pack their bags and head home. Instead, they are gearing up for round two by exploring new ways to harness the power of social media. Now that’s the way to run a social media strategy.

Real people, real voices, real time

While the SMB & mid-size market are using social media to just get their name out to the masses, large companies are learning how to use social media & communities in tandem for a whole new purpose. And it is literally changing the face of IT.

Large IT companies don’t need to get their name out there. I mean, who doesn’t know IBM, HP, Apple, or Dell? I would bet that even your grandmother knows the name of one of these companies, even if she doesn’t know how to use a mouse. So, aside from using social media as a slick press release system, how do large companies use these outlets to their advantage?

The death of the faceless organization: For eons IBM has had the reputation of the faceless organization. In fact, it became so faceless that the phrase “blue suits” was coined to describe it. Well, those days are over—welcome the people that are IBM. Over the past 12 months, IBM.com has started publishing the names & faces of the managers & executives who steer IBM—the company, the vision, and the products.  This initiative is supported by its online communities, Facebook pages, and Twitter personas. Now IBM customers & fans can engage with IBMers real time, and hear the perspectives of IBM employees on an ongoing basis. Now IBM has many, many faces & voices—and many of them wear golf shirts.

What goes around comes around: Large companies are made of people, and people want to hear what others are saying about them. This is where the social networking part of the equation comes into play. Forums like Facebook, MySpace, and Twitter offer companies a view into the opinions, wants, and needs of its potential buyers and customers like never before. Just type IBM, HP, Dell, or Apple into the Facebook search engine and see how many pages show up. Or cruise these Websites for links back to their Facebook pages or Twitter profiles. These companies are learning that these forums are more than places to meet & greet. They are the largest-scale, lowest-cost testing forums and CSAT studies they could ever want.

To err is human to deliver spin control divine: Strategies that didn’t pan out. Products that don’t cut the mustard. Far reaching visions that never came to fruition. All the major companies have them, only now social media gives them the platforms to take wrong turns into new directions. Executives use blogs, Webcasts, and Podcasts to subtly shift their company’s visions to accommodate an ever changing business landscape. Meanwhile, product managers use blogs & forums to announce “new & improved features” (or in other words, “fixes for glitches”) in an upcoming product update—or in the worst of cases, fall on their swords. But no matter the topic, issue, or intent, all these social mediums give companies the flexibility to shift, craft, and hone their visions and image to align to current realities.

What do all of these strategies have in common? The power that social media gives people to change a company’s image, direction, and products—whether it is the company’s executives, product managers, customers, or prospects. The benefits of these strategies & forums are not limited to only the mighty few. Any company that is willing to use social media to open a dialog and listen to what people say can become the flexible and responsive organization that potential buyers and customers crave.

I can remember when flash mobs were a hot topic in the media; essentially a group of people with more time on their hands than good sense. I figured this phenomenon would go the way of the mosh pit.

Who knew that Dell.com would try and harness it as a form of online community behavior?

Here, I’m talking about Dell.com’s Storm Sessions which nestle within its Idea Storm community. In simple terms, this is the place where Dell throws out an idea or question for community members, starts the clock, and lets community members throw in their opinions.

Gabfests can be open for a month or a few days. Once the clock has run out, the session is closed, Dell reviews the comments, and shares how/when Dell will act on the information.

I’ve been watching Storm Sessions for about six months to see if this idea would gain any traction within Dell’s community ecosystem and gauge the kinds of issues and questions Dell would pose.

From one perspective, I’m cautiously impressed. Topics range from how to improve the site’s support zones, to mobile technologies desires, to if and how Dell’s corporate responsibility behaviors influence purchase decisions. In other words, information that most technology companies need to know.

Member participation?  Not so impressed. Generally, less than 20 members weigh in with opinions—which doesn’t even represent a rounding error when Dell’s massive community population is added up.

This light user footprint underscores the community behaviors we see across the IT Web. Most conversations are happening in support and developer forums. When the conversation turns to business topics, most members would rather lurk and listen – which doesn’t do much when you are trying to spark a dialog.

Despite these realities I wouldn’t count this idea out.  Digital flash mobs could be a very powerful way to spark conversations within technical communities and, if handled properly, deliver a wealth of next generation product insights. They also allow a company to “cap” the conversation, distill the results into essential points, and feed back results to the whole community.

From this perspective, don’t think of them as “flash mobs” with a purpose. Think of them as flash research that can pay huge dividends.

I’ve been doing deep dives into some communities on the Web and have decided that many of them are time bombs waiting to go off in companies.

To see why, let me take you on a quick trip down memory lane.

In a previous life, I was a strategic consultant to major IT companies – and once found myself presenting to Wang Lab’s Board of Directors. The topic of the day was customer satisfaction. A cadre of VPs were telling Dr. Wang that customers loved the company. Our customer satisfaction surveys showed that the peasants were getting ready to storm the castle, pitch forks and all.

Wang’s solution to the problem was simple. The company’s engineers and support staffs were mobilized to visit all of Wang’s major customers, chat up product benefits, and listen to their needs. Six weeks later, the company had to sponsor a series of mandatory classes to teach its technical staff how to talk to customers, and wear a suit and tie. Customer satisfaction never improved — and the end game is now industry lore.

Fade out a couple of decades later and we have the same situations brewing in digital form. To see why, consider a couple of examples I ran across the other day.

In one case, a customer posted a comment that he was very unhappy with a particular product and was going to defect to a competitor. The response from the moderator? “I’m sorry you feel that way. You’ll be back though. Their product is as bad as ours.”

Or consider what is happening in the D-Link forums, where customers are threatening to form a class action lawsuit and telling prospective buyers to run away. The company’s reaction?  A great deal of silence punctuated by moderators hurling insults at angry users. If that’s not a pitchfork moment, I don’t know what is.

This brings me to the salient point. Operating any community without an army of well-trained moderators is a great way to destroy your brand. Once upon a time, companies thought they could stem the tide with some sartorial splendor. Today, even more damage can be done with a pair of fuzzy slippers and a laptop at home.

In my recent online communities strategy report, I stressed the importance of designing reward programs to encourage visitors and members to engage in the conversation, rather than lurk and listen. It’s the secret to the success of the largest and most vibrant communities – whether they are targeted at brand building or product support.

The issue is, of course, what kind of reward program? Should companies dangle an iPad or a cruise in front of members or go for bragging rights?

Bragging rights are the most popular and arguably the most effective. These programs are usually based on a point system. Members get points when they answer a question – and receive additional “Brownie points” when the answer actually works.

Point levels create a pecking order where members with the highest points qualify for a particular status – and thus, more community trust. Typical levels range from Contributor (clueless but participating) to Expert (actually knows what s/he’s talking about).

As an aside, it’s worth mentioning that Oracle.com’s Ace program has an interesting variation on the theme. It allows members to vote for other members to become an Oracle Ace – the highest status on the OTN site. It’s a great way to allow the community to decide who really provides value (vs. activity) – although the ballot process requires the company to sponsor the contest and handle the back end logistics. (Think of it as a really complicated Cosmo quiz with some bite.)

And then we have HP, who uses a series of point systems with some very odd twists.

Business Support (click to see panel)

Its Business Support Center forum levels range from “Shining” to “Radiant” to “Beaming” to “Sweltering” (which is, for some odd reason higher than “Hot” — I live in Arizona, so I should know).

Meanwhile, HP IT Resource Center members can achieve “Pro”, “Graduate”, “Wizard”, “Royalty”, “Pharaoh”, and various “Olympian” levels.

But wait, there’s more. If you are participating in the HP Support Forum, you have the opportunity to be a “Student”, “Top Student”, “Tutor”, “Teacher”, “Grad Student”, “PHD Student”, “Associate Professor”, “Tenured Professor” and (drum roll please) a “Distinguished Professor”.

IT Resources (click to see panel)

Which brings me to an interesting question. What happens if you participate in all three venues? Can you be a Beaming Wizard Grad Student? Or, perhaps a Sweltering PHD Student on your way to becoming a Tenured Professor?

And then there’s the bunny – the icon that shows a rabbit being pulled out of a hat. This icon is slapped on any question that gets answered. I guess that means that any support answer provided in an HP forum is –  well you know.

This brings me to a couple of salient points. First, if you’re going to have a reward system, it’s a good idea to be consistent. It’s also a good idea to design a system that’s relevant for the members who want to achieve bragging rights. How many business professionals do you know who want to become “Beaming” or “Radiant”?  Do you know many IT professionals who are burning the midnight oil to become a “Pharaoh”?  Finally, do you know what HP.com is thinking? I don’t.

Rabbit out of the hat (click to see the whole story)

Support Forum (click to see panel)

The French have a wonderful phrase “plus ça change, plus c’est la même chose”. It’s an elegant way to say that the more things change, the more they remain the same.

That’s certainly the case when you consider the complex B2B buying process. Since our last update in 2006, this process remains the same. It’s still dictated by task forces and committees who must wade through 17 steps to get to the finish line. Participants are still purchase influencers, rather than the decision makers companies crave. Websites still influence only 5 steps in the buying process.

But that doesn’t mean that everything remains the same.

What has changed are the tools in a Website’s arsenal. How social media, communities, and Web 2.0 tools and behaviors have changed how content is delivered, shared, and used by task forces who make the final recommendation.

But here’s the big “aha”. New media and tools are rifles, not shotguns. Each plays a different role at different points in the B2B buying process. Here’s brief playbook to consider.

• An active presence on social networks, mentions in heavily trafficked “expert” blogs, high ‘news’ visibility on popular search engines, and traction on the growing number of news aggregation sites are the tools companies need to move out of the shadows and make it onto the “long list.”
• Making the “short list” requires an additional set of tools that educate buyers, including video data sheets and customer testimonials, short podcasts and on demand Webinars, and a variety of company-operated communities.
• Once a company makes it into the finalist circle, the big guns come out. Interactive demonstrations and product tours (usually video). Live Webinars that allow task force members to pose questions. Try and buy options and short-term SAS subscriptions. Sales chat and other call to action behaviors that encourage task force members to engage with the company’s sales force.

If you look at Websites today, it’s pretty clear that most companies fall into one of two buckets. Those that are ignoring these innovations because they can’t figure out how to use them — and companies that are throwing them out willy nilly under the assumption that something is sure to stick.

I vote for the few and the brave in the third bucket. Those that understand the real power of these cool new tools, have mapped them to the B2B buying process — and end up getting the deal.

Related Research: How They Buy. Creating B2B Websites that outsell the competition and reduce the cost of sales. (Report excerpts & podcast.)

We’ve mixed up the siteIQ eBusiness Index for 2010. In some cases the decision was based on eliminating Websites that haven’t improved over the past year and don’t provide any good or best practices worth tracking. In other cases, sites have fallen off our list because they don’t exist anymore. That would be EDS.com which was integrated into HP.com — and Sun.com which went off the airwaves last week.

In both cases, these sites are a loss to anyone who cares about good and great practices. Don’t bother to follow them to their new homes. Their presence inside of their new parent’s sites is nothing to write home about.

Here’s some more about the new sites added to our roster – and why we’re singing “auld lang syne” to others. (If you are a siteIQ client you can read more about what we expect from these and other leading sites in 2010 in our new executive brief in the siteIntelligence Research Center).

IBM Software Group (zone): We’ve been evaluating and tracking IBM.com’s software zone for years as part of our custom client programs. This year it’s time for this zone to move onto the eBusiness roster. It should be interesting to see how this aging and oft cluttered design fares against competitors who are adopting new, highly efficient designs and Web 2.0 features that play directly to the B2B buying process. Don’t even get me started about communities — which are a blooming mess (tag: enterprise software)

SAS: We decided to replace Microsoft’s Dynamics zone with SAS.com this year for a simple reason. We’ve been tracking Microsoft’s enterprise software zone for years under the assumption that it would eventually figure out how to sell to large enterprises and step up to equal footing with the big boys. That never happened. This zone has been a step-child on this site from the get-go and it’s essentially invisible today. This year we decided to take off our miner’s helmets and replace this zone with SAS.com.

The case for SAS.com comes more from its industry presence than its Website footprint. It’s a perfectly mediocre site that seems to have missed the memos on Web 2.0 and social media marketing behaviors. If nothing else, it should be interesting to see how it stacks up against the other big boys in the enterprise software industry. If you are looking for a new leader though, I wouldn’t hold my breath. (tag: enterprise software)

EMC.com: EMC.com is new to our roster in 2010 and we’ve put it in the enterprise systems category. It’s a bit of an odd fit, but we think this is the segment that best matches its strategy and focus. Needless to say, EMC.com has come a long way over the past few years and it’s another example of a site that is maximizing Web 2.0 elements and creating standards for high impact designs. From a first blush perspective, it’s a small but mighty site in the spirit of Adobe.com.  It will be interesting to see where the site excels – and where it misses the mark in 2010. (tag: enterprise systems)

Sun.com: Last week, Sun.com went off the airwaves and was stuffed into a few pages on the Oracle.com site. Thus, ends the life of one of the IT industry’s most innovative and useful Websites.

To see why, let’s look at just a few of Sun.com’s industry “firsts.” It pioneered communities on the Web; embraced Web 2.0 when it was just a spec on the horizon; was the first to deploy tab top designs to consolidate content into manageable bites; automatically discounted prices in its (public) ecommerce system based on the customer’s contractual relationship; pioneered dashboards that allowed customers to customize information to their needs: operated the industry’s largest (java.com) and most diverse range of developer portals; and designed the industry’s first contextual search engine that organized results by objective, type, or role. Oh, and one thing else. It did all of this while ranking #1 in usability over the past 6 years.

All in all, a stellar run for the record book. It’s too bad that the company’s business strategy (and the CXOs that ran it) weren’t as effective as this site.

Intel.com: Intel.com is one of the industry’s most interesting studies because it marches to the tune of a truly unique drummer. Instead of marketing Intel’s products and services with sales as the end game, this site is focused on conditioning markets to prefer (and preferably demand) products built on Intel technologies — and then reward companies that sell Intel-based products by driving buyers directly to their Websites or retail stores. In simple terms, Intel.com isn’t chartered to sell Intel’s chips; it’s responsible for making sure that Intel is the buyers’ preferred brand.

From this perspective, Intel.com has always been one of few pure play ‘brand’ Websites. It’s also one of the first sites to effectively harness communities and Web 2.0 behaviors to start a conversation with users and buyers. For these and other reasons, we’re glad to welcome Intel.com to the 2010 eBusiness Index. May the best brand player win.

Brocade.com: Has anyone else noticed that the network systems industry is consolidating like a cardboard box in the rain? If not, spend some quality time looking for a fourth network systems Website (we also evaluate Cisco.com, Nortel.com and Juniper.net twice a year.  that has any best practices and is larger than a newt. After we spent some quality time considering the candidates, we flipped a coin between Brocade.com and Enterasys.com. Brocade.com won the toss.

We’ve got some real hope for Brocade.com in 2010 since it boasts a bevy of Web 2.0 behaviors, uses the same mega-menus as Juniper.net, and operates communities that show some promise. Of course, the devil’s in the details. Stay tuned for the results from our first evaluation which will be available in Q2 2010. (tag: network systems)

Deloitte.com: Deloitte’s inclusion on the 2010 eBusiness Index roster is due, in large part, to EDS.com riding into the sunset. Quite frankly, we’re glad to have a reason to shake up the professional services sites we cover. Most of them have been skating on the innovation and design fronts for too many years.

Deloitte provides a much needed breath of fresh air to our professional services roster. It’s built on a well crafted design and its lively first person voice takes much of the voodoo out of the consulting services conversation. All in all, we expect Deloitte.com to raise the bar for Accenture.com, and especially IBM Global Services. It should be interesting to see how those sites stack up now that there’s a new deputy sheriff in town.  (tag: professional services)

Here’s the rest of the sites we’ll be evaluating in 2010. If you are a siteIQ client, you can read more about these sites’ strengths and challenges in our new executive brief in the siteIntelligence Research Center.

• Adobe.com
• Microsoft.com
• Symantec.com
• CA.com
• Oracle.com
• SAP.com
• Dell.com
• HP.com
• IBM.com
• Cisco.com
• Nortel.com
• Juniper.net
• CDW.com
• Insight.com
• NewEgg.com
• Accenture.com
• IBM Global Services (zone)

What do you think of these sites? Sound off in comments

Idea communities factored greatly in our new report  about the best and worst  IT communities, and Dell.com’s IdeaStorm ended up as the poster child for an idea community gone off the rails.  For those of you who want to avoid this fate, Mike ‘s related blog Do it Wrong Quickly offers  three  points that can help keep a lot of companies out of the ditch.

Web 2.0 and 3.0 can and will deliver real value to site operators and their visitors and prospects in 2010. Communities? Not so much.

Recently, I presented how we’re defining Web 2.0, Web 3.0 and communities to several clients, including our POV on their futures in 2010. It’s interesting that these three areas are finally moving beyond all the hype and bright and shiny objects that blinded most Web teams to their real potential last year.

Web 2.0 and 3.0 are finally coalescing into two ends of an important spectrum. Web 2.0 is all about content and information – providing these assets in new forms that make them easier for visitors to digest, use and share. Web 3.0 is actually about changing the interface and how users access these new generations of content. Simply put, it’s about creating adaptive sites that intuit or know why the visitor is there.

From my perspective, Web 2.0 and 3.0 can and will deliver real value to site operators and the visitors and prospects they are trying to woo and win in 2010. Communities? Not so much.

In fact, our in-depth evaluations show that most communities represent a high risk business strategy for companies. Why? Let me count the ways. Most companies:

• Vastly overestimate the real value and allure of their communities to visitors and members;
• Assume their experiences with online support and developer communities provide templates for future success;
• Don’t understand the real culture changes these communities will require;
• Ignore staffing and training investments that are critical to success; and
• Underestimate how much communities will cost, both in terms of corporate treasure and potentially, the company’s online brand.

And that’s just the tip of the iceberg.

Of course, some community teams will ignore these realities, preferring to be at the forefront of this cool, new — and very sexy — revolution. For these teams, I offer an old saw: you can always tell the pioneers by the arrows in their backs. If communities are your charter in 2010 – and you still believe in TinkerBell — buckle up and grab your flack jacket. It’s going to be an interesting ride.