TheWayoftheWeb

I’m looking for the wisdom of MY crowd.

A thought that occurred to me commenting on a Robert Scoble post.

• I use Google Reader because I’ve selected the inputs.
• I get news from the people I’ve selected on Twitter and Friendfeed
• I get personal news from the people I’ve selected on Facebook.
• I don’t use Digg etc as much as I might because it’s the wisdom of a random crowd I haven’t selected.

It’s been interesting following recent events concerning Digg, especially considering my previous posts outlining the personal and idealogical problems I have with Digg, and the alternative ways available to get crowd sourced news.

I just read a great summary post by David Chen on Mashable, ‘Digg’s recent bans and the limits of crowdsourcing‘ which is a comprehensive look at what David calls ‘building a flawed system’, and the lessons learned – as well as looking at why Digg has banned top users, and how the business strategy is changing.

It reminded me of something I’d noticed recently. Here’s a graph from Compete, comparing Digg (blue line), with an alternative social content site, Mixx.com (red line).

Now, while it’s obvious that Digg is massively ahead at the moment, it appears to have levelled off significantly – and at the same time, notice how Mixx has grown around the same time as the Digg banhammer started making an appearance?

Now look at Mixx in detail:

Anyone else see something kickstarting some growth around Apri/May 2008?

Hmmmm.

Two things spring to mind. One is that Digg has vocally supported the community, and allowed top users a longstanding reign over the rest of the site – which means changing things, particularly without a clear warning, is always going to lead to problems – the question will be how big the problems will become?

And secondly, so many web services which seem to embrace, support and provide a Web 2.0, social media type approach, actually fall down on the Customer Service which is preached by those using them. I’ve always found Digg support to be incredibly varied, but always anonoymous. And the fall back is always on ‘the best for Digg users’ without ever explicitly saying what that means.

Other social news aggregations and voting sites like Reddit have also seen some growth, although in Reddit’s case, it may be down to other factors, such as opening up their service to Open Source installations.

But the fact that the high profile, long term devotees of Digg could be powering the rise in a close rival (in terms of the type of service provided) could prove to be a very interesting case study – if the very people Digg banned turn out to be able to power the rise of a challenge.

In an update to my last post on Digg, the good news is my colleagues and I are now unbanned from Digg, following several emails.  It’s good that the ban was lifted, as not only would it have left some of my colleagues unmotivated about social media, but I’m also quite determined to see if there’s a way to be successful at driving traffic on Digg without gaming the system – and to do that requires legitmate access rather than cheating and going through proxies, or using autoscripts etc. There’s quite an interesting profile of the top Digg user, MrBabyMan on ReadWriteWeb, and whilst our own Digg saga was developing, there was a rash of stories regarding whether or not he uses a script to autosubmit content.

But whatever happens, during my last post, I referneced quite a few sites which provide a major developing threat to Digg, as it continues to inhabit a world between the massive traffic of Yahoo Buzz, and the smaller focus of sites like Sphinn on the same technology premise.

RSS sharing: Personally, I use two web based RSS readers – Google Reader is my main reader, but I keep an account at Feedeachother because I got to know the sole person behind it, Udi Falkson, it’s got a lot of features which are as good as Google Reader, and it’s good to have a place for feeds which I read for pleasure and aren’t work related, stopping my main reader becoming incredibly overwhelmed. What both sites offer – and what almost every RSS reader contains now – is the ability to easily share stories with friends and fans via the sites themselves or emails etc. As user networks grow, this provides one major method for content of interest to appear in front of me.

Social news aggregators: In Web 1.0, this meant Digg. Now, however, it tends to mean sites like Twine and Socialmedian. These services combine uploading discovered content with methods to follow ‘friends’ who submit quality news, and to comment around it, re-share it, and discover more via recommendations. Numbers at the moment are small as Twine is still in private beta, and Socialmedian has recently emerged, but they’re growing:

Social networks/microblogging/lifestreaming: Lumping together everything from Facebook to Twitter to Friendfeed basically revoles around one thing. Recommendations from friends. I build networks on these sites from people I know or discover who share interests with me, and therefore, their recommendations carry a certain weight with them. Can I say the same about 800 random Digg users or Amazon reviewers? I know which of my friends can recommend video cameras, mobile phones, or good hiphop. And I’ve already built up this information in getting to know them and adding them. Do I want to have to start researching each Digg user, particularly when the top Diggers tend to submit and rate so much it’s seemingly random – one of the flaws of the Shout system and blind digging the content your friends send for reciprocal links.

Niche ranking sites: It doesn’t take much to create a ranking system – probably less work than getting an article on the front page of Digg! Hence sites like Sphinn, which concentrate on Marketing news and discussion. And bearing in mind that the ‘Digg Effect‘ is infamous for traffic which may have a high Bounce Rate, and low loyalty, the smaller niche sites have a benefit for attracting and interacting with likeminded individuals. Plus you need far less popularity to get to the front!

The majors: I’ve already mentioned Yahoo Buzz and the huge amounts of traffic it can bring. As major newspaper and media sites evolve towards becoming more digital, they’ll increasingly be seeking ways to aggregate and curate online content. They’ve got sizeable audiences, and a wider perceived range of interests than the ‘Digg Crowd’. If you’re not looking for technology specifically, would you browse Digg, or an aggregator run by the New York Times or the Guardian?

Something completely different – human search: I’ll be honest and admit I didn’t see a use for the likes of Mahalo when I first encountered it. Why search within a far smaller sea than Google can trawl, and rely on weak-minded humans rather than our robot overlords? But I always like to use something for a while before making a judgement, and I’m glad I did, because the Mahalo homepage and the plugin for Firefox have become really useful for seeing popular stories and sources of information (the plugin also improves Google search with a handy summary information box!). Now I get an overview of recent stories, which have had an element of human filtering. And it’s showing pretty good growth:

All of this information is coming to me without needing to visit the Digg homepage on a regular basis, or research the small group of Digg users who can effectively make or break a story. I could subscribe to sections by RSS, but given the churn of stories, and the fact I’m not interested in every car or internet news story, it becomes problematic. Instead, I can rely on friends and family who know me to be able to show me things they think are a) Cool, and b) Things I’d really like by their own self-selecting mechanism and no real effort on my part.

I’m intrigued to see how other people feel, especially after some of the great comments to my last post on Digg. At the moment, I can’t see an easy solution for Kevin Rose and the team, but I’ll try and outline some possibilities in my next post.

This post was partially inspired the fact Digg recently banned my user account – and by using IP blocking also blocked at least 8 other Digg users in an office of over 500 people.

Rather than focus on the individual implications, it prompted me to look deeper at the role Digg serves – and led me to a conclusion that Digg’s days are numbered.

The theory:

Digg doesn’t work as a proposition. Essentially, the site allows people to submit stories from around the web and vote on them, but that’s a tool or mechanism. It’s not a business or user proposition. And every time I think I might have figured out what the proposition could be, I come up against something within the Digg system that prevents it.

If you want an idea of the general news ranked by the opinions of the widest audience, you’d pick Yahoo Buzz. Sadly Compete seems to default to Yahoo.com for the url analytics, but even so, it provides reasonable size evidence.

The green line = Yahoo.com, the blue line = Digg, and the red line is Facebook.com.

And the reason I’ve included Facebook is the real reason I’m not digging Digg anymore. You might defend Digg as a niche site for technology and the bizarre (the actual submission trend has gone towards more lifestyle recently), and point to the size advantage it has over Slashdot and Techmeme, that’s not where my real niche news is coming from, and not where an increasing number of people are getting it.

The trend is towards communication, conversation, and friend/network recommendation. Hence Facebook, and the following graph comparing trends for Digg and Twitter:

If the trend continues, Digg has put itself in a corner. Because despite providing tools such as Friend Lists, and being about to ‘Shout’ stories to your friends, Digg bans you for being social.

Having multiple accounts from the same IP address should not have resulted in a ban – some were registered 2 years before the most recent, some were logged in at the same time etc – and a simple check on the IP would show it’s registered to a large company.

Submitting stories from your own sites is allowed in the Digg Terms of Service – as long as it doesn’t reach spam levels.

That only leaves the fact some of these accounts were ‘Digging’ the stories submitted by others on their Friends List – from the same IP address. Something which presumes we were trying to game the system, rather than the fact we happen to work together because we have a shared interest. ( We’re not the only ones!) Annoyingly, we did get a previous Bad IP address error which was lifted when I explained we all worked in the same building. Now, however, it’s an instant ban with no discussion – despite the fact it means I’d need to monitor the Digg accounts of anyone within a building of over 500 people.

And if you do have a friends network you’ve built up legitimately and there’s any hint of nepotism, Digg automatically gives it a lower ranking. Which would be fine over a certain point, but basically means there’s no point in friends who aren’t power users.

Anti-social behaviour:

The annoying aspect is that there isn’t a warning system, or an explanation. When you attempt to log in you get: