Crowdsourced curation, reputation systems, and the social graph

“A good example of manual curation vs. crowdsourced curation is the competing app markets on the Apple iPhone and Google Android phone operating systems. Apple fans complain that the Android marketplace has too many low-quality apps for any given task. They complain that it’s hard to find an “official” or “sanctioned” app. On the other hand, Android fans criticise Apple for limiting their choices. They don’t want to be beholden to the whims of a select few. Apple is a monarchy, albeit with a wise and benevolent king. Android is burgeoning democracy, inefficient and messy, but free. Apple is the last, best example of the Industrial Age and its top-down, mass market/mass production paradigm. They deal with the big head of the curve, and eschew the long tail. They manufacture cool. They rely on “consumers”, and they protect those consumers from too many choices by selecting what is worthy, and what is not. Google Android is building itself as a platform for bottom-up innovation. Their marketplace publishes first, filters second, utilizing little more than the rankings of the community.”

I have long been waiting for systems that allow crowdsourced judgment to be tweaked, not to the taste of the general mass, which produces lowest common denominator effects, but to people and experts that you can trust for their judgment. According to the article below by Eric Reasons, these systems are now implemented by Buzz and Digg 4, though I have not tried them. Important for me though, is that they don’t just take your social graph as is, because that mixes many different people for different reasons, but that you can tweak the groups.

Below, Eric Reasons gives a great overview of the issue, based on the comparison of Android vs. Apple’s application curation methods.

Eric Reasons:

“It is at this point that many people interject: “This is the problem with the internet! It’s full of crap!” Many would argue that without professional producers, editors, publishers, and the natural scarcity that we became accustomed to, there’s a flood of low-quality material that we can’t possible sift through on our own. From blogs to music to software to journalism, one of the biggest fears of the established order is how to handle the oncoming glut of mediocrity. Who shall tell us The Good from The Bad? “We need gatekeepers, and they need to be paid!”

All of this is true, to an extent. We do need ways to filter and discover content. And just because we can produce something and transmit it, doesn’t mean that it’s worth consuming. Luckily, the Internet not only gave us the means to produce and transmit on our own, but to curate as well. We do it every time we e-mail, share, “like”, tweet, or buzz a link. For example…

How did you find this article that you’re reading right now? I’m pretty sure it hasn’t been published in the New York Times. No professional editor or publisher made a determination for you as to this article’s quality, aside from the author himself. Did a friend send you a link? Did you see it posted to Facebook, Twitter, Buzz? The Internet has enabled us to build our social graph, and in turn, that social graph acts as an aggregate gatekeeper. The better that these systems for crowdsourcing the curation of content become, the more accurate the results will be.

This social-graph-as-curation is still relatively new, even by Internet standards. However, with tools like Buzz and Digg 4 (which allows you to see the aggregate ratings for content based on your social graph, and not the whole wide world) this technique is catching up to human publishers fast. For those areas where we don’t have strong social ties, we can count on reputation systems to help us “rate the raters”. These systems allow strangers to rate each other’s content, giving users some idea of who to trust, without having to know them personally. Yelp has a fairly mature reputation system, where locations are rated by users, but the users are rated, in turn, by each other.

Reputation systems and the social graph allow us to crowdsource curation. I’m not ready to argue that these systems are up to replacing individual human curation (yet), but they’re getting better every day, and I think they are well on their way.”

The rest of the article compares the iPhone’s and Android’s curation methods for application, with the above distinctions in mind.

And the author concludes:

“Consider for a moment that the Internet will only continue to increase our freedom and ability to produce content, whether they are smartphone apps, videos, music, pictures, or words. Can you imagine if Apple had to approve your videos for posting on Youtube, where every minute, 24 hours of footage are uploaded? There’s no way humans could keep up! The traditional forms of curation and gatekeeping simply can not scale to meet the increase in production and transmission that the Internet allows.

Crowdsourcing is the only curatorial/editorial mechanism that can scale to match the increased ability to produce that the Internet has given us. As the former “consumers” become “producers”, we’re going to see better and better implementations of reputation systems, and better integration with our social graph, because they are the only mechanisms that are feasible, but also because we just love to share. Twentieth century mechanisms for curating/editing are built for a top-down, mass-production age, and they can’t keep up with us, the former audience, as we make the leap into production.”

1 Comment Crowdsourced curation, reputation systems, and the social graph

  1. F. Randall Farmer

    Though it is true that peer-curation is the only scaling approach for very large data sets, like Youtube, care must be taken in sorting out the difference between (what Bryce Glass and I call) the Good, the Bad, and the Ugly – in our book Building Web Reputation Systems (a freely readable web draft is available.) I highly recommend reading the case study of the development and deployment of Yahoo! Answers’ Community Content Moderation model.

    Happy Curating!

Leave A Comment

Your email address will not be published. Required fields are marked *