I give this presentation pretty often and am happy to get into the details with you depending on your needs.

EDIT: Jesus, I didn't realize the notes didn't make it to SlideShare. The point is that it's best to gather all the data you can, then tweak how you do it. The more obscure your public-facing metric, the more flexibility you have to adapt it as time changes.

Here's a great intro guide from Yahoo: http://developer.yahoo.com/ypatterns/parent.php?pattern=repu...

I don't know what the subject matter of your site will be, but if this is your goal it will be very important to seed the initial membership of your site with persons who are known, on independent grounds, to have strong subject matter knowledge. I presume there will be back-and-forth among participants so that if one person says "Freedom is slavery," another person can disagree and say, "Actually, freedom is not slavery." But presumably your site will be intended to benefit visitors who really can't tell the difference about much more detailed and debatable issues.

I'll give an example. There is a site about college admission anxieties, College Confidential,

http://talk.collegeconfidential.com/

in which many participants make factual claims about how to reverse-engineer the college admission process. One common way for people to attempt to gain credibility on that site is to insinuate that they used to be admission officers at some highly selective college. Because the site is "confidential," that is its users use screen names rather than real names, it is often very difficult to verify such claims. It has not escaped my notice that users with chutzpah and a refusal to acknowledge push-back from other users can often be believed by onlookers even when making quite incredible and unverifiable statements. If you think this would be a problem on your site, you might want to think about how to designate verifiably more accurate and honest users in some way that is visible to all visitors.

In general, I think the initial seeding of the membership of an online community is very important. That is one thing that HN conspicuously got right.

Would it? Have you tested it? How do you know?

"I need to have a good way to identify not only spammers and users who might try to game the system, but users who may inadvertently put in noise/bad data."

Do you really need it? What's the worst that happens if you leave it out? What's the best? What's the likelihood of each? What data do you base that calculation on?

"Likewise, I also want both the system and other users to be able to implicitly and explicitly recognize those users that should be trusted, based on the value of the content they have submitted."

Sounds complicated. Why do you "need" to solve this problem? Do you even know it's a problem yet?

I'd put the simplest product in front of users and see what happens. Maybe you won't have to build your mega-meta-trust-network after all. That seems like a win.

- Use basic human motivation to encourage real participation. Points scored and virtual awards work well, but only if you can control system-gaming and inflation.

- Make sure that each account is connected to a confirmed email address or social network account.

- Spend time looking for people who are creating multiple accounts for the purpose of gaming your system (same IPS, repetitive names / passwords). Disable what appear to be the spoof accounts and email the core account about why you did it.

- Have a clearly defined policy for what is acceptable behavior and penalize as much bad behavior as is necessary. Make sure to show the community that there is enforcement.

- Maintain contact with helpful members of the community who can key you in on problematic individuals or exploitable flaws in the scoring system.

Also, if your "social" app requires friend requests/acceptance, as most "social" apps do, you further negate the need for a trust system. This is because the mere existence of friending/friend clusters are used in place of a trust system. You're implicitly trusting everyone you "friend".

So my comments probably don't apply as much.

Basically, we mimic the word-of-mouth recommendation model. The people you trust the most, are those you know well - yourself, friends, their friends, and then all trusted ones (the wisdom of the crowds). You build your own network of people you trust, and quickly slice and visualize information.

Here are a few links that you may find interesting. We're still tweaking stuff, so let me know if you have any thoughts on what we can do better. http://arnox.trustedones.com (my page)

http://maps.google.com/?q=http://www.trustedones.com/feeds/g... (all of my local recommendations on a map)

http://www.trustedones.com/topic/restaurant-local/listing/bo... (sample recommendation)

I haven't completely opened my website so I can't say how big of an effort it will require in the future. But I plan on possibly using Mechanical Turk or some other similar source paying penny's for accuracy verification.

I'm interested to know what your working on. Do you have an email address I can contact you at?