I've noticed this too. Unfortunately, my two most upvoted comments recently have been of the "throwaway" variety. Not mean, not trollish, but a couple of easy-to-agree with quickies that I banged off on the bus ride to work. One[1], perhaps legitimately upvoted so that the response would be more visible, was a simple statement that the submission was a couple years old and it would be interesting to see some newer data. The other[2] was a cheap, throwaway anti-SOPA line.
The danger here is that the easy-to-upvote throwaway comments become a self-fulfilling prophecy. Like it or not, karma is social validation. A long, thought-out reply that gets no upvotes is somewhat depressing; it makes you think that, even at your most thoughtful, your thoughts are unappreciated. Contrast that with a cheap and easy line that gets 40 points, and the social feedback is clear: your considered opinion isn't valued, but your cheap lines are.
It makes me think that the karma system itself might be fatally flawed for its lack of scalability. What works for a small, mostly homogenous community breaks when that community becomes larger and more diverse. We can deny it all we like, but I'm sure most people feel pretty good when they log on and see their karma significantly higher than it was the last time they checked. It's an ego boost, and few people are immune to enjoying that. When playing to the lowest common denominator gives you that fix of social validation, and a more thoughtful comment doesn't, it's pretty strong positive reinforcement for the less-desirable behaviour.
I'm not sure how to combat this in a scalable manner. It's difficult to think of examples of community-moderated forums maintaining quality in the face of rapid expansion. My previous suggestions of weighting the votes of those who consistently upvote quality comments higher than those of people who upvote fluff has met with little traction, and I can see some pretty big potential weaknesses there, myself.
In reviewing this comment, however, I have had a thought of something that seems ridiculously trivial, but might just work: pg has stated before his belief that comment length is a relatively good indicator of quality. I notice, however, that the comment box is relatively tiny. It makes a long, thought-out post appear, prima facie, no more substantive than a 3-liner. I would be interested to see what the result of making the comment box significantly larger would be. Would people, upon seeing that much empty space surrounding a throwaway comment, be likely to reconsider posting it? Are there easy cues like this that can be used to hack the behaviour of commenters? I'd be interested in seeing the results of running an A/B test on a seemingly simple change like this: Make half the users' comment boxes two or three times as long. Leave it that way for a few months, then take a random sample of the resulting comments and see if there's a difference in quality. I wouldn't be surprised if there was.
I don't know that this social problem has a technical solution, but you're right that the "points" system encourages shallow comments. Unfortunately some people will always see a number as a score, and start gaming the system to increase it.
The problem with many submissions recently has been they are taken directly from reddit, and have content that sits well with reddit but not HN. The "Programming prodigy passes away at 16" story is a good example: it's tragic news, but the fact that someone has died is not necessarily a good HN submission. Although much of the discussion about Erlang and Haskell went right over my head, I think this site did better when those stories were oft-submitted, if only because it tends to push away the people who would prefer fluffy "human interest" stories. I personally have no intention of creating a startup but find pointers to useful technical ideas and tools here - more of those, please
So I think we need to concentrate on the submissions, removing stories of marginal interest to hackers, and being diligent in upvoting good stories and comments, and downvoting crufty comments.
Yeah, submissions may be where the solution lies. I'm not sure if it's just me but has anyone else noticed that the New page gets completely saturated with spam? I have show dead turned on and I used to see a handful of dead stories. These days it's not uncommon for me to see 5 live stories with the rest having been killed. A month or so ago there was a discussion about HN's rank on Google. I'd be happy to see it close to the bottom of the first page if at all.
"It makes me think that the karma system itself might be fatally flawed
for its lack of scalability."
The motivation to be part of an online community should be the enjoyment
at the discussions and not the number of points.
I hate the points, and I hate it, that I look after them. I think that
just the presence of a karma system has a bad effect on the quality
of the comments. You're getting what you ask for.
It's symptomatic how often I read 'Please don't downvote me' or
'I hate to say this, but' in the last time.
The danger here is that the easy-to-upvote throwaway comments become a self-fulfilling prophecy. Like it or not, karma is social validation. A long, thought-out reply that gets no upvotes is somewhat depressing; it makes you think that, even at your most thoughtful, your thoughts are unappreciated. Contrast that with a cheap and easy line that gets 40 points, and the social feedback is clear: your considered opinion isn't valued, but your cheap lines are.
It makes me think that the karma system itself might be fatally flawed for its lack of scalability. What works for a small, mostly homogenous community breaks when that community becomes larger and more diverse. We can deny it all we like, but I'm sure most people feel pretty good when they log on and see their karma significantly higher than it was the last time they checked. It's an ego boost, and few people are immune to enjoying that. When playing to the lowest common denominator gives you that fix of social validation, and a more thoughtful comment doesn't, it's pretty strong positive reinforcement for the less-desirable behaviour.
I'm not sure how to combat this in a scalable manner. It's difficult to think of examples of community-moderated forums maintaining quality in the face of rapid expansion. My previous suggestions of weighting the votes of those who consistently upvote quality comments higher than those of people who upvote fluff has met with little traction, and I can see some pretty big potential weaknesses there, myself.
In reviewing this comment, however, I have had a thought of something that seems ridiculously trivial, but might just work: pg has stated before his belief that comment length is a relatively good indicator of quality. I notice, however, that the comment box is relatively tiny. It makes a long, thought-out post appear, prima facie, no more substantive than a 3-liner. I would be interested to see what the result of making the comment box significantly larger would be. Would people, upon seeing that much empty space surrounding a throwaway comment, be likely to reconsider posting it? Are there easy cues like this that can be used to hack the behaviour of commenters? I'd be interested in seeing the results of running an A/B test on a seemingly simple change like this: Make half the users' comment boxes two or three times as long. Leave it that way for a few months, then take a random sample of the resulting comments and see if there's a difference in quality. I wouldn't be surprised if there was.
[1]http://news.ycombinator.com/item?id=3458644
[2]http://news.ycombinator.com/item?id=3462700