I now have a semi-private "HN Reader" which has completely broken my HN habit while still feeding me stuff I might be interested in. Because I built it, I can make it do anything I want. I've started to turn it into a slightly broader search engine so that I can find the things that I'm looking for; I got sick of seeing search engines brag about the hundreds-of-thousands or millions of search results they were returning when I was trying to find something (really, what's the point of that?), so I'm building my own. I got sick of feeding some psychological trigger in my brain that made me nervously check the HN front page numerous times throughout the day, and I'd find myself clicking on items that had lots of comments and activity even if the subject was something I wasn't interested in. I guess I was thinking, "wow, lots of people over there, I should go check that out."

What did it for me was a bit of foggy nostalgia one day. I was thinking about "the good ol' days", how I -- we, all of us if we were lucky enough to be born at the right time in the right environment -- used to modify the crap of out of programs, change their interface, tweak their colors, cheat at games even when we were the only ones playing. We used to take things we didn't like and turn them into things we did like.

But nobody, or very few people, do that for the web, even though there are piles and piles of tools that make it easy and doable.

So I did it.

And it is glorious.

It's some of the most fun I've had at programming in years. Now when I'm feeling like a wet cat, I'll just go tweak my little reader-search-engine-toy, and then I feel better. Now I never feel like I'm missing out on something on HN, because my little toy is keeping an eye on it for me and saving the stuff I might care about it.

And if you're looking for a new community ... well, build that too! It's clear from numerous threads on HN and other places that people are ready for something new. Make what you want, share it if you feel like, if enough other people like it maybe they'll join in and you'll have your community.

http://telesc.pe/

It's a real-time, open source HN clone built on Meteor, with features like invite-only mode, notifications, and a lot more stuff.

And since it's open source, if something is missing you can always code it yourself and contribute it back to the project.

Forking!

Frustrated!

Bayesian filtering seems to work great on HN headlines. I trained mine with about two years of data scrapped from http://www.daemonology.net/hn-daily/ (apologies to whoever runs that), then just basically wrapped that with some code that grabs hacker news's main page and displays the filtered headlines to me. It nails politics and startup crap with stunning accuracy.

The only problem is that now I find myself using both that system and the website itself.

If you're looking to extract data from HN, use HNSearch. http://www.hnsearch.com/

There's no reason to scrape data from HN itself.

HTML APIs to come with the added benefit of being less prone to change, which I realize goes against conventional wisdom, but seems to hold in practice.

HTML APIs to come with the added benefit of being less prone to change

Really? I don't think that does hold in a lot of cases. Using HN as an example, it broke a year or so ago when PG changed how the job postings were listed. Again, the quality of an API can vary, but at least you would know what changed in that case.

Worthwhile communities need to be maintained by barriers. The kind of barrier that I personally enjoy most is, "Crappy enough interface that only people who really like the content will be motivated to show up."

I grant the existence of useful content there and a good API. However after one too many brushes with how differently moderators think about stuff than the people who I liked talking to did, I have demonstrated the truth of my comment by choosing to leave.

StackOverflow works for a lot of people. But for me, personally, a stripped down interface and less moderation is a better experience.

Presentation, along with content, is a better experience than just having good content.

You having the interface you want shouldn't impact my having the interface I want.

An API allows everyone to have the interface they want.

See http://ihackernews.com/

In lieu of that of course there's always greasemonkey.

However, please do not make it like HN or you will fail.

Be different. Brand it. Have a personality.

In answer to your non-question question, yes that site exists, there are literally hundreds of them. They get a few users, then they don't get new news, they get a few 'bad' things (people get scolded for being negative, or chastised for being 'off topic' or 'social') and then no one wants to post because they can't really tell what is "acceptable" and what isn't. People stop going there. It fades into obscurity.

My suggestion is that what your looking for isn't a web site its friends. Get together once a week and hang out. You can talk about programming languages, mathematics, science and other oddities. Invite people who stick to the program, shun people who keep wanting to talk about tech products.

https://dl.acm.org/citation.cfm?id=2254064 (click on Table of Contents)

If you want to discuss them, send email to the authors, try and find some grad students or professors in your CS department, or find a way to attend some CS conferences.

I don't think you'll find deep research-quality conversation in a news aggregator, mostly because the people who are interested in having and also able to have research-quality conversations are for the most part busy doing research, and also because coming up with a reasonable opinion about something complicated that you're not an expert in takes a lot of work.

There are specific blogs, mailing lists, and (maybe defunct) newsgroups where you can discuss more focused topics, e.g. http://lambda-the-ultimate.org/

Also, try Prismatic maybe.

The cliché is that humans are fundamentally social creatures, and I think one of the primary ways we maintain the high engagement in a discussion thread is by becoming interested in the other participants, not just the topic. That's usually a good thing, because it motivates us to respond and care about the discussion, but it also means that people will go off-topic, get into heated arguments, or bring up memes and social topics.

If it were possible to strictly enforce rules against off-topic or non-technical discussions, I suspect engagement would drop very quickly. Learning is a strong motivator, but I doubt it provides the same semi-addictive feedback loop that the social aspects of a site do.

(I don't have any evidence beyond my own gut feelings based on web forums over the years --- happy to be proved wrong! A high-engagement non-social discussion forum would be an interesting place.)

I'm very hopeful for Lobsters.

>Invitations

>Not for exclusivity, but rather, invitations will be used as a spam-control mechanism. New users must be invited by a current member and invitations will be unlimited (unless scaling problems temporarily prevent new accounts). If spammers are invited to the site and banned, the user that invited them may also be banned, going up the chain of invitations as needed.

https://lobste.rs/s/bkeYe9/about_lobsters

Me, I prefer to have discussions, but it's been tough going. I'm optimistic but it may take a while.

Last time lobste.rs was mentioned on HN there was an influx of people, but the only noticeable effect was that there were more submissions and slightly higher average submission up-vote numbers. Commenting seem to stay about the same.

(Cause a week ago I reached the threshold of wanting to leave a comment enough times)

So quick :) thank you

http://scr.im/2pty

jerod dot santo at gmail dot com

Thanks in advance!

ambros at gmail.

edit: thanks polyfractal!

https://github.com/jcs/lobsters

I'm sad this part of the net has been fading into obscurity the last ten years especially since its alternatives have proven to be inadequate. Its greatest power was it being decentralised and thus couldn't be policed by anyone, the variety and custom readers.

Its greatest weak point was perhaps the trolls. Maybe others can supply more weaknesses?

Really, what does Hackernews as a website have over f.e. a newsgroup alt.news.hackers?

> what does Hackernews as a website have over f.e. a newsgroup alt.news.hackers?

Communal voting?

> Its greatest weak point was perhaps the trolls. Maybe others can supply more weaknesses?

A rigid definition of what counts as spam (Breidbart index)?

People using all kinds of domains in the From: header. That's a problem because bots scrape the from header for sending spam, and there's nothing to stop people using valid domains that don't belong to them.

<nostalgia>But otherwise... I miss Usenet so much. I was raised by Usenet, it shaped my life philosophy, I found my best friend there, not to mention knowledge I got.</nostalgia>

So, maybe time to write a "Usenet 2.0", with: - up/downvotes, - markdown, - tags?

what do you mean by that?

ironically I'll probably sound negative, but are you looking for a place which enforces PC to an ubearable point where the only accepted state is, you know, people standing in a circle smiling while performing a certain activity?

I was disappointed. There's about three reasonable quality posts, and even they are short on technical and financial details. All the other top-level posts are basically noise.

I don't think HN needs "enforced PC" but I do think it would be improved if instead of just posting "it won't work" people posted "it will be difficult because of a, b and c which you need for x, y and z and which will cost i, j and k"

[1] http://news.ycombinator.com/item?id=4893776

This is how it works: You tell intigi what sources (RSS feeds and twitter accounts) you want to follow. Then you provide intigi with what's basically a lucene query that you want articles to match. Intigi then monitors your sources, indexes the article's title and full body, and delivers to you only the relevant results.

You can black list sources, domains, terms in the article. The advantage of this approach is that you can look through attention grabbing headlines and find fresh information that matters to you. You could also optimize for precision or for recall...

a) appear in the HN RSS feed AND b) that don't appear in the r/politics feed

In order to do set operations on URLs reliably, we also solve the issue of canonicalization (resolving redirects, removing tracking markers, etc.).

What you're looking for can be found in HN, and is appreciated by many people, but does perhaps need a bit of support and encouragement.

EDIT: Got it. Thanks. (So no one double-sends)

To me, all they needed were a small handful of dedicated posters/commenters (and to be open, i'm looking at you lobste.rs) to start to catch on; i'm very much reminded of the story of the reddit founders sockpuppeting to make the site not look like a ghost town; this kind of 'forced' activity doesn't seem to be necessary for long until the site would take on it's own life.

I'm not near as focused as you are, but I understand your discontent. I have two suggestions.

One, the easy thing (easy as in less work and commitment), which a few others have suggested. Help make HN better, make it what you want. It's probably already closest to what you want, among the alternatives.

Read /newest and upvote good content and good comments. I personally rarely downvote bad comments, I just avert my eyes. I do flag spam, it's like picking up litter. But downvoting is an option, and part of the deal. Just don't "be a downvoter," it will probably make you feel worse, and make reading HN like a burden. Don't carry a cross, just watch the parade. (Worst metaphor you'll read today.)

Write good comments. Respond seriously and helpfully to the occasional lame or hostile comment. Be the change you want to see, and all that.

But mainly, get good at ignoring what you don't like. There's still a lot to like.

Two ... this will take some work and time. You can do it in parallel with HN, or wherever you go from HN. Curate your own discussion group. Look around in your current social, academic and work circle, and talk to a small handful (less than five-ish, more than one), who you think might be interested in discussing what you're interested in. Establish some broad but focused discussion topics.

Set up a mechanism for you all to discuss privately. If you're all local, beer is a great mechanism. Otherwise the easiest and simplest thing would be email, and you should probably stop at that; don't focus on the tools (fun though they are, especially the beer), start discussing as quickly as possible; like, this afternoon. Really, this afternoon. If you like, one of you can maintain a forwarding address, so you don't have to each maintain lists. But you don't need that this afternoon, let that emerge naturally. I've been a member of exactly such a list for ... 15 years? Dayam.

Every once in awhile, invite a new individual in. Do it slowly and deliberately. Don't obsess or agonize over whether someone is right for the group; if you thought of them, they probably are. If they aren't, they'll stop participating. New people will change the dynamics and focus, and that's a good thing. Just do it slowly, not as a focus.

Be generous and engaging with your fellow list members. Accept heat, and let it dissipate quickly. The fewer rules you have, the less they'll be broken.

Over the years, your group might grow to five, or ten, or a hundred, it's up to you.

As you communicate with people you know personally (or online personally), there's a danger that the group might take on some social aspects. You may even become friends with some or all of them. That's a risk that you'll have to take. It's not so bad. :)

I think this advice is crucial. I try to follow it.

And I see a fundamental conflict in your criteria: You want a less "social" site. You also want less fighting but with good discussion. Generally speaking, people who have the social savvy to encourage polite discourse are going to have some interest in social stuff. It is the nature of the beast.

The good thing is that there are great (just IMHO).

The bad thing is that, not everything fits in the question-and-answer scheme. There are many interesting subject, which are:

open-ended,

or requiring polling,

or - brainstorming.

Also, not every topic is covered, but it that case you can create your own SE site (with http://area51.stackexchange.com/); just it takes time and effort to gather the critical mass (at least 200, usually - much more).

http://programmers.stackexchange.com/

http://cs.stackexchange.com/

http://cstheory.stackexchange.com/

http://scicomp.stackexchange.com/

http://math.stackexchange.com/

I could even "gamify" it and the players could earn points for passing tests to prove their knowledge, ultimately earning an "achievement" for mastering the craft at various levels of proficiency.

We'll allow a diverse range of players to apply and even come live there, maybe right out of high school. I'm thinking "friendiversity" for the name, because you'll make friends and there is diversity there, what do you think?

Not so clear cut. I, too, am sick of the gadget wars infesting every single discussion. Unfortunately, the inanities resulting from this modern day religious war are pervasive and unavoidable. Let's call it Shoopy's Law: Any topic that involves Apple, Google, Microsoft, ${CORPORATE_ENTITY}, no matter how superficial, is bound to devolve into religious flamewars about nonsequitur gadgety things like framerate stutters and marketplace app metrics.

Add the right subreddits and filter out sites, posters, and keywords to refine it into exactly the thing you want.

The text /prog/ board has some interesting discussion, but there's a lot of "Have you read your SICP today?".

And both those boards have some deeply NSFW content.

Some of the other chans are better, but still risky and with a lot less traffic.

There is a great bit of copypasta about installing Linux though.

Competition is always good for the consumer.

This has never made sense to me. On some subreddits that focus on disciplines (like AskHistorians among many others), you can easily get downvoted (ie "dissapeared") just by asking a question, let alone answering one. Other times you'll won't get upvoted nor downvoted when you've actually contributed to the discussion. In these instances I lurk even when I can contribute because it's just not worth it.

(1) Lots of strategic downvoting. That's why no thread stays above 75%. It happens to comments as well as threads. Some of it is spammers trying to improve their own ranking, and some is probably just trolls.

(2) "Stalker" downvoting, which is when a person retaliates to a comment by downvoting that person's contributions en masse.

I wonder if they've put any machine learning muscle behind this. Is it easy to fix, or is it just a hard problem?

In theory it may be a good idea, but in practice it's essentially a "CENSOR THIS POST" button that people use to get rid of content they don't agree with.

I've also noticed that negative points on a post completely destroys credibility despite cited evidence or a well thought out argument. I've seen identical comments on similar posts in the past where the first few voters determined the post's fate in that if the first few voters voted it down, the post would receive flames and even more downvotes, but if it was initially upvoted, more people would upvote it and it would receive well thought out replies.

Probably the most horrifying part of the downvote is that all it takes is 51% of voters to dislike your post to destroy it. For example, all it takes is 51% of viewers on the politics subreddit to turn the the /r/politics frontpage into a pseudo fox news where every single post praises democrats and slanders republicans.

It strikes me as disgusting that some people actually consider reddit to be a good place to have a controversial discussion when it's so pathetically easy to censor opinions that you don't agree with.

That having been said, I mostly read the programming section of reddit; I'm prepared to believe standards might be lower in e.g. the politics sections.

There are also some small safeguards in place against (2). If you literally go to someone's user page, and start downvoting every post, reddit will start ignoring your votes towards that user in actual score calculations.

This is why downvots (or upvotes) are not registered if you vote from the user-page.

As to the ops questions: you should look into creating a list of subreddits that cover these things you're interested in.

http://www.reddit.com/r/coding

Then follow the sidebar links for related stuff.

1-There is bound to be heated arguments whenever two rational people with different circumstances & a different thought process are asked to opine on any issue with less than an obvious resolution. If you're uncomfortable watching a rigorous back & forth. Try North Korea.

2- You do realize you're asking a question of a social nature. you're contributing to making HN less of a technical forum.

3- Nobody forces you to read comments. The community does a pretty good job both down-voting comments rife with negativity or without any value. (maybe even this one)

I'm sorry, but it seems to me you're trolling.

1) There is a very big difference between a heated argument and an argument. Maybe it's just a side effect of the medium we are using, but everyone seems to have so much trouble being civil. For example, instead of suggesting that OP go look to North Korea, you could have said "For me, the heated discussions are a good thing" or something similar.

2) OP realizes that HN isn't a technical forum, and isn't asking for it to change.

3) Comments are a massive part of the value that HN provides, and even if OP were to stop reading the comments, he'll still see the same mix of articles on the front page.

[1]: http://www.reddit.com/r/technology/comments/135b5z/yes_the_f...