OKCupid would probably get better matches if it dropped the user submitted weights and ideal match preferences completely and instead used its database of people in relationships as training data for a proper machine learning algorithm.
The current approach is entirely oriented to give people what they think is important and what people think they want. It would probably be better to derive that from existing relationships (successes).
I would be a little surprised if people at OKCupid hadn't already thought about this. Whether there is actually any momentum to change the core matching mechanic or not remains to be seen.
I agree, I wouldn't be surprised if the weights people manually assign to the questions have very little correlation with the success of their relationships. People probably suck at knowing what they want.
That's not to say they're meaningless though; e.g., if someone puts "mandatory" for all his questions, that definitely says something about his personality, and should be used as a feature in the ML algorithm used for matching.
People may suck at knowing what they want, but I think it's more that people do not necessarily know what criteria they should want to select given their goals for finding someone.
Another problem is that a lot of users are probably more casual about the weights, such as selecting mandatory. (It's common for people to select an answer and then mark that same answer as unacceptable in a match, for questions in which it makes no sense for them to do that.)
If all OKC knows is "did we get these two people to enter a relationship" (because they know nothing about how successful the relationship was) they're maximizing for something pretty different from what most users want.
I know a couple of OKC founders. They are hardcore math/statistics geeks. I'm sure they realize the quality of their service matters more than making a bit of money quickly.
They are doing pretty well, considering their competitors have ads on TV.
They have produced some really cool blog posts, which I enjoyed reading. The site itself is fun, I've had 2 accounts on there - 1 I used in London, 1 in Shanghai.
I got some profile views and a couple of conversations, but never met anyone through it; with the exception of seeing my co-workers on it, in Shanghai.
In each city, within a few weeks of creating a profile, I've been in a relationship, but not through the site.
The current approach is entirely oriented to give people what they think is important and what people think they want. It would probably be better to derive that from existing relationships (successes).
I would be a little surprised if people at OKCupid hadn't already thought about this. Whether there is actually any momentum to change the core matching mechanic or not remains to be seen.