> Where I think FB should be sheepish about is allowing the optimization to go on directionless. The only goals are quantitive. Quality is a foreign concept. No one has asked the question "what should FB be." The answer is simply "popular."
Problem is: Quality itself is defined by quantitative measurements. It's not some separate product attribute. If something increases your business KPIs, it is ipso facto higher quality. Motivated only by the imperative to optimize these KPIs, a company will do so, believing that it is continually making the product better. Allowed to run its course, you end up with a Paperclip Maximizer [1].
Problem is: Quality itself is defined by quantitative measurements. It's not some separate product attribute. If something increases your business KPIs, it is ipso facto higher quality. Motivated only by the imperative to optimize these KPIs, a company will do so, believing that it is continually making the product better. Allowed to run its course, you end up with a Paperclip Maximizer [1].
1: https://wiki.lesswrong.com/wiki/Paperclip_maximizer