Because the capital market is opaque and corrupt. Execs aren't selected based on their ability to perform, they're selected based on how acceptable they are to the upper classes that control the capital. Even for those (few) who work their way up the ranks, jumping up levels will be based on politics and ability to be perceived as upper class and not based on performance (which is basically impossible to measure in a managerial role unless you want to give all of the credit of the worker's output to the manager).
So shareholders are dumb and execs are overpaid because they are good at duping them out of their money? Wouldn't a company that does NOT overpay their execs outcompete a company that does?
Shareholders are not dumb, but they have created a rigged game of castles in the sky paid for by greater fools. They look for execs that will successfully con the next bag holder, not people who create innovation, great products, healthy teams...
There's a lot of backscratching going on at that level as well. As an example, more often than you would think, VCs will invest in the parasitic children of potential LPs despite their terrible startup idea because they know it will make raising their own funds easier.
So the CEO is the face of the company, and companies are willing to pay a lot for a face that projects a certain image? If that's the case, then couldn't you argue that it's worth a lot for the company to pay to improve its brand image?
It's worth a lot to the shareholders to manipulate the stock price upwards, that's of no benefit to the customers though. Quite the contrary, choosing executives based on how well they sell equity opposed to how well they can operate a company produces worse results for the market (of goods/services, not of capital -- that's the distinction).
You're confusing the market for equity with the market for the goods and services produced by the company. The shareholders are optimizing for the former at the expense of the latter.