> If you could cheaply send stuff to random people to make them pay huge fees when they get something they didn't even order, that would be quite bad...

You can just refuse to accept the item, and it'll be returned to the sender or destroyed.