We can kill, enslave, and psychologically torture them as well as rape them to make them pregnant (farmers and breeders do this routinely) and most people are OK with that. The real reason people don't like the sex part is because it's seen as disgusting, perverted and weird, not because of consent. Very similar to the real reason people don't like gays, not because they aren't good at having babies.
(I think polyamorous relationships should definitely be recognized by the government, though.)