Neither server nor peer will notice if you perform the serious attack: exfiltrate the plaintexts or key material and keep the OMEMO/Signal dance around as kabuki theater :-)

For paranoid users there is also an option of running their own servers and clients (both Prosody and Conversations are open source). XMPP at least provides that option and if you choose it you can still communicate with the rest of the ecosystem.

No such luck for Whatsapp and Signal. And although they may be fair now I think putting all eggs in one company's basket is a bad idea in general (see e.g.: Google).