Depends very much on the hosts and network; but yeah, I have two different aliases for one host in such cases: one for data-trafficking, another for latency-sensitive applications (e.g. interactive jobs). This way, there are two SSH connections up, and while they're still contending for the link, it's quite an usable setup (in other words, I'm letting the TCP/IP stack handle the contention, rather than the SSH multiplexer).

I've found it just tends to have the primary connection die, and never tries to reconnect. ServerKeepAlives or what have you don't seem to help either - the link goes dead, and I won't be able to connect any other sessions because SSH will just keep on routing them into the control master.

I have had this problem, although fairly rarely. I have the following in my ~/.ssh/config:

    ControlPath /tmp/ssh_mux_%h_%p_%r

This sets the path of the control file used to share the connection. If it ever hangs, I can just delete the file. But in practice I found this doesn't happen often and I appreciate the speed boost I get from connection sharing.

one minor annoyance is that there's a max limit to the ControlPath string (seemingly due to there being a max path length for Unix Sockets) which I've occasionally hit when connecting to hosts with very long hostnames (AWS default hostnames can sometimes hit it, IIRC).

Also note that the docs recommend against using publicly accessible dirs such as /tmp/ for storing your mux sockets. I'm not sure of the exact threat (maybe just info leakage about what hosts you're connected to, since the socket permissions themselves are strict), but I use ~/.ssh/mux/ for mine.

Good point about the permission. I'm not doing this on a shared host anyway, so no else has access to that directory, but good to keep in mind.

Using autossh for establishing the master connection helps immensely here - if it dies, it will automagically reconnect.