In the context of hosting/cloud network what would probably be load balancer / reverse proxy which has the most state/sessions. And incidentally, LBs are probably the piece of dedicated network iron most ripe to be replaced with a bunch of commodity servers and some software.
Regardless, statefulness is bad of course, and less of it you have, the better you are off - we've seen this time and again: share nothing arch[1], Amdahl's law[2], locks-vs-RCU[3], memorizing tcp flows-vs-IP header hash[4,5], etc... Being able to make decision on current packet/memory access here and now, independently, without having to keep track of other flows/connections/sessions/threads beats alternative approaches every time [6].
[6] Intel's heroic effort to maintain cache coherency at all cost is a notable exception, although opinions vary wildly on whether this is actually the best thing to do, and how long they will be able to sustain it.
Regardless, statefulness is bad of course, and less of it you have, the better you are off - we've seen this time and again: share nothing arch[1], Amdahl's law[2], locks-vs-RCU[3], memorizing tcp flows-vs-IP header hash[4,5], etc... Being able to make decision on current packet/memory access here and now, independently, without having to keep track of other flows/connections/sessions/threads beats alternative approaches every time [6].
[1] https://en.wikipedia.org/wiki/Shared_nothing_architecture
[2] https://en.wikipedia.org/wiki/Amdahl's_law
[3] http://www.rdrop.com/users/paulmck/RCU/hart_ipdps06.pdf
[4] https://tools.ietf.org/html/rfc2992
[5] http://www.juniper.net/techpubs/en_US/junos13.2/topics/conce...
[6] Intel's heroic effort to maintain cache coherency at all cost is a notable exception, although opinions vary wildly on whether this is actually the best thing to do, and how long they will be able to sustain it.