TLDR: "Nginx will fail by refusing or dropping connections, Caddy will fail by slowing everything down"
To me it seem that Caddy suffer from BufferBloat.
Under heavy congestion the goodput (useful throughput) will drop to 0 because client will start timing-out before the server get a chance to respond.
(I implemented Reno in college, but times have changed)
The nice thing about Caddy's failure mode is that the server won't give up; the server has no control over if or when a client will time out, so it I never felt it made much sense to optimize for that.
thanks for considering the feedback!
What i was trying to explain is Server not giving up is never a good thing under congestion.
This is why in ip switches they drop packet when the outbound queue is full. And letting the caller retry sending the packet.
When there is no congestion it’s ok for the server to not timeout at all and wait for the queue to drain and let the client decide when to close the tcp connection.
but when there is congestion you don’t want the queue inside the server to get too long. Because the server ( including caddy) might be behind a tcp load-balancer so it’s better for the client to queue it’s request inside another Caddy instance that is less busy.
To me it seem that Caddy suffer from BufferBloat. Under heavy congestion the goodput (useful throughput) will drop to 0 because client will start timing-out before the server get a chance to respond.
Caddy should use an algorithm similar to : https://github.com/Netflix/concurrency-limits
Basically check what was the best request latency, and decrease concurrency limit until latency stop improving.