We had really bizzare problems with Varnish ~6 years ago even under mild load (hundreds or requests per second). Like it would randomly let all the requests through to our origin servers, bringing them down. We struggled for months, multiple devops/sysadmins looked into it, couldn't figure it out.
That sounds like the dreaded "Long TTL hit-for-pass object" issue.
The situation happens when an object which should be cached gets hit with a request which causes 'pass' (typically because of headers tested in builtin.vcl) and then VCL (by accident) setting the long TTL for caching on the "hit-for-pass" object.
It's an ugly corner case which we have struggled a lot with finding a solution for, which doesn't have worse side-effects than the problem we are trying to fix.
I think it is mostly a solved problem from Varnish 5.
But when it worked, it was fast.