I was surprised to find that it is possible (though unlikely, as it has to happen in a very narrow window of time between two particular bytecodes) for a KeyboardInterrupt to cause a with block to exit without the __exit__ method of the context manager running.
https://bugs.python.org/issue29988