Which is why the key should be physically given to the system when it is started and then only stored in memory. The key file should not be available on any network-attached machine. Of course there's still potential for exploits in this scenario, but it does help minimize the attack surface.
Many emails are signed with DKIM now, which does help with verifiability.
> but I would hope this is still stored encrypted
Encryption is pointless when the keys for decryption are on the same server. Given their hack in 2012, I doubt there's any protection at all.