The volume of spam is mind boggling. When we first implemented DMARC / DKIM etc our legit emails were 3% or so of all outbound mail per the reporting we got back! We have a somewhat trusted ___domain.
Some mail delivery systems start to notice that your ___domain name is being used as part of a lot of spam / bogus emails, so you can have perfect IP history / no spam and STILL start triggering some random filters (no big players but smaller protection product filters).
So the game must be absolutely never ending for everyone and the inbox a valuable target - especially now that unsolicited phone calls really do seem to get ignored these days - I feel like spammers killed the golden goose on phone calls and the telcos let them.
I will say DKIM / DMARC is working well, except google (which we now use for outbound) gives us transient SPF errors even though we are 100% using their IPs. Not sure why that is (ie, SPF failure on an IP that should clear)
As someone who works in email security I can tell you that it is a never ending cat and mouse game with a lot of false positives as to who is a mouse. Even our customers that have good ___domain rep, SPF, DKIM, DAMRC, etc still randomly get blocked by filters and large email providers. Part of the problem I see it those large providers and some of the filtering systems send back NDRs that basically provide no information as to why the message was blocked so trying to get your legitimate mail flow working again once someone has flagged you can take days to months as you randomly adjust all the knobs trying to figure out why they are blocking you. I'm sure they would say providing more informant NDRs would make it easier for the spammers to work around the filters. I don't know what the answer is but the current system sucks, it is a huge pita and a dumpster fire for the most part but it's what we have to work with. I'm sure someone will come up with another scheme like DKIM that will become another check box on the list of things you have to set up and once agian few will understand how to implement correctly.
I think we are just going towards centralized trust proxies.
Ie, if you are paying for google apps, have credit card on file, meet their rate limiting rules for outbound with SPF / Dkim etc then you are probably OK. Some random IP doing direct mail? Much less likely to be OK.
Given govt has done such a bad job in this space, these big corps are essentially picking up the slack / trust that you'd normally say govt was responsible for. Gives them a metric ton of power, and they don't tax so have no money to provide any corresponding service.
For your last point, check https://toolbox.googleapps.com/apps/checkmx/check. Chances are, google doesn't like your DNS server answers to its ANY requests, regardless for them complying to the RFCs.
Some mail delivery systems start to notice that your ___domain name is being used as part of a lot of spam / bogus emails, so you can have perfect IP history / no spam and STILL start triggering some random filters (no big players but smaller protection product filters).
So the game must be absolutely never ending for everyone and the inbox a valuable target - especially now that unsolicited phone calls really do seem to get ignored these days - I feel like spammers killed the golden goose on phone calls and the telcos let them.
I will say DKIM / DMARC is working well, except google (which we now use for outbound) gives us transient SPF errors even though we are 100% using their IPs. Not sure why that is (ie, SPF failure on an IP that should clear)