Subject: amavisd-new: Amavis fails while parsing headers
Date: Thu, 31 Mar 2016 12:38:21 +0200
Package: amavisd-new
Version: 1:2.10.1-1
Severity: minor
Dear Maintainer,
In short: A trojan was delivered as mail attachment to a user though amavis is
configured to block this kind of attachments.
Amavis claimed that it couldn't check the attachment because of a "BAD HEADER
SECTION", but the headers - afai can see - are well formed and interpretable.
The story: A user called me, that he had received several suspicious mails. I
moved those mails into quarantine and inspected them. The mails, three in
total, all contained a zipped file as attachment and this file contained
javascript (filename.js). Amavis is configured to block zip-files containing
..js-file, so I wondered why this mails had gone through.
A further investigation showed, that amavis claimed the attachments to be
unreadable due to a bad header. That is not uncommon, this special kind of
trojans is spreaded in vast amounts during the past few days, and it is in
fact mostly unreadable due to miscrafted headers. But in any other case I
investigated, not one of several tested mail clients was able to read the
attachment, no unzipper was able to unzip it.
But in this special case, all the three mails the user had got where perfectly
readable and unzippable with any tested client. So I wonder, why amavis claims
a bad header. Fortunately this user did not open the attachment and double
click the .js...
And now it becomes queer: I tried to attach one of these mails to my bug
report, but sending the report failed because... this time amavis was able
to read the headers, it was able to check the attachment, found the trojan
and subsequently blocked the outgoing mail.
So I will now resend this report without sample and hope to be able to upload
it later.
-- System Information:
Debian Release: 8.3
APT prefers stable
APT policy: (700, 'stable'), (500, 'stable-updates')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores)
Locale: LANG=de_DE, LC_CTYPE=de_DE (charmap=ISO-8859-1)
Dear maintainer,
since there is no upload function on bugs.debian.org (or I'm unable to find
it), I will try to resend the sample per mail - but now via my private
mailserver, which is not so hypercritical about trojans...
Best regards
herrmann ([email protected])
Today, when I take the sample and resend it to my mailserver, amavis will
identify it as trojan infested. But a few days ago my virusscanner did
not know this variant of Locky. To reproduce today, what had happened then, I
had to disable the virus scan temporarily.
When I now resend the sample to my mailserver, amavis will scan the
attachment, identify the javascript inside and, according to its
configuration, quarantine the message. That is exactly what should be expected
- but this is not, what happened a few days ago.
So I manipulated the sample a little bit: After line 53 (Content-Type:
application/x-compressed; x-unix-mode=0600) I inserted a new line containing
nothing but a binary zero and a newline 0x00 0x0a.
And now amavis stumbles into the trap! Now Amavis adds an "BAD HEADER
SECTION" alert and does not scan the attachment for illegitimate
attachments! Since I can't quarantine each "BAD HEADER SECTION" alert, this
would block a lot too much legitimate mail, the mail will be delivered to the
intended recipient, and he has no problem to open the attachment and to
infect his computer.
In my eyes that is a really serious problem! Once again: I can't quarantine
each "BAD HEADER SECTION" alert, this would block a lot too much legitimate
mail. And I can't trust virus scanners, since they are mostly one day behind
reality.
So the last barrage between the trojan and the user is a check of the
attachments for unwanted, illegitimate file formats. But each rascal can
bypass this barrage by simply crafting a few bytes of whitespace into the
mimeheaders.
I filed this bug with a minor severity, but I think it should be lifted to a
higher, to serious severity, because checking of unwanted attachments is a
really important feature of amavis, and if this feature can be bypassed in
such a simple way, it should be considered broken.
Acknowledgement sent
to Henrique de Moraes Holschuh <[email protected]>:
Extra info received and forwarded to list. Copy sent to Brian May <[email protected]>.
(Fri, 01 Apr 2016 12:21:04 GMT) (full text, mbox, link).
Subject: Re: Bug#819634: Could reproduce the problem
Date: Fri, 01 Apr 2016 09:17:58 -0300
severity 819634 important
thanks
On Fri, Apr 1, 2016, at 07:01, Georg Herrmann wrote:
> And now amavis stumbles into the trap! Now Amavis adds an "BAD HEADER
> SECTION" alert and does not scan the attachment for illegitimate
> attachments! Since I can't quarantine each "BAD HEADER SECTION" alert,
I don't know what we should do about this (but see below). The fact is
that different MUAs (and even mail spoolers) will tolerate and process
badly-formed headers and messages in very different ways. Letting bad
headers through is always going to be a security risk, one that can be
actively exploited.
The safe (from a security PoV) option is to refuse delivery of the
majority of the subclasses of bad headers and MIME malformed messages.
And while crazy non-canonical utf-8 encondings of something might not be
a big problem in a Subject header, it can be used in a MIME header to
bypass file name checks, etc.
Reading the amavisd-new source seems to be required to understand this
thing.
Look for $defang_bad_headers. And also, at the "CC_BADH" major categoty
for %final_destiny_maps_by_ccat, which is usually delivered through
$final_bad_header_destiny (which defaults to D_PASS upstream).
You can influence which bad header checks are going to be done:
# controls which header section tests are performed in
check_header_validity,
# keys correspond to minor contents categories for CC_BADH
$allowed_header_tests{lc($_)} = 1 for qw(
other mime syntax empty long control 8bit utf8 missing
multiple);
$allowed_header_tests{'utf8'} = 0; # turn this test off by default
Taken from the default contents of %smtp_reason_by_ccat, here are the
possible subcategories for CC_BADH:
CC_BADH.',1', 'id=%n - BAD HEADER: MIME error',
CC_BADH.',2', 'id=%n - BAD HEADER: nonencoded 8-bit character',
CC_BADH.',3', 'id=%n - BAD HEADER: contains invalid control
character',
CC_BADH.',4', 'id=%n - BAD HEADER: line made up entirely of
whitespace',
CC_BADH.',5', 'id=%n - BAD HEADER: line longer than RFC 5322
limit',
CC_BADH.',6', 'id=%n - BAD HEADER: syntax error',
CC_BADH.',7', 'id=%n - BAD HEADER: missing required header field',
CC_BADH.',8', 'id=%n - BAD HEADER: duplicate header field',
CC_BADH, 'id=%n - BAD HEADER',
(I assume the catchall at the end is "other").
amavisd-new will use/report the highest minor category that applies
(more than one might apply).
And the default for %defang_maps_by_ccat gives some helpful hints:
%defang_maps_by_ccat = (
# compatible with legacy %defang_by_ccat: value may be a scalar
CC_VIRUS, sub { c('defang_virus') },
CC_BANNED, sub { c('defang_banned') },
CC_UNCHECKED, sub { c('defang_undecipherable') },
CC_SPAM, sub { c('defang_spam') },
CC_SPAMMY, sub { c('defang_spam') },
# CC_BADH.',3', 1, # NUL or CR character in header section
# CC_BADH.',5', 1, # header line longer than 998 characters
# CC_BADH.',6', 1, # header field syntax error
CC_BADH, sub { c('defang_bad_header') },
);
You can let some CC_BADH through easily enough, while refusing others,
using $final_destiny_by_ccat, here's an example I found:
$final_destiny_by_ccat{ CC_BADH.',7' } = D_REJECT;
Please take a look at those options. You can use log level 2 to log the
result of check_header_validity() when a bad header is found. Using log
level 4, you might be able to check what check_for_banned_names() thinks
of the message that has the bad header.
IME, the only bad header syndrome one *has* to tolerate is 8-bit data
with missing enconding information, and that's how I use amavisd-new,
but YMMV. Please share your experience after you play with the above,
we might want to change the Debian defaults based on your report.
Also, you may want to ensure you changed @keep_decoded_original_maps to
always pass the full original message to your anti-virus. If this helps,
we need to change that default in Debian... the reasons why we left it
disabled by default might not hold true anymore...
> this would block a lot too much legitimate mail, the mail will be delivered to
Check which subtypes of CC_BADH are "too much legitimate mail", it might
be possible to avoid triggering on most of the
legitimate-but-created-or-mangled-by-piece-of-trash-software mail.
> I filed this bug with a minor severity, but I think it should be lifted
> to a higher, to serious severity, because checking of unwanted attachments is
Raised it to severity "important". Let's wait until we know more before
we raise it all the way up to "grave", please.
--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique de Moraes Holschuh <[email protected]>
Debbugs is free software and licensed under the terms of the GNU General
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.