Subject: amavisd-new: Amavis database corrupted infrequently - Suicide in child_init_hook: BDB can't connect db env

From: Dominik <[email-masked]>
Date: Mon, 06 Dec 2021 16:56:58 +0100

Package: amavisd-new
Version: 1:2.11.1-5
Severity: important

Dear Maintainer,

after upgrading to bullseye, I see a database corruption every few days, rendering amavis unusable until the database is built up again. To recover from the error, something like this is needed:

	service amavis stop
	mv /var/lib/amavis/db /var/lib/amavis/db~
	mkdir /var/lib/amavis/db
	chown amavis:amavis /var/lib/amavis/db
	chmod o-rx /var/lib/amavis/db
	service amavis start

Unfortunatelly, I'm not able to tell the exact trigger for the database corruption. Maybe it is related to the system being somewhat memory limited, which sometimes leads to swapping and slow processing times.

In one thread, switching to a redis database is recommended:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=214218

The logs look for examlple like this:
Dec  6 13:35:53 emailserver amavis[1559665]: (1559665-02-7) (!!)TROUBLE in check_mail, but must continue (1): update_snmp FAILED: update_snmp_variables: BDB S db_cursor: BDB0061 PANIC: Das Argument ist ungültig, . at (eval 91) l
ine 107.
Dec  6 13:35:53 emailserver amavis[1559664]: (1559664-03-2) (!!)TROUBLE in check_mail, but must continue (1): update_snmp FAILED: update_snmp_variables: BDB S c_get: BDB0060 PANIC: fatal region error detected; run recovery, . at
 (eval 91) line 140.
Dec  6 13:35:53 emailserver amavis[1559665]: (1559665-02-7) (!!)TROUBLE in process_request: register_proc: BDB N db_cursor: BDB0060 PANIC: fatal region error detected; run recovery, . at (eval 91) line 220.
Dec  6 13:35:53 emailserver amavis[1559665]: (1559665-02-7) (!)Requesting process rundown after fatal error
Dec  6 13:35:53 emailserver amavis[1559664]: (1559664-03-2) (!!)TROUBLE in process_request: register_proc: BDB N db_cursor: BDB0060 PANIC: fatal region error detected; run recovery, . at (eval 91) line 220.
Dec  6 13:35:53 emailserver amavis[1559664]: (1559664-03-2) (!)Requesting process rundown after fatal error
Dec  6 13:35:54 emailserver amavis[1559665]: (1559665-02-7) (!)TempDir removal: tempdir is to be PRESERVED: /var/lib/amavis/tmp/amavis-20211206T132604-1559665-foq2xpsA
Dec  6 13:35:56 emailserver amavisd-new[1559665]: register_proc: BDB N db_cursor: BDB0060 PANIC: fatal region error detected; run recovery, . at (eval 91) line 220.
Dec  6 13:35:56 emailserver amavis[1559665]: (1559665-02-7) (!)_DIE: register_proc: BDB N db_cursor: BDB0060 PANIC: fatal region error detected; run recovery, . at (eval 91) line 220.
Dec  6 13:35:56 emailserver amavisd-new[1559664]: register_proc: BDB N db_cursor: BDB0060 PANIC: fatal region error detected; run recovery, . at (eval 91) line 220.
Dec  6 13:35:56 emailserver amavis[1559665]: (1559665-02-7) (!)Amavis::END: DB unregistering failed: 
Dec  6 13:35:56 emailserver amavis[1559664]: (1559664-03-2) (!)TempDir removal: tempdir is to be PRESERVED: /var/lib/amavis/tmp/amavis-20211206T132604-1559664-xiTb_xfl
Dec  6 13:35:56 emailserver amavis[1559664]: (1559664-03-2) (!)_DIE: register_proc: BDB N db_cursor: BDB0060 PANIC: fatal region error detected; run recovery, . at (eval 91) line 220.
Dec  6 13:35:57 emailserver amavis[1559664]: (1559664-03-2) (!)Amavis::END: DB unregistering failed: 
Dec  6 13:36:47 emailserver amavis[1559785]: (!!)TROUBLE in child_init_hook: BDB can't connect db env. at /var/lib/amavis/db: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery, No such file or directory. at (eval 91) line 338.
Dec  6 13:36:47 emailserver amavisd-new[1559785]: Suicide in child_init_hook: BDB can't connect db env. at /var/lib/amavis/db: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery, No such file or directory. at (eval 91) line 338.
Dec  6 13:36:48 emailserver amavisd-new[1559786]: Suicide in child_init_hook: BDB can't connect db env. at /var/lib/amavis/db: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery, No such file or directory. at (eval 91) line 338.
Dec  6 13:36:48 emailserver amavis[1559786]: (!!)TROUBLE in child_init_hook: BDB can't connect db env. at /var/lib/amavis/db: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery, No such file or directory. at (eval 91) line 338.
Dec  6 13:36:48 emailserver amavis[1559785]: (!)_DIE: Suicide in child_init_hook: BDB can't connect db env. at /var/lib/amavis/db: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery, No such file or directory. at (eval 91) line 338.
Dec  6 13:36:48 emailserver amavis[1559786]: (!)_DIE: Suicide in child_init_hook: BDB can't connect db env. at /var/lib/amavis/db: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery, No such file or directory. at (eval 91) line 338.
Dec  6 13:42:40 emailserver amavis[1559809]: (!!)TROUBLE in child_init_hook: BDB can't connect db env. at /var/lib/amavis/db: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery, No such file or directory. at (eval 91) line 338.

These TROUBLE / _DIE  messages continue until the database is built up again.

-- System Information:
Debian Release: 11.1
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 5.10.0-9-amd64 (SMP w/4 CPU threads)
Kernel taint flags: TAINT_SOFTLOCKUP
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages amavisd-new depends on:
ii  adduser                    3.118
ii  debconf [debconf-2.0]      1.5.77
ii  file                       1:5.39-3
ii  init-system-helpers        1.60
pn  libarchive-tar-perl        <none>
ii  libarchive-zip-perl        1.68-1
ii  libberkeleydb-perl         0.64-1+b1
ii  libconvert-tnef-perl       0.18-1.1
ii  libconvert-uulib-perl      1:1.5~dfsg-1+b3
pn  libdigest-md5-perl         <none>
ii  libio-stringy-perl         2.111-3
ii  libmail-dkim-perl          1.20200907-1
ii  libmailtools-perl          2.21-1
pn  libmime-base64-perl        <none>
ii  libmime-tools-perl         5.509-1
ii  libnet-libidn-perl         0.12.ds-3+b3
ii  libnet-server-perl         2.009-2
ii  libnet-snmp-perl           6.0.1-6
ii  libunix-syslog-perl        1.1-3+b3
ii  lsb-base                   11.1.0
ii  pax                        1:20201030-1
ii  perl [libtime-hires-perl]  5.32.1-4+deb11u2

Versions of packages amavisd-new recommends:
ii  altermime             0.3.10-12
ii  libnet-patricia-perl  1.22-1+b7
pn  ripole                <none>

Versions of packages amavisd-new suggests:
pn  apt-listchanges      <none>
ii  arj                  3.10.22-24
ii  cabextract           1.9-3
ii  clamav               0.103.3+dfsg-0+deb11u1
ii  clamav-daemon        0.103.3+dfsg-0+deb11u1
ii  cpio                 2.13+dfsg-4
pn  dspam                <none>
pn  lhasa                <none>
ii  libauthen-sasl-perl  2.1600-1.1
pn  libdbi-perl          <none>
ii  libmail-dkim-perl    1.20200907-1
pn  libnet-ldap-perl     <none>
pn  libsnmp-perl         <none>
pn  libzeromq-perl       <none>
pn  lzop                 <none>
ii  nomarch              1.4-4
pn  p7zip                <none>
pn  rpm                  <none>
ii  spamassassin         3.4.6-1
pn  unrar                <none>

-- Configuration Files:
/etc/amavis/conf.d/15-content_filter_mode changed:
use strict;
@bypass_virus_checks_maps = (
   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);
@bypass_spam_checks_maps = (
   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);
1;  # ensure a defined return

/etc/amavis/conf.d/50-user changed:
use strict;
$hdrfrom_notify_sender = "postmaster\@$mydomain";  #Des Weiteren möchten wir hier noch auf einen Fehler in den Paketen von Debian hinweisen. Dieser bewirkt, dass Reports über Virenmails nicht zugestellt werden können.  
$child_timeout=1200; #timeout child processes after 20 minutes (attempt to avoid messeages about time-out with clamav)
1;  # ensure a defined return

-- no debconf information