Subject: dovecot-sieve: Just discovered imap_sieve/sieve_imapsieve is not set up to work with virtual mailboxes.

From: Thurgood Angelou <[email-masked]>
Date: Thu, 21 Sep 2017 11:56:14 +0100

Package: dovecot-sieve
Version: 1:2.2.32-2
Severity: normal

Dear Maintainer,

I've just discovered a bug where the sieve plugin (especially IMAP) will not work with a virtual mailbox. I recently found this out when testing a virtual mailbox setup and I use imap_sieve for SPAM filtering so to lose that would be painful.

This is what is reported in dovecot's debug log...

Sep 21 10:53:01 imap([email-masked]): Panic: file mail-index-map.c: line 549 (mail_index_map_lookup_seq_range): assertion failed: (first_uid > 0)
Sep 21 10:53:01 imap([email-masked]): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x9f0a2) [0x7f4226fd50a2] -> /usr/lib/dovecot/libdovecot.so.0(+0x9f19a) [0x7f4226fd519a] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f4226f65cf8] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map_lookup_seq_range+0x120) [0x7f422733dae0] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xe8d1d) [0x7f422734cd1d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_lookup_seq+0xf) [0x7f4227350e9f] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x82aa) [0x7f422678a2aa] -> /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so(+0x708c) [0x7f422657d08c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x52) [0x7f42272adca2] -> dovecot/imap(+0x10084) [0x556502adb084] -> dovecot/imap(command_exec+0x5c) [0x556502ae7d9c] -> dovecot/imap(+0x1b322) [0x556502ae6322] -> dovecot/imap(+0x1b3bc) [0x556502ae63bc] -> dovecot/imap(client_handle_input
 +0x18d) [0x556502ae677d] -> dovecot/imap(client_input+0xac) [0x556502ae6ccc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f4226fea742] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12e) [0x7f4226febd8e] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x36) [0x7f4226fea7d6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f4226fea988] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f4226f70353] -> dovecot/imap(main+0x329) [0x556502ad9139] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f4226bb92e1] -> dovecot/imap(_start+0x2a) [0x556502ad92ca]
Sep 21 10:53:01 imap([email-masked]): Fatal: master: service(imap): child 14102 killed with signal 6 (core dumps disabled)

I switched back to a regular setup for now but I want to switch to a virtual mailbox in the future and I can't do that with imap_sieve broken like this.

Is it fixable? I really want to use IMAP sieve in absense of dovecot_antispam which does not work anymore on this debian version yet I can't do it.

-- Package-specific info:

-- System Information:
Debian Release: 8.2
  APT prefers oldstable-updates
  APT policy: (500, 'oldstable-updates'), (500, 'unstable'), (500, 'testing'), (500, 'oldstable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.16.0-4-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages dovecot-sieve depends on:
ii  dovecot-core  1:2.2.32-2
ii  libc6         2.24-17
ii  ucf           3.0030

dovecot-sieve recommends no packages.

dovecot-sieve suggests no packages.

Versions of packages dovecot-sieve is related to:
ii  dovecot-core [dovecot-common]  1:2.2.32-2
pn  dovecot-dbg                    <none>
pn  dovecot-dev                    <none>
ii  dovecot-gssapi                 1:2.2.32-2
ii  dovecot-imapd                  1:2.2.32-2
ii  dovecot-ldap                   1:2.2.32-2
ii  dovecot-lmtpd                  1:2.2.32-2
ii  dovecot-managesieved           1:2.2.32-2
ii  dovecot-mysql                  1:2.2.32-2
pn  dovecot-pgsql                  <none>
pn  dovecot-pop3d                  <none>
ii  dovecot-sieve                  1:2.2.32-2
pn  dovecot-sqlite                 <none>

-- no debconf information

---

From: Apollon Oikonomopoulos <[email-masked]>
Date: Thu, 21 Sep 2017 14:47:25 +0300

Control: tags -1 + moreinfo upstream

[Forwarding this to the Dovecot mailing list, just in case someone can help]

Hi,

Thanks for the report! See my comments inline.

On 11:56 Thu 21 Sep     , Thurgood Angelou wrote:
> Package: dovecot-core
> Version: 1:2.2.32-2
>
> I've just discovered a bug where the sieve plugin (especially IMAP) 
> will not work with a virtual mailbox. I recently found this out when 
> testing a virtual mailbox setup and I use imap_sieve for SPAM 
> filtering so to lose that would be painful.
> 

After mixing in the debug symbols, this backtrace becomes:

/usr/lib/dovecot/libdovecot.so.0(+0x9f0a2) [0x7f4226fd50a2]
    default_fatal_finish at ./src/lib/failures.c:195
/usr/lib/dovecot/libdovecot.so.0(+0x9f19a) [0x7f4226fd519a]
    ?? at ./src/lib/failures.c:670
/usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f4226f65cf8]
    i_fatal at ./src/lib/failures.c:280
/usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map_lookup_seq_range+0x120) [0x7f422733dae0]
    mail_index_map_lookup_seq_range at ./src/lib-index/mail-index-map.c:550
/usr/lib/dovecot/libdovecot-storage.so.0(+0xe8d1d) [0x7f422734cd1d]
    tview_lookup_seq_range at ./src/lib-index/mail-index-transaction-view.c:178
/usr/lib/dovecot/libdovecot-storage.so.0(mail_index_lookup_seq+0xf) [0x7f4227350e9f]
    mail_index_lookup_seq at ./src/lib-index/mail-index-view.c:522
/usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x82aa) [0x7f422678a2aa]
    virtual_mail_set_uid at ./src/plugins/virtual/virtual-mail.c:213                                              
/usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so(+0x708c) [0x7f422657d08c]
    imap_sieve_mailbox_transaction_run at ./pigeonhole/src/plugins/imapsieve/imap-sieve-storage.c:752 (inlined by: imap_sieve_mailbox_transaction_commit at ./pigeonhole/src/plugins/imapsieve/imap-sieve-storage.c:807)
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x52) [0x7f42272adca2]
    mailbox_transaction_commit_get_changes at ./src/lib-storage/mail-storage.c:2083
dovecot/imap(+0x10084) [0x556502adb084]
    cmd_copy_full at ./src/imap/cmd-copy.c:146
dovecot/imap(command_exec+0x5c) [0x556502ae7d9c]
    command_exec at ./src/imap/imap-commands.c:200
dovecot/imap(+0x1b322) [0x556502ae6322]
    client_command_input at ./src/imap/imap-client.c:1088
dovecot/imap(+0x1b3bc) [0x556502ae63bc]
    client_command_input at ./src/imap/imap-client.c:1150
dovecot/imap(client_handle_input+0x18d) [0x556502ae677d]
    client_handle_input at ./src/imap/imap-client.c:1203
dovecot/imap(client_input+0xac) [0x556502ae6ccc]
    client_input at ./src/imap/imap-client.c:1249
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f4226fea742]
    io_loop_call_io at ./src/lib/ioloop.c:600
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12e) [0x7f4226febd8e]
    io_loop_handler_run_internal at ./src/lib/ioloop-epoll.c:223
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x36) [0x7f4226fea7d6]
    io_loop_handler_run at ./src/lib/ioloop.c:648
/usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f4226fea988]
    io_loop_run at ./src/lib/ioloop.c:622
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f4226f70353]
    master_service_run at ./src/lib-master/master-service.c:667
dovecot/imap(main+0x329) [0x556502ad9139]
    main at ./src/imap/main.c:491                                                         
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f4226bb92e1]
    __libc_start_main at ../csu/libc-start.c:325
dovecot/imap(_start+0x2a) [0x556502ad92ca]
    start at ??:0                                             

> ii  dovecot-sieve                  1:2.2.32-2

Could you also include the output of doveconf -n?

Regards,
Apollon

---

From: Thurgood Angelou <[email-masked]>
Date: Thu, 21 Sep 2017 17:09:32 -0700

Hi... The output of my doveconf -n is as folllows.

# 2.2.32 (dfbe293d4): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.20 (7cd71ba)
doveconf: Warning: /etc/dovecot/dovecot.conf line 105: Global setting
mail_plugins won't change the setting inside an earlier filter at
/etc/dovecot/conf.d/15-lda.conf line 47 (if this is intentional, avoid this
warning by moving the global setting before /etc/dovecot/conf.d/15-lda.conf
line 47)
doveconf: Warning: /etc/dovecot/dovecot.conf line 105: Global setting
mail_plugins won't change the setting inside an earlier filter at
/etc/dovecot/conf.d/15-lda.conf line 47 (if this is intentional, avoid this
warning by moving the global setting before /etc/dovecot/conf.d/15-lda.conf
line 47)
# OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.2 ext4
auth_default_realm = othersider92.com
auth_mechanisms = plain login
first_valid_uid = 100
log_path = /var/log/dovecot.log
mail_location = maildir:/var/mail/%d/%n
mail_plugins = " virtual"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date index ihave
duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    auto = create
    special_use = \Drafts
  }
  mailbox Junk {
    auto = create
    autoexpunge = 1 weeks
    special_use = \Junk
  }
  mailbox Promotions {
    auto = subscribe
  }
  mailbox Sent {
    auto = create
    special_use = \Sent
  }
  mailbox Trash {
    auto = create
    autoexpunge = 30 days
    special_use = \Trash
  }
  prefix =
  separator = /
  type = private
}
namespace virtual {
  disabled = yes
  hidden = yes
  list = no
  location = virtual:/var/mail/virtual
  prefix = virtual/
  separator = /
  type = private
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  imapsieve_mailbox1_before = file:/etc/sieve/report-spam.sieve
  imapsieve_mailbox1_causes = COPY
  imapsieve_mailbox1_name = Junk
  imapsieve_mailbox2_before = file:/etc/sieve/report-ham.sieve
  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_from = Junk
  imapsieve_mailbox2_name = *
  imapsieve_mailbox3_before = file:/etc/sieve/report-spam-promo.sieve
  imapsieve_mailbox3_causes = COPY
  imapsieve_mailbox3_name = Promotions
  imapsieve_mailbox4_before = file:/etc/sieve/report-ham-promo.sieve
  imapsieve_mailbox4_causes = COPY
  imapsieve_mailbox4_from = Promotions
  imapsieve_mailbox4_name = *
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_after = /etc/sieve/conf.d/after
  sieve_before = /etc/sieve/conf.d/before
  sieve_global_extensions = +editheader +vnd.dovecot.pipe
  sieve_pipe_bin_dir = /usr/local/bin
  sieve_plugins = sieve_extprograms sieve_imapsieve
}
protocols = imap lmtp
service auth-worker {
  user = vmail
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0666
    user = postfix
  }
  unix_listener auth-userdb {
    mode = 0600
    user = vmail
  }
  user = dovecot
}
service imap-login {
  inet_listener imap {
    port = 0
  }
  inet_listener imaps {
    port = 993
    ssl = yes
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    mode = 0600
    user = postfix
  }
}
ssl = required
ssl_cert = </etc/ssl/mail/ssl.cert
ssl_client_ca_dir = /etc/ssl/certs
ssl_key =  # hidden, use -P to show it
userdb {
  args = uid=vmail gid=vmail home=/var/mail/%d/%n
  driver = static
}
protocol lmtp {
  info_log_path = /var/log/dovecot-lmtp.log
  mail_plugins = " virtual"
}
protocol lda {
  mail_plugins = " sieve virtual"
}
protocol imap {
  mail_plugins = " virtual imap_sieve"
}

I hope this helps.

On Thu, Sep 21, 2017 at 4:47 AM, Apollon Oikonomopoulos <[email-masked]>
wrote:

> > Sep 21 10:53:01 imap([email-masked]): Panic: file
> mail-index-map.c: line 549 (mail_index_map_lookup_seq_range): assertion
> failed: (first_uid > 0)
> > Sep 21 10:53:01 imap([email-masked]): Error: Raw backtrace:
> /usr/lib/dovecot/libdovecot.so.0(+0x9f0a2) [0x7f4226fd50a2] ->
> /usr/lib/dovecot/libdovecot.so.0(+0x9f19a) [0x7f4226fd519a] ->
> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f4226f65cf8] ->
> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map_lookup_seq_range+0x120)
> [0x7f422733dae0] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xe8d1d)
> [0x7f422734cd1d] -> /usr/lib/dovecot/libdovecot-st
> orage.so.0(mail_index_lookup_seq+0xf) [0x7f4227350e9f] ->
> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x82aa)
> [0x7f422678a2aa] -> /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so(+0x708c)
> [0x7f422657d08c] -> /usr/lib/dovecot/libdovecot-st
> orage.so.0(mailbox_transaction_commit_get_changes+0x52) [0x7f42272adca2]
> -> dovecot/imap(+0x10084) [0x556502adb084] -> dovecot/imap(command_exec+0x5c)
> [0x556502ae7d9c] -> dovecot/imap(+0x1b322) [0x556502ae6322] ->
> dovecot/imap(+0x1b3bc) [0x556502ae63bc] -> dovecot/imap(client_handle_inp
> ut
> >  +0x18d) [0x556502ae677d] -> dovecot/imap(client_input+0xac)
> [0x556502ae6ccc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52)
> [0x7f4226fea742] -> /usr/lib/dovecot/libdovecot.so
> .0(io_loop_handler_run_internal+0x12e) [0x7f4226febd8e] ->
> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x36)
> [0x7f4226fea7d6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
> [0x7f4226fea988] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
> [0x7f4226f70353] -> dovecot/imap(main+0x329) [0x556502ad9139] ->
> -> dovecot/imap(_start+0x2a) [0x556502ad92ca]
>
>     imap_sieve_mailbox_transaction_run at ./pigeonhole/src/plugins/imapsieve/imap-sieve-storage.c:752
> (inlined by: imap_sieve_mailbox_transaction_commit at
> ./pigeonhole/src/plugins/imapsieve/imap-sieve-storage.c:807)