Subject: Re: Bug#876364: dovecot-sieve: Just discovered imap_sieve/sieve_imapsieve is not set up to work with virtual mailboxes.

From: Thurgood Angelou <[email-masked]>
Date: Thu, 21 Sep 2017 17:13:10 -0700

Hi... The output of my doveconf -n is as folllows.

# 2.2.32 (dfbe293d4): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.20 (7cd71ba)
doveconf: Warning: /etc/dovecot/dovecot.conf line 105: Global setting
mail_plugins won't change the setting inside an earlier filter at
/etc/dovecot/conf.d/15-lda.conf line 47 (if this is intentional, avoid this
warning by moving the global setting before /etc/dovecot/conf.d/15-lda.conf
line 47)
doveconf: Warning: /etc/dovecot/dovecot.conf line 105: Global setting
mail_plugins won't change the setting inside an earlier filter at
/etc/dovecot/conf.d/15-lda.conf line 47 (if this is intentional, avoid this
warning by moving the global setting before /etc/dovecot/conf.d/15-lda.conf
line 47)
# OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.2 ext4
auth_default_realm = othersider92.com
auth_mechanisms = plain login
first_valid_uid = 100
log_path = /var/log/dovecot.log
mail_location = maildir:/var/mail/%d/%n
mail_plugins = " virtual"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date index ihave
duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    auto = create
    special_use = \Drafts
  }
  mailbox Junk {
    auto = create
    autoexpunge = 1 weeks
    special_use = \Junk
  }
  mailbox Promotions {
    auto = subscribe
  }
  mailbox Sent {
    auto = create
    special_use = \Sent
  }
  mailbox Trash {
    auto = create
    autoexpunge = 30 days
    special_use = \Trash
  }
  prefix =
  separator = /
  type = private
}
namespace virtual {
  disabled = yes
  hidden = yes
  list = no
  location = virtual:/var/mail/virtual
  prefix = virtual/
  separator = /
  type = private
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  imapsieve_mailbox1_before = file:/etc/sieve/report-spam.sieve
  imapsieve_mailbox1_causes = COPY
  imapsieve_mailbox1_name = Junk
  imapsieve_mailbox2_before = file:/etc/sieve/report-ham.sieve
  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_from = Junk
  imapsieve_mailbox2_name = *
  imapsieve_mailbox3_before = file:/etc/sieve/report-spam-promo.sieve
  imapsieve_mailbox3_causes = COPY
  imapsieve_mailbox3_name = Promotions
  imapsieve_mailbox4_before = file:/etc/sieve/report-ham-promo.sieve
  imapsieve_mailbox4_causes = COPY
  imapsieve_mailbox4_from = Promotions
  imapsieve_mailbox4_name = *
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_after = /etc/sieve/conf.d/after
  sieve_before = /etc/sieve/conf.d/before
  sieve_global_extensions = +editheader +vnd.dovecot.pipe
  sieve_pipe_bin_dir = /usr/local/bin
  sieve_plugins = sieve_extprograms sieve_imapsieve
}
protocols = imap lmtp
service auth-worker {
  user = vmail
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0666
    user = postfix
  }
  unix_listener auth-userdb {
    mode = 0600
    user = vmail
  }
  user = dovecot
}
service imap-login {
  inet_listener imap {
    port = 0
  }
  inet_listener imaps {
    port = 993
    ssl = yes
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    mode = 0600
    user = postfix
  }
}
ssl = required
ssl_cert = </etc/ssl/mail/ssl.cert
ssl_client_ca_dir = /etc/ssl/certs
ssl_key =  # hidden, use -P to show it
userdb {
  args = uid=vmail gid=vmail home=/var/mail/%d/%n
  driver = static
}
protocol lmtp {
  info_log_path = /var/log/dovecot-lmtp.log
  mail_plugins = " virtual"
}
protocol lda {
  mail_plugins = " sieve virtual"
}
protocol imap {
  mail_plugins = " virtual imap_sieve"
}

I hope this helps.

On Thu, Sep 21, 2017 at 4:47 AM, Apollon Oikonomopoulos <[email-masked]>
wrote:

> Control: tags -1 + moreinfo upstream
>
> [Forwarding this to the Dovecot mailing list, just in case someone can
> help]
>
> Hi,
>
> Thanks for the report! See my comments inline.
>
> On 11:56 Thu 21 Sep     , Thurgood Angelou wrote:
> > Package: dovecot-core
> > Version: 1:2.2.32-2
> >
> > I've just discovered a bug where the sieve plugin (especially IMAP)
> > will not work with a virtual mailbox. I recently found this out when
> > testing a virtual mailbox setup and I use imap_sieve for SPAM
> > filtering so to lose that would be painful.
> >
> > This is what is reported in dovecot's debug log...
> >
> > Sep 21 10:53:01 imap([email-masked]): Panic: file
> mail-index-map.c: line 549 (mail_index_map_lookup_seq_range): assertion
> failed: (first_uid > 0)
> > Sep 21 10:53:01 imap([email-masked]): Error: Raw backtrace:
> /usr/lib/dovecot/libdovecot.so.0(+0x9f0a2) [0x7f4226fd50a2] ->
> /usr/lib/dovecot/libdovecot.so.0(+0x9f19a) [0x7f4226fd519a] ->
> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f4226f65cf8] ->
> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map_lookup_seq_range+0x120)
> [0x7f422733dae0] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xe8d1d)
> [0x7f422734cd1d] -> /usr/lib/dovecot/libdovecot-st
> orage.so.0(mail_index_lookup_seq+0xf) [0x7f4227350e9f] ->
> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x82aa)
> [0x7f422678a2aa] -> /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so(+0x708c)
> [0x7f422657d08c] -> /usr/lib/dovecot/libdovecot-st
> orage.so.0(mailbox_transaction_commit_get_changes+0x52) [0x7f42272adca2]
> -> dovecot/imap(+0x10084) [0x556502adb084] -> dovecot/imap(command_exec+0x5c)
> [0x556502ae7d9c] -> dovecot/imap(+0x1b322) [0x556502ae6322] ->
> dovecot/imap(+0x1b3bc) [0x556502ae63bc] -> dovecot/imap(client_handle_inp
> ut
> >  +0x18d) [0x556502ae677d] -> dovecot/imap(client_input+0xac)
> [0x556502ae6ccc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52)
> [0x7f4226fea742] -> /usr/lib/dovecot/libdovecot.so
> .0(io_loop_handler_run_internal+0x12e) [0x7f4226febd8e] ->
> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x36)
> [0x7f4226fea7d6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
> [0x7f4226fea988] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
> [0x7f4226f70353] -> dovecot/imap(main+0x329) [0x556502ad9139] ->
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f4226bb92e1]
> -> dovecot/imap(_start+0x2a) [0x556502ad92ca]
>
> After mixing in the debug symbols, this backtrace becomes:
>
> /usr/lib/dovecot/libdovecot.so.0(+0x9f0a2) [0x7f4226fd50a2]
>     default_fatal_finish at ./src/lib/failures.c:195
> /usr/lib/dovecot/libdovecot.so.0(+0x9f19a) [0x7f4226fd519a]
>     ?? at ./src/lib/failures.c:670
> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f4226f65cf8]
>     i_fatal at ./src/lib/failures.c:280
> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map_lookup_seq_range+0x120)
> [0x7f422733dae0]
>     mail_index_map_lookup_seq_range at ./src/lib-index/mail-index-map
> .c:550
> /usr/lib/dovecot/libdovecot-storage.so.0(+0xe8d1d) [0x7f422734cd1d]
>     tview_lookup_seq_range at ./src/lib-index/mail-index-tra
> nsaction-view.c:178
> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_lookup_seq+0xf)
> [0x7f4227350e9f]
>     mail_index_lookup_seq at ./src/lib-index/mail-index-view.c:522
> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x82aa) [0x7f422678a2aa]
>     virtual_mail_set_uid at ./src/plugins/virtual/virtual-mail.c:213
> /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so(+0x708c)
> [0x7f422657d08c]
>     imap_sieve_mailbox_transaction_run at ./pigeonhole/src/plugins/imapsieve/imap-sieve-storage.c:752
> (inlined by: imap_sieve_mailbox_transaction_commit at
> ./pigeonhole/src/plugins/imapsieve/imap-sieve-storage.c:807)
> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x52)
> [0x7f42272adca2]
>     mailbox_transaction_commit_get_changes at
> ./src/lib-storage/mail-storage.c:2083
> dovecot/imap(+0x10084) [0x556502adb084]
>     cmd_copy_full at ./src/imap/cmd-copy.c:146
> dovecot/imap(command_exec+0x5c) [0x556502ae7d9c]
>     command_exec at ./src/imap/imap-commands.c:200
> dovecot/imap(+0x1b322) [0x556502ae6322]
>     client_command_input at ./src/imap/imap-client.c:1088
> dovecot/imap(+0x1b3bc) [0x556502ae63bc]
>     client_command_input at ./src/imap/imap-client.c:1150
> dovecot/imap(client_handle_input+0x18d) [0x556502ae677d]
>     client_handle_input at ./src/imap/imap-client.c:1203
> dovecot/imap(client_input+0xac) [0x556502ae6ccc]
>     client_input at ./src/imap/imap-client.c:1249
> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f4226fea742]
>     io_loop_call_io at ./src/lib/ioloop.c:600
> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12e)
> [0x7f4226febd8e]
>     io_loop_handler_run_internal at ./src/lib/ioloop-epoll.c:223
> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x36)
> [0x7f4226fea7d6]
>     io_loop_handler_run at ./src/lib/ioloop.c:648
> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f4226fea988]
>     io_loop_run at ./src/lib/ioloop.c:622
> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f4226f70353]
>     master_service_run at ./src/lib-master/master-service.c:667
> dovecot/imap(main+0x329) [0x556502ad9139]
>     main at ./src/imap/main.c:491
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f4226bb92e1]
>     __libc_start_main at ../csu/libc-start.c:325
> dovecot/imap(_start+0x2a) [0x556502ad92ca]
>     start at ??:0
>
> > -- System Information:
> > Debian Release: 8.2
> >   APT prefers oldstable-updates
> >   APT policy: (500, 'oldstable-updates'), (500, 'unstable'), (500,
> 'testing'), (500, 'oldstable')
> > Architecture: amd64 (x86_64)
> >
> > Kernel: Linux 3.16.0-4-amd64 (SMP w/2 CPU cores)
> > Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8),
> LANGUAGE=en_GB:en (charmap=UTF-8)
> > Shell: /bin/sh linked to /bin/dash
> > Init: systemd (via /run/systemd/system)
> >
> > Versions of packages dovecot-sieve depends on:
> > ii  dovecot-core  1:2.2.32-2
> > ii  libc6         2.24-17
> > ii  ucf           3.0030
> >
> > dovecot-sieve recommends no packages.
> >
> > dovecot-sieve suggests no packages.
> >
> > Versions of packages dovecot-sieve is related to:
> > ii  dovecot-core [dovecot-common]  1:2.2.32-2
> > pn  dovecot-dbg                    <none>
> > pn  dovecot-dev                    <none>
> > ii  dovecot-gssapi                 1:2.2.32-2
> > ii  dovecot-imapd                  1:2.2.32-2
> > ii  dovecot-ldap                   1:2.2.32-2
> > ii  dovecot-lmtpd                  1:2.2.32-2
> > ii  dovecot-managesieved           1:2.2.32-2
> > ii  dovecot-mysql                  1:2.2.32-2
> > pn  dovecot-pgsql                  <none>
> > pn  dovecot-pop3d                  <none>
> > ii  dovecot-sieve                  1:2.2.32-2
> > pn  dovecot-sqlite                 <none>
>
> Could you also include the output of doveconf -n?
>
> Regards,
> Apollon
>

---

From: Thurgood Angelou <[email-masked]>
Date: Sat, 21 Oct 2017 16:47:48 -0700

Hey.

So I just tested this with a blank (0KB) sieve script. Apparently the
plugin crashes whenever it executes the BEFORE/AFTER function and when it
checks to see if the uid's are still the same.

Copying without the plugin (tested with telnet) works but copying with the
plugin causes the plugin to crash due to the BEFORE/AFTER function being
broken for virtual folders.

Just want to know what's the ETA for a fix if at possible. I don't know how
the thing works so I'm wondering if it's at all possible to fix this simple
thing...

On Thu, Sep 21, 2017 at 5:13 PM, Thurgood Angelou <[email-masked]

> /etc/dovecot/conf.d/15-lda.conf line 47 (if this is intentional, avoid
> this warning by moving the global setting before
> /etc/dovecot/conf.d/15-lda.conf line 47)
> /etc/dovecot/conf.d/15-lda.conf line 47 (if this is intentional, avoid
> this warning by moving the global setting before
> /etc/dovecot/conf.d/15-lda.conf line 47)
> On Thu, Sep 21, 2017 at 4:47 AM, Apollon Oikonomopoulos <
> [email-masked]> wrote:
>
>> [0x7f422678a2aa]