Debian Bug report logs - #750525
mailman: Master qrunner detected subprocess exit while decoding utf8 character

Reply or subscribe to this bug.

Display info messages

Message #5 received at [email protected] (full text, mbox, reply):

From: Laurent Mazet <[email protected]>

To: Debian Bug Tracking System <[email protected]>

Subject: mailman: Master qrunner detected subprocess exit while decoding utf8 character

Date: Wed, 04 Jun 2014 08:07:08 +0200

Package: mailman
Version: 1:2.1.16-2
Severity: important

Dear Maintainer,

Thanks to mailman an postfix, I'm running multiple mailing list for 2 virtuals ___domain since 3 years without any majour issues. But one mailing list fall down 2 days ago, just after upgrading to 2.1.16-1. 80% of mail are lock in the queue. Others mailing lists still work fine. I got this type of log in /var/lib/mailman/log/error

--8<--
Jun 02 11:24:49 2014 (3899) Uncaught runner exception: 'utf8' codec can't decode byte 0xe9 in position 8: invalid continuatio
Jun 02 11:24:49 2014 (3899) Traceback (most recent call last):
  File "/var/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop
    self._onefile(msg, msgdata)
  File "/var/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 223, in process
    i18ndesc = uheader(mlist, mlist.description, 'List-Id', maxlinelen=998)
  File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 65, in uheader
    return Header(s, charset, maxlinelen, header_name, continuation_ws)
  File "/usr/lib/python2.7/email/header.py", line 183, in __init__
    self.append(s, charset, errors)
  File "/usr/lib/python2.7/email/header.py", line 267, in append
    ustr = unicode(s, incodec, errors)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 8: invalid continuation byte

Jun 02 11:24:49 2014 (3899) SHUNTING: 1401701086.211024+3c3ed99893324dcbcacc1c719fbc0a8b3004d4bb
-->8--

Finaly I downgrad mailmain to 2.1.16-1 and the problem disappears. By the way, mails lock in the queue were finally send.

Regards,

    Laurent

*** Reporter, please consider answering these questions, where appropriate ***

   * What led up to the situation?
   * What exactly did you do (or not do) that was effective (or
     ineffective)?
   * What was the outcome of this action?
   * What outcome did you expect instead?

*** End of the template - remove these template lines ***


-- System Information:
Debian Release: jessie/sid
  APT prefers testing
  APT policy: (500, 'testing'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.14-1-amd64 (SMP w/1 CPU core)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages mailman depends on:
ii  apache2                      2.4.9-1
ii  apache2-bin [httpd]          2.4.9-1
ii  apache2-mpm-prefork [httpd]  2.4.9-1
ii  cron                         3.0pl1-124
ii  debconf [debconf-2.0]        1.5.53
ii  libc6                        2.18-7
ii  logrotate                    3.8.7-1
ii  lsb-base                     4.1+Debian12
pn  python:any                   <none>
ii  ucf                          3.0029

Versions of packages mailman recommends:
ii  postfix [mail-transport-agent]  2.11.0-1

Versions of packages mailman suggests:
ii  listadmin     2.40-4
ii  lynx          2.8.8pre5-1
ii  spamassassin  3.4.0-1

-- Configuration Files:
/etc/init.d/mailman changed:
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/lib/mailman/bin/mailmanctl
PIDFILE=/var/run/mailman/mailman.pid
test -x $DAEMON || exit 0
set -e
if ! [ -d /var/run/mailman ]; then
      install -d -o list -g list /var/run/mailman
fi
if ! [ -d /var/lock/mailman ]; then
      install -d -o root -g list -m 2775 /var/lock/mailman
fi
.. /lib/lsb/init-functions
if ! [ -x /usr/bin/python ]; then
	log_warning_msg "Python interpreter not available, exiting."
	exit 0;
fi
case "$1" in
  start)
    SITE_LIST=$( sed -rne "s/^[[:space:]]*MAILMAN_SITE_LIST[[:space:]]*=[[:space:]]*(['\"])([^'\"]+)\\1/\\2/p" /etc/mailman/mm_cfg.py )
    [ -n "$SITE_LIST" ] || SITE_LIST='mailman'
    if ! /var/lib/mailman/bin/list_lists -b | grep -q "^${SITE_LIST}$"; then
        log_warning_msg "Site list for mailman missing (looking for list named '${SITE_LIST}')."
        log_warning_msg "Please create it; until then, mailman will refuse to start."
        exit 0;
    fi
    log_daemon_msg "Starting Mailman master qrunner" "mailmanctl"
    if $DAEMON -s -q start; then
        log_end_msg 0
    else
        log_end_msg 1
    fi
    ;;
  stop)
    log_daemon_msg "Stopping Mailman master qrunner" "mailmanctl"
    if $DAEMON -q stop; then
        rm -f $PIDFILE
        log_end_msg 0
    else
        log_end_msg 1
    fi
    ;;
  reload)
    log_begin_msg "Reloading Mailman master qrunner configuration"
    if $DAEMON -q restart; then
        log_end_msg 0
    else
        log_end_msg 1
    fi
  ;;
  restart|force-reload)
    PID=`cat $PIDFILE 2>/dev/null` || true
    log_daemon_msg "Restarting Mailman master qrunner" "mailmanctl"
    $DAEMON -q stop
    if test -n "$PID" && kill -0 $PID 2>/dev/null ; then
        log_action_begin_msg "Waiting"
        for cnt in `seq 1 5`; do
            sleep 1
            kill -0 $PID 2>/dev/null || break
        done;
        if kill -0 $PID 2>/dev/null ; then
            log_action_end_msg 1
        else
            log_action_end_msg 0
        fi
    fi
    if $DAEMON -q start; then
        log_end_msg 0
    else
        log_end_msg 1
    fi
    ;;
  *)
    echo "Usage: /etc/init.d/mailman {start|stop|restart|reload|force-reload}" >&2
    exit 1
    ;;
esac
exit 0


-- debconf information:
* mailman/used_languages: en fr
* mailman/default_server_language: en
* mailman/queue_files_present: continue regardless
* mailman/create_site_list:
  mailman/gate_news: false
* mailman/site_languages: fr, en

Message #10 received at [email protected] (full text, mbox, reply):

From: Thorsten Glaser <[email protected]>

To: Laurent Mazet <[email protected]>, [email protected]

Subject: Re: [Pkg-mailman-hackers] Bug#750525: mailman: Master qrunner detected subprocess exit while decoding utf8 character

Date: Wed, 4 Jun 2014 09:12:35 +0200 (CEST)

On Wed, 4 Jun 2014, Laurent Mazet wrote:

> Thanks to mailman an postfix, I'm running multiple mailing list for 2
> virtuals ___domain since 3 years without any majour issues. But one
> mailing list fall down 2 days ago, just after upgrading to 2.1.16-1.

> Jun 02 11:24:49 2014 (3899) Uncaught runner exception: 'utf8' codec
> can't decode byte 0xe9 in position 8: invalid continuatio

Do you have any custom language files? The new version requires
them to be in UTF-8 encoding, whereas the old version used a
language-specific legacy encoding (and, in some cases, even one
that wasn’t even good for that language).

The language files the package installs by itself would have been
upgraded to UTF-8 encoding during the package upgrade, but if you
made any modifications, you have to do that yourself. I see you
already have the /etc/init.d/mailman file changed, so, a local
modification, e.g. to the fr language files, may be a cause.

bye,
//mirabilos
-- 
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

Message #15 received at [email protected] (full text, mbox, reply):

From: Laurent Mazet <[email protected]>

To: Thorsten Glaser <[email protected]>

Cc: [email protected]

Subject: Re: [Pkg-mailman-hackers] Bug#750525: mailman: Master qrunner detected subprocess exit while decoding utf8 character

Date: Wed, 04 Jun 2014 17:52:58 +0200

Package: mailman
Version: 1:2.1.16-2
Severity: important

Hi

I don't remenber that I've done any modification in language language file

Moreover, when  I downgrade mailman package to 2.1.16-1, I saw it  
rebuilding language file for en and fr. Thus, it certainly do the same  
while upgrading to 2.1.16-2.

Regards

     Laurent

Quoting Thorsten Glaser <[email protected]> at mer., 04 juin 2014:

> On Wed, 4 Jun 2014, Laurent Mazet wrote:
>
>> Thanks to mailman an postfix, I'm running multiple mailing list for 2
>> virtuals ___domain since 3 years without any majour issues. But one
>> mailing list fall down 2 days ago, just after upgrading to 2.1.16-1.
>
>> Jun 02 11:24:49 2014 (3899) Uncaught runner exception: 'utf8' codec
>> can't decode byte 0xe9 in position 8: invalid continuatio
>
> Do you have any custom language files? The new version requires
> them to be in UTF-8 encoding, whereas the old version used a
> language-specific legacy encoding (and, in some cases, even one
> that wasn’t even good for that language).
>
> The language files the package installs by itself would have been
> upgraded to UTF-8 encoding during the package upgrade, but if you
> made any modifications, you have to do that yourself. I see you
> already have the /etc/init.d/mailman file changed, so, a local
> modification, e.g. to the fr language files, may be a cause.
>
> bye,
> //mirabilos


-- 
Dr. Laurent Mazet   -=-  "Use the source, Luke"  -=-   [email protected]

Message #20 received at [email protected] (full text, mbox, reply):

From: Thorsten Glaser <[email protected]>

To: Laurent Mazet <[email protected]>

Cc: [email protected]

Subject: Re: [Pkg-mailman-hackers] Bug#750525: mailman: Master qrunner detected subprocess exit while decoding utf8 character

Date: Thu, 5 Jun 2014 09:41:51 +0200 (CEST)

On Wed, 4 Jun 2014, Laurent Mazet wrote:

> I don't remenber that I've done any modification in language language file

Hm, okay. Then, I have to ask, maybe you forgot to do the
upgrade instructions from /usr/share/doc/mailman/NEWS.Debian.gz ?

mailman (1:2.1.16-1exp1) experimental; urgency=low

  This version has changed the encoding of most strings, templates
  and pages to UTF-8 to meet the Debian release goal of full UTF-8
  support in all packages. It also no longer automatically converts
  mails to ISO-8859-1.

  If you have been using any nōn-ASCII strings in places such as
  the mailing list description, these were be stored wrongly in the
  list configuration file (config.pck), so you will need to change
  those (e.g. via the webinterface) again in order to have them be
  displayed correctly.

 -- Thorsten Glaser <[email protected]>  Sun, 29 Dec 2013 14:35:50 +0000

Read the second paragraph of this… although, in my tests, I did
not get error messages, only garbled characters.

bye,
//mirabilos
-- 
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

Message #27 received at [email protected] (full text, mbox, reply):

From: Sebastian Reichel <[email protected]>

To: [email protected]

Subject: Re: [Pkg-mailman-hackers] Bug#750525: mailman: Master qrunner detected subprocess exit while decoding utf8 character

Date: Sun, 26 Apr 2015 23:16:45 +0200

[Message part 1 (text/plain, inline)]

Hi,

On Thu, 5 Jun 2014 09:41:51 +0200 (CEST) Thorsten Glaser <[email protected]> wrote:
> On Wed, 4 Jun 2014, Laurent Mazet wrote:
> 
> > I don't remenber that I've done any modification in language language file
> 
> Hm, okay. Then, I have to ask, maybe you forgot to do the
> upgrade instructions from /usr/share/doc/mailman/NEWS.Debian.gz ?
> 
> mailman (1:2.1.16-1exp1) experimental; urgency=low
> 
>   This version has changed the encoding of most strings, templates
>   and pages to UTF-8 to meet the Debian release goal of full UTF-8
>   support in all packages. It also no longer automatically converts
>   mails to ISO-8859-1.
> 
>   If you have been using any nōn-ASCII strings in places such as
>   the mailing list description, these were be stored wrongly in the
>   list configuration file (config.pck), so you will need to change
>   those (e.g. via the webinterface) again in order to have them be
>   displayed correctly.
> 
>  -- Thorsten Glaser <[email protected]>  Sun, 29 Dec 2013 14:35:50 +0000
> 
> Read the second paragraph of this… although, in my tests, I did
> not get error messages, only garbled characters.

I just upgraded to jessie and had the same problem; fixing German
umlauts in the list short descriptions via the Webinterface made
it working again.

Would have been nice to have some post-inst script, which
checks/fixes this.

-- Sebastian

[signature.asc (application/pgp-signature, inline)]

Message #32 received at [email protected] (full text, mbox, reply):

From: Thorsten Glaser <[email protected]>

To: Sebastian Reichel <[email protected]>, [email protected]

Subject: Re: [Pkg-mailman-hackers] Bug#750525: Bug#750525: mailman: Master qrunner detected subprocess exit while decoding utf8 character

Date: Mon, 27 Apr 2015 09:15:04 +0200 (CEST)

On Sun, 26 Apr 2015, Sebastian Reichel wrote:

> Would have been nice to have some post-inst script, which
> checks/fixes this.

Well it’s too late for jessie now. SOL.

bye,
//mirabilos
-- 
Sometimes they [people] care too much: pretty printers [and syntax highligh-
ting, d.A.] mechanically produce pretty output that accentuates irrelevant
detail in the program, which is as sensible as putting all the prepositions
in English text in bold font.	-- Rob Pike in "Notes on Programming in C"

Send a report that this bug log contains spam.