Debian Bug report logs - #872036
AH00060: seg fault or similar nasty error detected in the parent process

Reply or subscribe to this bug.

Display info messages

Message #5 received at [email protected] (full text, mbox, reply):

From: lkcl <[email protected]>

To: Debian Bug Tracking System <[email protected]>

Subject: AH00060: seg fault or similar nasty error detected in the parent process

Date: Sun, 13 Aug 2017 17:35:38 +0100

Package: apache2-mpm-event
Version: 2.4.10-10+deb8u10
Severity: important
Tags: upstream

i have a slightly off-the-beaten-track setup on a live-running i386 (not amd64)
server which has (had) apache-mpm-event activated using the standard
debian configuration for that module.

there was (is) absolutely NO threading modules enabled or in use with
the exception of google's mod_pagespeed, where the developers inform
me that they do regular testing of mod_pagespeed in both the mpm-event
and mpm-worker configuration

the modules utilised for the main web serving are mod-fcgid (WSGI) and
standard CGI-bin for running PGP (not the normal way to do it but it
works very well with mpm_worker and mpm_event).  the FCGI application
is a python2.7 WSGI web service NOT USING ANY FRAMEWORK.  it uses
absolutely standard python 2.7 modules that come *with* python, such
as the cgi module and other standard HTML-parsing and processing modules,
where absolutely no use of threading is made whatsover.  mysql access
is performed with python-mysqldb, and, again, does not use any threading
of any kind.

unfortunately it is not... convenient... to do experimentation on the
live server.  this is more of an "informational" report that "Something
Bad Happened" - apache2 segfaulted and left the customer completely
without service... twice.... and switching off apache2-mpm-event
and going back to apache2-mpm-prefork "solved" the problem.

it's also worth noting that absolutely no problems whatsoever have been
noted with prefork.  stable as a rock, both before mod_pagespeed was
deployed and after.


-- System Information:
Debian Release: 8.1
  APT prefers testing
  APT policy: (500, 'testing'), (500, 'stable'), (500, 'oldstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.11.0-1-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages apache2-mpm-event depends on:
ii  apache2  2.4.10-10+deb8u10

apache2-mpm-event recommends no packages.

apache2-mpm-event suggests no packages.

-- no debconf information

Message #10 received at [email protected] (full text, mbox, reply):

From: Luke Kenneth Casson Leighton <[email protected]>

To: [email protected]

Subject: Re: Bug#872036: Acknowledgement (AH00060: seg fault or similar nasty error detected in the parent process)

Date: Sat, 19 Aug 2017 12:18:58 +0100

ok so a little more info here: the segfault occurs *directly* after a
logrotate-inspired signal is received.

[Sat Aug 19 06:25:37.746265 2017] [mpm_event:notice] [pid 21345:tid
3074504512] AH00493: SIGUSR1 received.  Doing graceful restart
[Sat Aug 19 06:25:39.647852 2017] [core:notice] [pid 21345] AH00060:
seg fault or similar nasty error detected in the parent process

an attempt to get more information by using mod_pagespeed's crash
handler *failed*.  one of the developers mentioned that there's an
option to get stack trace output put into the error logs, but
bizarrely (no explanation yet) we got.... absolutely no output when
the segfault occurred: just the standard apache2 AH00060 notice.

this is all very strange.... and i can't keep risking a live-running
client's system by using unstable software.  so, apologies, i'm going
to have to return to using mpm_prefork (i.e. can't risk doing further
tests with mpm_event on the live system).  performance was
*significantly* degraded around the time of the segfault.

Message #15 received at [email protected] (full text, mbox, reply):

From: Athanasius <[email protected]>

To: [email protected]

Subject: Re: Bug#872036: Acknowledgement (AH00060: seg fault or similar nasty error detected in the parent process)

Date: Mon, 6 Nov 2017 12:48:07 +0000

  Apparently this can be caused by something like logrotate sending many
"reload" signals to apache in quick succession:

<https://access.redhat.com/solutions/2626601>

I experienced such an apache crash this morning when the logs were
rotated.  I've now adjusted my logrotate config to only reload apache
once (despite many sections for different web site's logs, some of which
have specifically different user/group settings for the files) and will
see if that avoids another crash tomorrow.

Message #20 received at [email protected] (full text, mbox, reply):

From: Athanasius <[email protected]>

To: [email protected]

Subject: Re: Bug#872036: Acknowledgement (AH00060: seg fault or similar nasty error detected in the parent process)

Date: Wed, 8 Nov 2017 13:33:24 +0000

  mpm-event based apache has now survived two nightly runs of logrotate
doing '/etc/init.d/apache2 reload' since I ensured it will only do that
once per run.

  So what is it about mpm-event that causes apache2 to crash when
multiple '/etc/init.d/apache2 reload' invocations happen in very quick
succession ?  Smells like an upstream bug, yes ?
  Not that I can seem to reproduce this with:

COUNT=1 ; while [ $COUNT -lt 80 ] ; do ( /etc/init.d/apache2 reload &); COUNT=`expr $COUNT + 1` ; done

  Perhaps it's not just the rapid reloads, but combining those with
needing to create new log files ?

Although it does leave me with only processes like:

	/usr/sbin/apache2 -k graceful

running, rather than the more usual:

	/usr/sbin/apache2 -k start

And some instances of the init.d script complaining "Apache2 is not
running" on subsequent invocations.

Message #30 received at [email protected] (full text, mbox, reply):

From: Jeremy Stanley <[email protected]>

To: [email protected]

Subject: Re: Bug#872036: Acknowledgement (AH00060: seg fault or similar nasty error detected in the parent process)

Date: Tue, 21 Aug 2018 15:17:32 +0000

This sounds a lot like
https://bugzilla.redhat.com/show_bug.cgi?id=1376835 which includes a
potential patch (unfortunately no indication if this got upstreamed,
or whether it's reproducible with newer Apache releases).
-- 
Jeremy Stanley

Message #35 received at [email protected] (full text, mbox, reply):

From: Jérôme <[email protected]>

To: [email protected]

Subject: Re: AH00060: seg fault or similar nasty error detected in the parent process

Date: Mon, 10 Dec 2018 09:48:51 +0100

In case that helps, I just noticed a similar error.

I have a maintenance script that

- modifies a config file to put the application in maintenance mode
- systemctl reload apache2
- performs database maintenance
- sets the old config back
- systemctl reload apache2

When using this on a new installation where the DB is empty, the 
maintenance task takes an instant, and the script basically does

    systemctl reload apache2; systemctl reload apache2

which is enough to reproduce the crash.


Jeremy Stanley wrote:

> This sounds a lot like 
> https://bugzilla.redhat.com/show_bug.cgi?id=1376835

This bug report reads :

> The problem (httpd with worker/event mpm segfaults after multiple 
> successive graceful reloads) was fixed in 2.4.6-53.el7 (Development 
> Version)

However, I'm using Debian Stretch's 2.4.25-3+deb9u6 so AFAIU, this 
should include the patch.


Unless there is something specific to my configuration, the issue can be 
easily reproduced with

    systemctl reload apache2; systemctl reload apache2

When adding a sleep of a few hundreds milliseconds in between, the crash 
does not happen.

Also worth noting that despite the error message, apache is up and 
running at the end of the process, so AFAICT, this is just noise in the 
logs.

-- 
Jérôme

Send a report that this bug log contains spam.