Subject: Bug#801575: piuparts.d.o (piu-slave-bm-a) fails to start PostgreSQL

From: Evgeni Golov <[email-masked]>
Date: Mon, 12 Oct 2015 09:53:34 +0200

Package: piuparts.debian.org
Severity: normal

Ohai!

I have uploaded a new Bareos to sid yesterday, and it seems piuparts
failed to test it properly:
 https://piuparts.debian.org/sid/fail/bareos-database-common_14.2.5-2.log

Dissecting the log further, one sees that piuparts calls its
pre_install_database-server script, which decides to run
 apt-get -y install postgresql
and that seems to fail as the port it wants to use is already use:

  Setting up postgresql-common (170) ...
  Adding user postgres to group ssl-cert
  
  Creating config file /etc/postgresql-common/createcluster.conf with new version
  Building PostgreSQL dictionaries from installed myspell/hunspell packages...
  Removing obsolete dictionary files:
  Running in chroot, ignoring request.
  No PostgreSQL clusters exist; see "man pg_createcluster" ... (warning).
  Setting up postgresql-9.4 (9.4.5-1) ...
  Creating new cluster 9.4/main ...
    config /etc/postgresql/9.4/main
    data   /var/lib/postgresql/9.4/main
    locale C
    port   5433
  update-alternatives: using /usr/share/postgresql/9.4/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode
  Running in chroot, ignoring request.
  Starting PostgreSQL 9.4 database server: mainThe PostgreSQL server failed to start. Please check the log output: 2015-10-12 00:24:03 UTC [19161-1] LOG: could not bind IPv6 socket: Address already in use 2015-10-12 00:24:03 UTC [19161-2] HINT: Is another postmaster already running on port 5433? If not, wait a few seconds and retry. 2015-10-12 00:24:03 UTC [19161-3] LOG: could not bind IPv4 socket: Address already in use 2015-10-12 00:24:03 UTC [19161-4] HINT: Is another postmaster already running on port 5433? If not, wait a few seconds and retry. 2015-10-12 00:24:03 UTC [19161-5] WARNING: could not create listen socket for "localhost" 2015-10-12 00:24:03 UTC [19161-6] FATAL: could not create any TCP/IP sockets ... failed!
   failed!
  invoke-rc.d: initscript postgresql, action "start" failed.
  dpkg: error processing package postgresql-9.4 (--configure):
   subprocess installed post-installation script returned error exit status 1
  dpkg: dependency problems prevent configuration of postgresql:
   postgresql depends on postgresql-9.4; however:
    Package postgresql-9.4 is not configured yet.
  
  dpkg: error processing package postgresql (--configure):
   dependency problems - leaving unconfigured
  Processing triggers for libc-bin (2.19-22) ...
  Processing triggers for systemd (227-2) ...
  E: Sub-process /usr/bin/dpkg returned an error code (1)
0m24.8s ERROR: Command failed (status=100): ['chroot', '/srv/piuparts.debian.org/tmp/tmpyDdDgr', 'tmp/scripts/pre_install_database-server']

As PostgreSQL works just fine in a fresh Sid chroot after installation,
I suspect a problem on piu-slave-bm-a or in its specific configuration.

Regards
Evgeni

-- System Information:
Debian Release: 7.9
  APT prefers oldstable-updates
  APT policy: (500, 'oldstable-updates'), (500, 'oldstable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.2.0-4-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

---

From: Andreas Beckmann <[email-masked]>
Date: Mon, 12 Oct 2015 10:26:42 +0200

> and that seems to fail as the port it wants to use is already use:

Thats right, that happens if there are several packages wanting
postgresql are tested in parallel because the chroots have no network
separation. These logs are frequently rescheduled and will eventually
succeed.

Andreas

---

From: Holger Levsen <[email-masked]>
Date: Mon, 12 Oct 2015 10:54:01 +0200

control: reassign -1 piuparts.debian.org
control: retitle -1 use network seperation or start postrgresql on different ports

Hi,

thanks for the bug report Evgeni!

> separation. These logs are frequently rescheduled and will eventually

while this is nice, we should try to avoid temporarily false negatives completly.

cheers,
	Holger

---

From: Evgeni Golov <[email-masked]>
Date: Tue, 13 Oct 2015 23:17:25 +0200

Heya,

> while this is nice, we should try to avoid temporarily false negatives completly.

So "just" unshare(1) or systemd-nspawn(1) at the right moment would 
help, right?

-- 
Bruce Schneier can read and understand Perl programs.

---

From: Holger Levsen <[email-masked]>
Date: Wed, 14 Oct 2015 11:29:28 +0200

Hi,

> So "just" unshare(1) or systemd-nspawn(1) at the right moment would

I guess so, yes. Help with that (read: sending patches) would be much 
appreciated, I haven't had time to get familar with systemd-nspawn yet…

cheers,
	Holger

---

From: Christoph Berg <[email-masked]>
Date: Wed, 23 Dec 2015 11:31:50 +0100

Re: Evgeni Golov 2015-10-13 <[email-masked]>
> So "just" unshare(1) or systemd-nspawn(1) at the right moment would 

The NIH department has developed "newpid -n" (aka "newnet") as an
alternative lightweight variant.

Version 7 I'll be uploading in a minute supports "newpid -N newpidns1"
to join a network namespace that can allow network access (plain
"unshare -n" doesn't even have localhost by default). I'm using that
for simple separation of several testsuites running in parallel in the
apt.postgresql.org Jenkins setup. "newpid -N ... piuparts" should run
pretty transparently with that.

Christoph
-- 
[email-masked] | http://www.df7cb.de/

---

From: Holger Levsen <[email-masked]>
Date: Wed, 23 Dec 2015 11:48:49 +0100

Hi Christoph,

> pretty transparently with that.

sounds cool! do you plan to maintain this in jessie-bpo too?

cheers,
	Holger

---

From: Christoph Berg <[email-masked]>
Date: Wed, 23 Dec 2015 11:54:18 +0100

Re: Holger Levsen 2015-12-23 <[email-masked]>
> sounds cool! do you plan to maintain this in jessie-bpo too?

If it actually gets used, I can do that, sure. (For my own purposes
I'm self-hosting it on apt.pg.o.)

Also welcome would be feedback on how to set up namespaced networking
automatically, without introducing any security problems (it allows
all users to run it and uses CAP_SYS_ADMIN and CAP_NET_ADMIN to work).

Christoph

---

From: Holger Levsen <[email-masked]>
Date: Wed, 23 Dec 2015 12:02:59 +0100

Hi,

> I'm self-hosting it on apt.pg.o.)

to be able to use it on piuparts.d.o it needs to be in jessie-bpo… same for 
jenkins.debian.org/net basically, though there we are a bit more flexible…

cheers,
	Holger