Subject: autopkgtest: Possible race condition leading to build-needed being ignored
Date: Wed, 18 Aug 2021 22:15:01 -0700
Package: autopkgtest
Version: 5.17
Severity: important
Tags: upstream
X-Debbugs-Cc: [email protected]
Dear Maintainer,
My source tree contains DEP-8 tests in debian/tests. My debian/tests/control
contains the following test:
Tests: test-in-tree-unit-tests.sh
Depends: @builddeps@, postgresql-13
Restrictions: rw-build-tree, build-needed, allow-stderr, isolation-
container
I have another DEP-8 test, but it tests the installed artifacts without any
issue. The above test, however, calls a simple one liner (after the sh-bang),
to test local in-tree unit tests:
make -j check || { find . -iname "test-suite.log" -exec cat {} \; ; exit
99; }
Some times these tests are run as expected. Other times they don't ever run.
When they don't, it is because autopkgtest reports a failure when make(1) is
unable to run the `check` target. It appears as though there is no Makefile.
When I shell into the test bed into the ___location in which make was being run,
the source tarball has been unpacked as expected, but it has not been
configured. That is, I see my Makefile.am and other source files, but not the
generated Makefile or any other files generated after ./configure does its
thing.
Sometimes this happens and sometimes it does not. There does not appear to be
any rhyme or reason to it, so I suspect this may be a race condition.
-- System Information:
Debian Release: bullseye/sid
APT prefers hirsute-updates
APT policy: (500, 'hirsute-updates'), (500, 'hirsute-security'), (500,
'hirsute-proposed'), (500, 'hirsute'), (100, 'hirsute-backports')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 5.11.0-25-generic (SMP w/16 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE,
TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages autopkgtest depends on:
ii apt-utils 2.2.4ubuntu0.1
ii libdpkg-perl 1.20.9ubuntu1
ii procps 2:3.3.16-5ubuntu3.1
ii python3 3.9.4-1
ii python3-debian 0.1.39
Versions of packages autopkgtest recommends:
ii autodep8 0.24ubuntu1
Versions of packages autopkgtest suggests:
ii fakemachine 0.0~git20201127.9e6ee78-1
ii lxc 1:4.0.6-0ubuntu1
ii lxd 1:0.9
ii ovmf 2020.11-4
ii ovmf-ia32 2020.11-4
ii qemu-efi-aarch64 2020.11-4
ii qemu-efi-arm 2020.11-4
ii qemu-system 1:5.2+dfsg-9ubuntu3.1
ii qemu-utils 1:5.2+dfsg-9ubuntu3.1
ii schroot 1.6.10-11ubuntu2
ii vmdb2 0.22-1
Acknowledgement sent
to Kip Warner <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian CI team <[email protected]>.
(Mon, 23 Aug 2021 07:36:04 GMT) (full text, mbox, link).
Dear Maintainer,
Apologies for the delay in providing logs. It took several days to be
able to reproduce the above problem.
Here is an example of a pass:
https://pastebin.com/edit/Nnqkhsdr
Here is an example of a fail:
https://pastebin.com/FZ5qRQ1m
These are both in respect of the same source package with nothing
material changed between runs.
I'll draw your attention to L2996-3002 of the pass log, but apparently
not in the fail. I am not sure if these lines are important to take
note of.
--
Kip Warner -- Senior Software Engineer
OpenPGP signed/encrypted mail preferred
https://www.thevertigo.com
Acknowledgement sent
to Paul Gevers <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian CI team <[email protected]>.
(Mon, 23 Aug 2021 08:39:03 GMT) (full text, mbox, link).
Hi Kip,
Can you please provide the command line that you used?
I'm seeing this twice in the successful log while it's only there once
in the failed one and I don't understand where it's coming from:
Adding repository.
Adding deb entry to
/etc/apt/sources.list.d/pistache_team-ubuntu-unstable-hirsute.list
Adding disabled deb-src entry to
/etc/apt/sources.list.d/pistache_team-ubuntu-unstable-hirsute.list
Adding key to /etc/apt/trusted.gpg.d/pistache_team-ubuntu-unstable.gpg
with fingerprint 2EEA295DCBF66B6DE281E0A193E2268577BD194B
Also, I'm even missing the "build needed" log line in your successful
run. Which backend are you using?
Paul
Acknowledgement sent
to Kip Warner <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian CI team <[email protected]>.
(Tue, 24 Aug 2021 04:12:14 GMT) (full text, mbox, link).
On Mon, 2021-08-23 at 10:36 +0200, Paul Gevers wrote:
> Hi Kip,
Hey Paul,
> Can you please provide the command line that you used?
Certainly:
$ autopkgtest @/home/me/Projects/sbuild/autopkgtest/autopkgtest.cfg &> log.txt
And the autopkgtest.cfg is as follows:
$ cat /home/me/Projects/sbuild/autopkgtest/autopkgtest.cfg
--shell-fail
--apt-upgrade
--setup-commands=/home/me/Projects/sbuild/scripts/setup.sh
--
qemu
--ram-size=8192
--cpus=8
--show-boot
/home/me/Projects/sbuild/images/autopkgtest-hirsute-amd64.img
--qemu-options=-enable-kvm
And inside of the setup.sh:
$ cat /home/me/Projects/sbuild/scripts/setup.sh
#!/bin/bash
# Bail on errors...
set -e
apt install squid-deb-proxy-client avahi-utils --assume-yes
echo 'Acquire::http::proxy "http://my-workstation:3142";' | tee
/etc/apt/apt.conf.d/01acng > /dev/null
sed -i 's/^nameserver.*$/nameserver 192.168.1.1/' /etc/resolv.conf
apt install software-properties-common gnupg2 --assume-yes --no-
install-recommends
( ... add a private PPA ... )
add-apt-repository ppa:pistache+team/unstable --component main --component main/debug
> I'm seeing this twice in the successful log while it's only there once
> in the failed one and I don't understand where it's coming from:
>
> Adding repository.
> Adding deb entry to
> /etc/apt/sources.list.d/pistache_team-ubuntu-unstable-hirsute.list
> Adding disabled deb-src entry to
> /etc/apt/sources.list.d/pistache_team-ubuntu-unstable-hirsute.list
> Adding key to /etc/apt/trusted.gpg.d/pistache_team-ubuntu-unstable.gpg
> with fingerprint 2EEA295DCBF66B6DE281E0A193E2268577BD194B
>
> Also, I'm even missing the "build needed" log line in your successful
> run. Which backend are you using?
I'm using the QEMU backend with both host and guest amd64. I've also
regenerated the logs, this time redirecting stderr and stdout to the
log. This time they shouldn't be truncated, though I did make some
redactions because this is a non-free source package:
Pass (1/2):
https://pastebin.com/Nnqkhsdr
Pass (2/2):
https://pastebin.com/jfEFh6Wy
Fail:
https://pastebin.com/FZ5qRQ1m
--
Kip Warner -- Senior Software Engineer
OpenPGP signed/encrypted mail preferred
https://www.thevertigo.com
Acknowledgement sent
to Paul Gevers <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian CI team <[email protected]>.
(Thu, 03 Feb 2022 12:39:03 GMT) (full text, mbox, link).
Hi Kip,
On Mon, 23 Aug 2021 21:08:03 -0700 Kip Warner <[email protected]> wrote:
I ran across this issue again. Just a sanity check, you did take care of
this, right?
"""
rw-build-tree
The test(s) needs write access to the built source tree (so it may
need to be copied first). Even with this restriction, the test is
not allowed to make any change to the built source tree which (i)
isn't cleaned up by debian/rules clean, (ii) affects the future
results of any test, or (iii) affects binary packages produced by
the build tree in the future.
"""
Paul
Acknowledgement sent
to Kip Warner <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian CI team <[email protected]>.
(Thu, 03 Feb 2022 21:51:03 GMT) (full text, mbox, link).
On Thu, 2022-02-03 at 13:37 +0100, Paul Gevers wrote:
> I ran across this issue again. Just a sanity check, you did take care
> of this, right?
>
> """
> rw-build-tree
Hey Paul,
Yes, indeed. The test's Restrictions stanza is as follows: rw-build-
tree, build-needed, allow-stderr, isolation-container.
--
Kip Warner -- Senior Software Engineer
OpenPGP signed/encrypted mail preferred
https://www.thevertigo.com
Acknowledgement sent
to Paul Gevers <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian CI team <[email protected]>.
(Sat, 05 Feb 2022 07:48:02 GMT) (full text, mbox, link).
Hi Kip,
On 03-02-2022 22:48, Kip Warner wrote:
> On Thu, 2022-02-03 at 13:37 +0100, Paul Gevers wrote:
>> I ran across this issue again. Just a sanity check, you did take care
>> of this, right?
>>
>> """
>> rw-build-tree
>
> Hey Paul,
>
> Yes, indeed. The test's Restrictions stanza is as follows: rw-build-
> tree, build-needed, allow-stderr, isolation-container.
Sorry, that was not what I meant. I spotted rw-build-tree and noticed in
the description that *you* have to take care of things if you use this
restriction. Did you do that?
Paul
Acknowledgement sent
to Kip Warner <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian CI team <[email protected]>.
(Sat, 05 Feb 2022 22:33:03 GMT) (full text, mbox, link).
On Sat, 2022-02-05 at 08:44 +0100, Paul Gevers wrote:
> Sorry, that was not what I meant. I spotted rw-build-tree and noticed
> in the description that *you* have to take care of things if you use
> this restriction. Did you do that?
Hey Paul,
This is all the aforementioned DEP-8 test does. It's a trivial two-
liner:
# Bail on any errors...
set -e
# Perform all in-tree unit tests or show log on failure...
make -j check || { find . -iname "test-suite.log" -exec cat {} \; ; exit 99; }
All build artifacts are automatically cleaned up when debian/rules
clean is invoked. Nothing in these unit tests should affect any other
DEP-8 test, nor affect any binary packages produced by the build tree
in the future as far as I am aware.
--
Kip Warner -- Senior Software Engineer
OpenPGP signed/encrypted mail preferred
https://www.thevertigo.com
Debbugs is free software and licensed under the terms of the GNU General
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.