Debian Bug report logs - #1078514
autopkgtest: tmpfails if removal of tmp dir fails

Reply or subscribe to this bug.

Display info messages

Message #5 received at [email protected] (full text, mbox, reply):

From: Paul Gevers <[email protected]>

To: Debian Bug Tracking System <[email protected]>

Subject: autopkgtest: several packages triggers tmpfail instead of fail

Date: Thu, 8 Aug 2024 12:40:15 +0200

[Message part 1 (text/plain, inline)]

Package: autopkgtest
Version: 5.38
Severity: normal

Hi,

This bug is mostly to document and track an issue that has been annoying 
me for a long time already. On ci.d.n we have reject_listed [0] several 
packages because they tmpfail instead of failing. This points at 
autopkgtest not being able to handle the situation as I think it should.

Examples where I just verified this is the case:
* llvm-toolchain-11 on armhf in bullseye [1] as reported in bug #980957
* llvm-toolchain-13 on armhf in bullseye [2] as reported in bug #980957

Examples where the reject_list mentions tmpfails:
* libreoffice on armel
* linux (qemu) on stable and oldstable
* openjdk-8
* openjdk-11 (I verified it no longer tmpfails on unstable, maybe
              oldstable is OK too, but I didn't check)
* openjdk-21 on armel and riscv64
* openjdk-22 on riscv64
* openjdk-23 on armel, loong64 and riscv64

I recall that most of those openjdk-* failures actually happen during 
cleanup where the `rm` complains that the /tmp/$some-autopkgtest-dir 
directory isn't empty. That appears a race condition to me, where 
cleanup starts before the tests really have stopped writing to files 
after being killed. I wonder if retrying the removal at least once isn't 
going to go a long way to solve that particular issue.

Paul

[0] https://ci.debian.net/status/reject_list/
[1] 
https://ci.debian.net/packages/l/llvm-toolchain-11/oldstable/armhf/50087518/
156s clang-$VERSION -flto foo.c -o foo
156s clang: error: unable to execute command: Segmentation fault
156s clang: error: linker command failed due to signal (use -v to see 
invocation)
161s autopkgtest [08:22:06]: ERROR: testbed failure: testbed auxverb 
failed with exit code 254

[2] 
https://ci.debian.net/packages/l/llvm-toolchain-13/oldstable/armhf/50087616/
151s clang-$VERSION -flto foo.c -o foo
151s clang: error: unable to execute command: Segmentation fault
151s clang: error: linker command failed due to signal (use -v to see 
invocation)
156s autopkgtest [09:03:07]: ERROR: testbed failure: testbed auxverb 
failed with exit code 254

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #10 received at [email protected] (full text, mbox, reply):

From: Paul Gevers <[email protected]>

To: [email protected]

Subject: Re: autopkgtest: several packages triggers tmpfail instead of fail

Date: Sun, 11 Aug 2024 20:03:12 +0200

[Message part 1 (text/plain, inline)]

Control: clone -1 -2
Control: retitle -1 autopkgtest: tmpfails if test exits with 254 or 255
Control: tags -1 pending
Control: retitle -2 autopkgtest: tmpfails if removal of tmp dir fails

On Thu, 8 Aug 2024 12:40:15 +0200 Paul Gevers <[email protected]> wrote:
> I recall that most of those openjdk-* failures actually happen during 
> cleanup where the `rm` complains that the /tmp/$some-autopkgtest-dir 
> directory isn't empty. That appears a race condition to me, where 
> cleanup starts before the tests really have stopped writing to files 
> after being killed.
Let's clone this one, because I have a fix worked out for the exit 254 case.

Paul

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #19 received at [email protected] (full text, mbox, reply):

From: Paul Gevers <[email protected]>

To: [email protected]

Subject: Re: [1078514] autopkgtest: tmpfails if removal of tmp dir fails

Date: Thu, 16 Jan 2025 19:02:12 +0100

[Message part 1 (text/plain, inline)]

Hi,

On Sun, 11 Aug 2024 20:03:12 +0200 Paul Gevers <[email protected]> wrote:
> On Thu, 8 Aug 2024 12:40:15 +0200 Paul Gevers <[email protected]> wrote:
> > I recall that most of those openjdk-* failures actually happen during 
> > cleanup where the `rm` complains that the /tmp/$some-autopkgtest-dir 
> > directory isn't empty. That appears a race condition to me, where 
> > cleanup starts before the tests really have stopped writing to files 
> > after being killed.
> Let's clone this one, because I have a fix worked out for the exit 254 case.
Current example (with slightly different failure message): cmake on 
riscv64 [1]:

20988s The following tests FAILED:
20988s 	622 - Qt5Autogen.RerunMocPlugin (Failed)                Qt5
20988s Errors while running CTest
21033s autopkgtest [05:39:11]: ERROR: "rm -rf 
/tmp/autopkgtest-lxc.8323tz_i/downtmp/testsuite-artifacts 
/tmp/autopkgtest-lxc.8323tz_i/downtmp/autopkgtest_tmp" failed with 
status 137

Paul

[1] https://ci.debian.net/packages/c/cmake/unstable/riscv64/54908817/

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Send a report that this bug log contains spam.