Subject: Bug#1078214: autopkgtest: several packages triggers tmpfail instead of fail

From: Paul Gevers <[email-masked]>
Date: Thu, 8 Aug 2024 12:40:15 +0200

Package: autopkgtest
Version: 5.38
Severity: normal

Hi,

This bug is mostly to document and track an issue that has been annoying 
me for a long time already. On ci.d.n we have reject_listed [0] several 
packages because they tmpfail instead of failing. This points at 
autopkgtest not being able to handle the situation as I think it should.

Examples where I just verified this is the case:
* llvm-toolchain-11 on armhf in bullseye [1] as reported in bug #980957
* llvm-toolchain-13 on armhf in bullseye [2] as reported in bug #980957

Examples where the reject_list mentions tmpfails:
* libreoffice on armel
* linux (qemu) on stable and oldstable
* openjdk-8
* openjdk-11 (I verified it no longer tmpfails on unstable, maybe
               oldstable is OK too, but I didn't check)
* openjdk-21 on armel and riscv64
* openjdk-22 on riscv64
* openjdk-23 on armel, loong64 and riscv64

I recall that most of those openjdk-* failures actually happen during 
cleanup where the `rm` complains that the /tmp/$some-autopkgtest-dir 
directory isn't empty. That appears a race condition to me, where 
cleanup starts before the tests really have stopped writing to files 
after being killed. I wonder if retrying the removal at least once isn't 
going to go a long way to solve that particular issue.

Paul

[0] https://ci.debian.net/status/reject_list/
[1] 
https://ci.debian.net/packages/l/llvm-toolchain-11/oldstable/armhf/50087518/
156s clang-$VERSION -flto foo.c -o foo
156s clang: error: unable to execute command: Segmentation fault
156s clang: error: linker command failed due to signal (use -v to see 
invocation)
161s autopkgtest [08:22:06]: ERROR: testbed failure: testbed auxverb 
failed with exit code 254

[2] 
https://ci.debian.net/packages/l/llvm-toolchain-13/oldstable/armhf/50087616/
151s clang-$VERSION -flto foo.c -o foo
151s clang: error: unable to execute command: Segmentation fault
151s clang: error: linker command failed due to signal (use -v to see 
invocation)
156s autopkgtest [09:03:07]: ERROR: testbed failure: testbed auxverb 
failed with exit code 254

---

From: Paul Gevers <[email-masked]>
Date: Sun, 11 Aug 2024 20:03:12 +0200

Control: clone -1 -2
Control: retitle -1 autopkgtest: tmpfails if test exits with 254 or 255
Control: tags -1 pending
Control: retitle -2 autopkgtest: tmpfails if removal of tmp dir fails

On Thu, 8 Aug 2024 12:40:15 +0200 Paul Gevers <[email-masked]> wrote:
> after being killed.
Let's clone this one, because I have a fix worked out for the exit 254 case.

Paul

---

From: Paul Gevers <[email-masked]>
Date: Thu, 16 Jan 2025 19:02:12 +0100

Hi,

> Let's clone this one, because I have a fix worked out for the exit 254 case.
Current example (with slightly different failure message): cmake on 
riscv64 [1]:

20988s The following tests FAILED:
20988s 	622 - Qt5Autogen.RerunMocPlugin (Failed)                Qt5
20988s Errors while running CTest
21033s autopkgtest [05:39:11]: ERROR: "rm -rf 
/tmp/autopkgtest-lxc.8323tz_i/downtmp/testsuite-artifacts 
/tmp/autopkgtest-lxc.8323tz_i/downtmp/autopkgtest_tmp" failed with 
status 137

Paul

[1] https://ci.debian.net/packages/c/cmake/unstable/riscv64/54908817/