Subject: lintian: Testsuite failure on some systems: 1h time offset in test ancient-source

From: Axel Beckert <[email-masked]>
Date: Sun, 5 Feb 2023 12:43:36 +0100 (CET)

Package: lintian
Version: 2.116.3
Severity: important
Tags: ftbfs

While running Lintian's testsuite on a much faster system compared to
my Sid amd64 running development workstation, I noticed the following
test suite failure when running "private/runtests" or trying to build
the package on a more modern and more performant box with Bookworm
amd64. (Use "private/runtests -o test:ancient-source" to just run the
affected test.)

# Hints do not match
# 
# --- debian/test-out/eval/checks/unpack/ancient-source/hints.specified.calibrated
# +++ debian/test-out/eval/checks/unpack/ancient-source/hints.actual.parsed
# -ancient-source (source): unpack-message-for-source tar: ancient-source-1.0/README: implausibly old time stamp 1969-12-31 23:59:59
# +ancient-source (source): unpack-message-for-source tar: ancient-source-1.0/README: implausibly old time stamp 1970-01-01 00:59:59
# 
debian/test-out/eval/checks/unpack/ancient-source/generic.t .. 1/1 #   Failed test 'Lintian passes for ancient-source'
#   at /home/abe/debian/lintian/lib/Test/Lintian/Run.pm line 343.
# Looks like you failed 1 test of 1.

But I neither get that failure on my Sid amd64 development workstation
nor on SalsaCI (https://salsa.debian.org/lintian/lintian/-/pipelines)
nor in pbuilder nor on the buildd
(https://buildd.debian.org/status/package.php?p=lintian).

So I tried to find differences.

$ als debian/test-out/packages/checks/unpack/ancient-source/ancient-source_1.0.orig.tar.gz ancient-source-1.0/README
-rw-r--r-- root/root        21 1970-01-01 00:59 ancient-source-1.0/README
$ env TZ=GMT als debian/test-out/packages/checks/unpack/ancient-source/ancient-source_1.0.orig.tar.gz ancient-source-1.0/README
-rw-r--r-- root/root        21 1969-12-31 23:59 ancient-source-1.0/README

But these two outputs are identical on both hosts, the one where the
test fails and one of those where it doesn't fail.  So the timestamp
actually seems to be the same and it's just Lintian's parsing of it
seems to differ.

The only potentially relevant package version difference I found was
tar, which is at 1.34+dfsg-1.1 in Sid and 1.34+dfsg-1 in Bookworm due
to a recent FTBFS on 32-bit architectures. But the sole change was in
debian/copyright and also downgrading the tar version in Sid to the
one from Bookworm didn't make the test failing on Sid.

Also running the test suite with "env TZ=GMT" prepended didn't make a
difference.

Additionally, build 2.116.2 did work on that very same box where
2.116.3 now FTBFS. So it seems to have caused been a recent change
(since 2023-01-29) in Bookworm. Or maybe such a change just uncovered
an older bug.

The locales of two systems where it builds and where it no more
builds:

Builds fine:

LANG=C.UTF-8
LANGUAGE=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=

No more builds fine (but hasn't change since when it still built
fine):

LANG=C
LANGUAGE=
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=

Also both systems have "Europe/Zurich" in their /etc/timezone.

So I currently have no idea where the difference comes from or which
environmental change (variables or package versions) triggers
it. Cc'ing the Reproducible Builds project in the hope that they have
an idea what could have caused the 1h offset in the testsuite on that
one box.

Bug report written on the system where the test failed.

-- System Information:
Debian Release: bookworm/sid
  APT prefers testing-debug
  APT policy: (500, 'testing-debug'), (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 6.1.0-1-amd64 (SMP w/16 CPU threads; PREEMPT)
Locale: LANG=C, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: sysvinit (via /sbin/init)
LSM: AppArmor: enabled

Versions of packages lintian depends on:
ii  binutils                        2.40-2
ii  bzip2                           1.0.8-5+b1
ii  clzip [lzip-decompressor]       1.13-5
ii  diffstat                        1.65-1
ii  dpkg                            1.21.19
ii  dpkg-dev                        1.21.19
ii  file                            1:5.44-3
ii  gettext                         0.21-10
ii  gpg                             2.2.40-1
ii  intltool-debian                 0.35.0+20060710.6
ii  iso-codes                       4.12.0-1
ii  libapt-pkg-perl                 0.1.40+b2
ii  libarchive-zip-perl             1.68-1
ii  libberkeleydb-perl              0.64-2+b1
ii  libcapture-tiny-perl            0.48-2
ii  libclass-xsaccessor-perl        1.19-4+b1
ii  libclone-perl                   0.46-1
ii  libconfig-tiny-perl             2.28-2
ii  libconst-fast-perl              0.014-2
ii  libcpanel-json-xs-perl          4.32-1+b1
ii  libdata-dpath-perl              0.58-2
ii  libdata-validate-domain-perl    0.10-1.1
ii  libdata-validate-uri-perl       0.07-2
ii  libdevel-size-perl              0.83-2+b1
ii  libdigest-sha-perl              6.03-1+b1
ii  libdpkg-perl                    1.21.19
ii  libemail-address-xs-perl        1.05-1+b1
ii  libencode-perl                  3.19-1+b1
ii  libfile-basedir-perl            0.09-2
ii  libfile-find-rule-perl          0.34-3
ii  libfont-ttf-perl                1.06-2
ii  libhtml-html5-entities-perl     0.004-3
ii  libhtml-tokeparser-simple-perl  3.16-4
ii  libio-interactive-perl          1.023-2
ii  libipc-run3-perl                0.048-3
ii  libjson-maybexs-perl            1.004004-1
ii  liblist-compare-perl            0.55-2
ii  liblist-someutils-perl          0.59-1
ii  liblist-utilsby-perl            0.12-2
ii  libmldbm-perl                   2.05-4
ii  libmoo-perl                     2.005005-1
ii  libmoox-aliases-perl            0.001006-2
ii  libnamespace-clean-perl         0.27-2
ii  libpath-tiny-perl               0.144-1
ii  libperlio-gzip-perl             0.20-1+b1
ii  libperlio-utf8-strict-perl      0.010-1
ii  libproc-processtable-perl       0.634-1+b2
ii  libregexp-wildcards-perl        1.05-3
ii  libsereal-decoder-perl          5.001+ds-1+b1
ii  libsereal-encoder-perl          5.001+ds-2
ii  libsort-versions-perl           1.62-3
ii  libsyntax-keyword-try-perl      0.28-1
ii  libterm-readkey-perl            2.38-2+b1
ii  libtext-levenshteinxs-perl      0.03-5+b1
ii  libtext-markdown-discount-perl  0.16-1
ii  libtext-xslate-perl             3.5.9-1+b2
ii  libtime-duration-perl           1.21-2
ii  libtime-moment-perl             0.44-2+b1
ii  libtimedate-perl                2.3300-2
ii  libunicode-utf8-perl            0.62-2
ii  liburi-perl                     5.17-1
ii  libwww-mechanize-perl           2.15-1
ii  libwww-perl                     6.67-1
ii  libxml-libxml-perl              2.0207+dfsg+really+2.0134-1+b1
ii  libyaml-libyaml-perl            0.86+ds-1
ii  lzip [lzip-decompressor]        1.23-5
ii  lzop                            1.04-2
ii  man-db                          2.11.2-1
ii  patchutils                      0.4.2-1
ii  perl [libencode-perl]           5.36.0-7
ii  plzip [lzip-decompressor]       1.10-5
ii  t1utils                         1.41-4
ii  unzip                           6.0-27
ii  xz-utils                        5.4.1-0.1

lintian recommends no packages.

Versions of packages lintian suggests:
ii  binutils-multiarch     2.40-2
ii  libtext-template-perl  1.61-1

-- no debconf information

---

From: Russ Allbery <[email-masked]>
Date: Sun, 05 Feb 2023 09:03:24 -0800

> # +ancient-source (source): unpack-message-for-source tar: ancient-source-1.0/README: implausibly old time stamp 1970-01-01 00:59:59

The exactly one hour difference makes me suspicious something is going on
with time zone conversions.  That's also consistent with the one hour time
difference between UTC and Europe/Zurich at New Years.

Looking at the source of tar, the output timestamp for this error seems to
be in local time by default, which would certainly explain the problem but
not why we're not seeing it everywhere.  I would be curious if it went
away if you added --utc to the flags to tar in
Lintian::IO::Select::unpack_and_index_piped_tar or (bigger hammer) just
set TZ=UTC when running Lintian.

Lintian should probably force all output it controls to UTC for
reproducibility, including tar's, but I'm still mystified as to why it
works on the other system.  This part of tar doesn't seem to have changed,
and as you mentioned replacing tar didn't change anything.

-- 
Russ Allbery ([email-masked])              <https://www.eyrie.org/~eagle/>

---

From: Vagrant Cascadian <[email-masked]>
Date: Sun, 05 Feb 2023 12:35:20 -0800

> # +ancient-source (source): unpack-message-for-source tar: ancient-source-1.0/README: implausibly old time stamp 1970-01-01 00:59:59

Wild guess would be timezone differences between the build environments?

live well,
  vagrant

---

From: Axel Beckert <[email-masked]>
Date: Sun, 5 Feb 2023 21:59:49 +0100

Hi Vagrant,

> Wild guess would be timezone differences between the build

Yeah, that's what I looked for first, but both boxes have
"Europe/Zurich" as timezone and prepending "env TZ=GMT" didn't make a
difference on either side.

Thanks for caring nevertheless!

		Regards, Axel
-- 
 ,''`.  |  Axel Beckert <[email-masked]>, https://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-    |  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

---

From: Axel Beckert <[email-masked]>
Date: Sun, 5 Feb 2023 22:01:54 +0100

Hi Russ,

Russ Allbery wrote:
> Lintian::IO::Select::unpack_and_index_piped_tar

Nice idea! Will definitely try.

> or (bigger hammer) just set TZ=UTC when running Lintian.

I tried with TZ=GMT. I also tried TZ=UTC, but that had no effect. I
think you need to use TZ=Etc/UTC there instead.

> and as you mentioned replacing tar didn't change anything.

Exactly. All of that. :-)

		Regards, Axel