Acknowledgement sent
to Ben Hutchings <[email protected]>:
New Bug report received and forwarded. Copy sent to Debian Lintian Maintainers <[email protected]>.
(Mon, 10 Jan 2022 13:45:07 GMT) (full text, mbox, link).
Package: lintian
Version: 2.114.0
Severity: important
I tried running lintian on a full amd64 build of linux. lintian
allocated about 23 GiB of VM and then was OOM-killed before generating
any output.
I notice that <https://lintian.debian.org/sources/linux> has no
information, probably because of this.
This is a regression in lintian, but I don't know how long ago it
happened.
Ben.
-- System Information:
Debian Release: bookworm/sid
APT prefers unstable-debug
APT policy: (500, 'unstable-debug'), (500, 'oldstable-updates'), (500, 'unstable'), (500, 'oldstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 5.15.0-2-amd64 (SMP w/2 CPU threads)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages lintian depends on:
ii binutils 2.37-10
ii bzip2 1.0.8-5
ii diffstat 1.64-1
ii dpkg 1.21.1
ii dpkg-dev 1.21.1
ii file 1:5.41-2
ii gettext 0.21-4
ii gpg 2.2.27-2
ii intltool-debian 0.35.0+20060710.5
ii libapt-pkg-perl 0.1.40
ii libarchive-zip-perl 1.68-1
ii libcapture-tiny-perl 0.48-1
ii libclass-xsaccessor-perl 1.19-3+b7
ii libclone-perl 0.45-1+b1
ii libconfig-tiny-perl 2.27-1
ii libconst-fast-perl 0.014-1.1
ii libcpanel-json-xs-perl 4.27-1
ii libdata-dpath-perl 0.58-1
ii libdata-validate-___domain-perl 0.10-1.1
ii libdata-validate-uri-perl 0.07-1
ii libdevel-size-perl 0.83-1+b2
pn libdigest-sha-perl <none>
ii libdpkg-perl 1.21.1
ii libemail-address-xs-perl 1.04-1+b3
ii libencode-perl 3.16-1
ii libfile-basedir-perl 0.09-1
ii libfile-find-rule-perl 0.34-1
ii libfont-ttf-perl 1.06-1.1
ii libhtml-html5-entities-perl 0.004-1.1
ii libio-interactive-perl 1.023-1
ii libio-prompt-tiny-perl 0.003-1
ii libipc-run3-perl 0.048-2
ii libjson-maybexs-perl 1.004003-1
ii liblist-compare-perl 0.55-1
ii liblist-someutils-perl 0.58-1
ii liblist-utilsby-perl 0.11-1
ii libmoo-perl 2.005004-3
ii libmoox-aliases-perl 0.001006-1.1
ii libnamespace-clean-perl 0.27-1
ii libpath-tiny-perl 0.120-1
ii libperlio-gzip-perl 0.19-1+b7
ii libperlio-utf8-strict-perl 0.008-1+b1
ii libproc-processtable-perl 0.634-1
ii libsereal-decoder-perl 4.018+ds-1+b1
ii libsereal-encoder-perl 4.018+ds-1+b1
ii libsort-versions-perl 1.62-1
ii libsyntax-keyword-try-perl 0.26-1
ii libterm-readkey-perl 2.38-1+b2
ii libtext-glob-perl 0.11-2
ii libtext-levenshteinxs-perl 0.03-4+b8
ii libtext-markdown-discount-perl 0.13-1
ii libtext-xslate-perl 3.5.9-1
ii libtime-duration-perl 1.21-1
ii libtime-moment-perl 0.44-1+b3
ii libtimedate-perl 2.3300-2
ii libunicode-utf8-perl 0.62-1+b2
ii liburi-perl 5.10-1
ii libxml-libxml-perl 2.0134+dfsg-2+b1
ii libyaml-libyaml-perl 0.83+ds-1
ii lzip 1.22-4
ii lzop 1.04-2
ii man-db 2.9.4-2
ii patchutils 0.4.2-1
ii perl [libencode-perl] 5.32.1-6
ii t1utils 1.41-4
ii unzip 6.0-26
ii xz-utils 5.2.5-2
lintian recommends no packages.
Versions of packages lintian suggests:
ii binutils-multiarch 2.37-10
ii libtext-template-perl 1.60-1
-- no debconf information
Acknowledgement sent
to Felix Lechner <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian Lintian Maintainers <[email protected]>.
(Mon, 10 Jan 2022 14:03:03 GMT) (full text, mbox, link).
Subject: Re: Bug#1003456: Excessive memory use with large binaries
Date: Mon, 10 Jan 2022 05:56:13 -0800
Hi,
On Mon, Jan 10, 2022 at 5:45 AM Ben Hutchings <[email protected]> wrote:
>
> lintian allocated about 23 GiB
Thanks for reporting this!
That is one of several mysterious results of what happened when I
simplified the code. It's possibly due to excessive regex usage,
although Perl usually catches that. Another possibility is an
accidental recursion in Lintian's copy of your package's file index.
Either way, I will shortly add helpful profiling information that you
will be able to examine on our website.
On the positive side, I broke Lintian's 23 original checks into 353
pieces (and counting). It makes issues much easier to monitor and
isolate.
Kind regards
Felix Lechner
Acknowledgement sent
to Ben Hutchings <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian Lintian Maintainers <[email protected]>.
(Mon, 10 Jan 2022 15:27:04 GMT) (full text, mbox, link).
On Mon, 2022-01-10 at 05:56 -0800, Felix Lechner wrote:
> Hi,
>
> On Mon, Jan 10, 2022 at 5:45 AM Ben Hutchings <[email protected]> wrote:
> >
> > lintian allocated about 23 GiB
>
> Thanks for reporting this!
>
> That is one of several mysterious results of what happened when I
> simplified the code. It's possibly due to excessive regex usage,
I'm not aware of regex processing ever requiring lots of memory.
> although Perl usually catches that. Another possibility is an
> accidental recursion in Lintian's copy of your package's file index.
> Either way, I will shortly add helpful profiling information that you
> will be able to examine on our website.
I don't really want to examine it, I want you or some other lintian
contributor to fix it.
> On the positive side, I broke Lintian's 23 original checks into 353
> pieces (and counting). It makes issues much easier to monitor and
> isolate.
OK, but it's no longer working for me or probably anyone working on a
larger package.
I just tested with some older releases. The performance got worse
between buster and bullseye, but the bullseye version is still usable
for me (~ 1.3 GiB VM used, 17 min wallclock time). So the regression
is since bullseye.
Ben.
--
Ben Hutchings
I say we take off; nuke the site from orbit.
It's the only way to be sure.
Acknowledgement sent
to Felix Lechner <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian Lintian Maintainers <[email protected]>.
(Fri, 21 Jan 2022 19:42:03 GMT) (full text, mbox, link).
Subject: Re: Bug#1003456: Excessive memory use with large binaries
Date: Fri, 21 Jan 2022 11:36:26 -0800
Hi,
So far, the most convenient way to reproduce this bug is the following
command. It exercises only a single, very simple check. [1]
bin/lintian -C linda --debug --no-override --no-tag-display-limit
../picolibc-arm-none-eabi_1.7.4-1_all.deb
Kind regards,
Felix Lechner
[1] https://salsa.debian.org/lintian/lintian/-/blob/master/lib/Lintian/Check/Linda.pm#L32-39
Control: tag -1 pending
Hello,
Bug #1003456 in lintian reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:
https://salsa.debian.org/lintian/lintian/-/commit/192e15ae2eda7535622d44d2f3f382783b110ee5
------------------------------------------------------------------------
Store ELF information from readelf in an MLDBM database. (Closes: #1003456)
Works but is slow. Perhaps not quite so much of the readelf information needs
to be retained.
The impact on average library packages has yet to be evaluated.
It may only is be issue for some installables. The one from the bug, for
example (picolibc-arm-none-eabi_1.7.4-1_all.deb) seemed to generate several
million symbols whose purpose was not clear to me. That impression is dated
because I looked into the reason for the high memory usage in January.
------------------------------------------------------------------------
(this message was generated automatically)
--
Greetings
https://bugs.debian.org/1003456
Acknowledgement sent
to Ben Hutchings <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian Lintian Maintainers <[email protected]>.
(Mon, 20 Jun 2022 20:18:02 GMT) (full text, mbox, link).
Control: reopen -1
Control: found -1 2.115.0
On Fri, 2022-03-11 at 20:39 +0000, Felix Lechner wrote:
> Control: tag -1 pending
>
> Hello,
>
> Bug #1003456 in lintian reported by you has been fixed in the
> Git repository and is awaiting an upload. You can see the commit
> message below and you can check the diff of the fix at:
>
> https://salsa.debian.org/lintian/lintian/-/commit/192e15ae2eda7535622d44d2f3f382783b110ee5
[...]
This didn't fix the issue. It is still reproducible by building linux
for amd64 and then running lintian over the resulting .changes file.
Running lintian over a single linux-image-<version>-amd64-dbg binary
package also shows the issue, but requires "only" about 12 GB of VM and
does complete on my development system.
Ben.
--
Ben Hutchings
Q. Which is the greater problem in the world today,
ignorance or apathy?
A. I don't know and I couldn't care less.
Acknowledgement sent
to Axel Beckert <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian Lintian Maintainers <[email protected]>.
(Mon, 20 Jun 2022 21:09:03 GMT) (full text, mbox, link).
Subject: Re: Bug#1003456: marked as pending in lintian
Date: Mon, 20 Jun 2022 23:06:05 +0200
Hi Ben,
Ben Hutchings wrote:
> > https://salsa.debian.org/lintian/lintian/-/commit/192e15ae2eda7535622d44d2f3f382783b110ee5
> [...]
>
> This didn't fix the issue. It is still reproducible by building linux
> for amd64 and then running lintian over the resulting .changes file.
>
> Running lintian over a single linux-image-<version>-amd64-dbg binary
> package also shows the issue, but requires "only" about 12 GB of VM and
> does complete on my development system.
Thanks for these details and for reopening!
Will also have a look at it.
Given that it is a debug package, I'll see if I can either
* find the cause by debugging the code,
* find the cause by bisecting, or
* reduce the amount of checks run against it as many
make no sense for debug packages.
Regards, Axel
--
,''`. | Axel Beckert <[email protected]>, https://people.debian.org/~abe/
: :' : | Debian Developer, ftp.ch.debian.org Admin
`. `' | 4096R: 2517 B724 C5F6 CA99 5329 6E61 2FF9 CD59 6126 16B5
`- | 1024D: F067 EA27 26B9 C3FC 1486 202E C09E 1D89 9593 0EDE
Debbugs is free software and licensed under the terms of the GNU General
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.