Subject: Bug#1014156: lintian: very-long-line-length-in-source-file for non-text source files

From: Daniel Kahn Gillmor <[email-masked]>
Date: Fri, 01 Jul 2022 01:08:14 -0400

Package: lintian
Version: 2.115.2
Severity: minior
Control: affects -1 src:gnupg2

lintian 2.115.2 complains (in --pedantic) in the following way about
these non-text files in the gnupg2 sources:

P: gnupg2 source: very-long-line-length-in-source-file 1008 > 512 [po/eo.gmo:7]
P: gnupg2 source: very-long-line-length-in-source-file 1092 > 512 [po/hu.gmo:14]
P: gnupg2 source: very-long-line-length-in-source-file 1128 > 512 [po/id.gmo:10]
P: gnupg2 source: very-long-line-length-in-source-file 1240 > 512 [po/pt.gmo:12]
P: gnupg2 source: very-long-line-length-in-source-file 1372 > 512 [po/fi.gmo:14]
P: gnupg2 source: very-long-line-length-in-source-file 1488 > 512 [po/sk.gmo:7]
P: gnupg2 source: very-long-line-length-in-source-file 1544 > 512 [po/ca.gmo:15]
P: gnupg2 source: very-long-line-length-in-source-file 1555 > 512 [g10/t-keydb-get-keyblock.gpg:1091]
P: gnupg2 source: very-long-line-length-in-source-file 1819 > 512 [po/ro.gmo:39]
P: gnupg2 source: very-long-line-length-in-source-file 2016 > 512 [po/et.gmo:14]
P: gnupg2 source: very-long-line-length-in-source-file 2116 > 512 [po/gl.gmo:15]
P: gnupg2 source: very-long-line-length-in-source-file 2832 > 512 [po/el.gmo:11]
P: gnupg2 source: very-long-line-length-in-source-file 2921 > 512 [po/sv.gmo:1671]
P: gnupg2 source: very-long-line-length-in-source-file 3272 > 512 [po/zh_TW.gmo:54]
P: gnupg2 source: very-long-line-length-in-source-file 3508 > 512 [po/da.gmo:62]
P: gnupg2 source: very-long-line-length-in-source-file 3663 > 512 [po/ja.gmo:1530]
P: gnupg2 source: very-long-line-length-in-source-file 3667 > 512 [po/[email-masked]:1555]
P: gnupg2 source: very-long-line-length-in-source-file 3667 > 512 [po/[email-masked]:1550]
P: gnupg2 source: very-long-line-length-in-source-file 3772 > 512 [po/zh_CN.gmo:103]
P: gnupg2 source: very-long-line-length-in-source-file 4054 > 512 [po/cs.gmo:2986]
P: gnupg2 source: very-long-line-length-in-source-file 4227 > 512 [po/de.gmo:3034]
P: gnupg2 source: very-long-line-length-in-source-file 4235 > 512 [po/pl.gmo:3016]
P: gnupg2 source: very-long-line-length-in-source-file 4264 > 512 [po/nb.gmo:83]
P: gnupg2 source: very-long-line-length-in-source-file 4285 > 512 [po/it.gmo:2998]
P: gnupg2 source: very-long-line-length-in-source-file 4324 > 512 [po/fr.gmo:2628]
P: gnupg2 source: very-long-line-length-in-source-file 4344 > 512 [po/ru.gmo:3005]
P: gnupg2 source: very-long-line-length-in-source-file 4564 > 512 [po/uk.gmo:2931]
P: gnupg2 source: very-long-line-length-in-source-file 4628 > 512 [po/tr.gmo:41]
P: gnupg2 source: very-long-line-length-in-source-file 4900 > 512 [po/es.gmo:74]
P: gnupg2 source: very-long-line-length-in-source-file 520 > 512 [tests/gpgsm/cert_dfn_pca15.der:5]
P: gnupg2 source: very-long-line-length-in-source-file 536 > 512 [tests/openpgp/tofu/cross-sigs/871C2247-2.gpg:5]
P: gnupg2 source: very-long-line-length-in-source-file 536 > 512 [tests/openpgp/tofu/cross-sigs/871C2247-3.gpg:7]
P: gnupg2 source: very-long-line-length-in-source-file 536 > 512 [tests/openpgp/tofu/cross-sigs/871C2247-4.gpg:8]
P: gnupg2 source: very-long-line-length-in-source-file 544 > 512 [tests/openpgp/tofu/conflicting/BE04EB2B-secret.gpg:11]
P: gnupg2 source: very-long-line-length-in-source-file 562 > 512 [tests/openpgp/tofu/cross-sigs/EC38277E-1.gpg:3]
P: gnupg2 source: very-long-line-length-in-source-file 562 > 512 [tests/openpgp/tofu/cross-sigs/EC38277E-2.gpg:4]
P: gnupg2 source: very-long-line-length-in-source-file 570 > 512 [tests/openpgp/tofu/conflicting/1C005AF3-secret.gpg:2]
P: gnupg2 source: very-long-line-length-in-source-file 585 > 512 [tests/openpgp/tofu/conflicting/B662E42F.gpg:6]
P: gnupg2 source: very-long-line-length-in-source-file 610 > 512 [tests/openpgp/tofu/cross-sigs/EC38277E-secret.gpg:6]
P: gnupg2 source: very-long-line-length-in-source-file 687 > 512 [tests/openpgp/tofu/conflicting/B662E42F-secret.gpg:8]
P: gnupg2 source: very-long-line-length-in-source-file 692 > 512 [build-aux/speedo/w32/gnupg-logo-150x57.bmp:1]
P: gnupg2 source: very-long-line-length-in-source-file 719 > 512 [g10/t-keydb-keyring.kbx:1]
P: gnupg2 source: very-long-line-length-in-source-file 925 > 512 [build-aux/speedo/w32/gnupg-logo-164x314.bmp:1]

I could add some overrides, but it doesn't really make sense to do so.
I'd prefer it if lintian instead just wouldn't flag non-text source
files with this tag.

We could argue about whether upstream *should* be including non-text
files in source, of course, but there are some not-implausible reasons:

 - some of these files are test vectors (OpenPGP certificates in binary
   form, DER-formatted X.509 certificates, kbx files)

 - some of them are graphics files (gnupg-logo-*.bmp)

 - some of them are GNU message catalogs -- compiled output of .po files
   that upstream prefers to ship in the tarball for folks building the
   package without l10n toolchains.  we rebuild them in debian, but i'd
   still rather ship the upstream tarball if possible.

And i'd rather not try to convince upstream that they should ship a
different tarball.

If there's some way that i need to flag these files as non-text for
lintian's sake, i'm willing to do that, but i don't know how to do it.

          --dkg

---

From: Peter B <[email-masked]>
Date: Fri, 1 Jul 2022 12:27:08 +0100

> lintian's sake, i'm willing to do that, but i don't know how to do it.

Hi,

I'm also seeing this with strawberry. Several hits from binary sound files in it's test suite.

> P: strawberry source: very-long-line-length-in-source-file 1147 > 512 [tests/data/audio/strawberry.oga:126]
> P: strawberry source: very-long-line-length-in-source-file 1297 > 512 [tests/data/audio/strawberry.wv:101]
> P: strawberry source: very-long-line-length-in-source-file 14620 > 512 [tests/data/audio/strawberry.aif:399]
> P: strawberry source: very-long-line-length-in-source-file 2139 > 512 [tests/data/audio/strawberry.opus:44]
> P: strawberry source: very-long-line-length-in-source-file 2140 > 512 [tests/data/audio/strawberry.m4a:39]
> P: strawberry source: very-long-line-length-in-source-file 3435 > 512 [dist/macos/strawberry.icns:5678]
> P: strawberry source: very-long-line-length-in-source-file 4235 > 512 [tests/data/audio/strawberry.flac:5]
> P: strawberry source: very-long-line-length-in-source-file 5165 > 512 [tests/data/audio/strawberry.asf:68]
> P: strawberry source: very-long-line-length-in-source-file 543 > 512 [CMakeLists.txt:535]
> P: strawberry source: very-long-line-length-in-source-file 559 > 512 [data/schema/schema-8.sql:587]
> P: strawberry source: very-long-line-length-in-source-file 566 > 512 [data/schema/schema-11.sql:235]
> P: strawberry source: very-long-line-length-in-source-file 654 > 512 [tests/data/audio/strawberry.spx:48]
> P: strawberry source: very-long-line-length-in-source-file 687 > 512 [3rdparty/SPMediaKeyTap/README.md:4]
> P: strawberry source: very-long-line-length-in-source-file 756 > 512 [3rdparty/SPMediaKeyTap/LICENSE:8]

Cheers,
Peter

---

From: Matt Barry <[email-masked]>
Date: Fri, 01 Jul 2022 17:21:48 -0400

Looking at the check, it seems there is an exemption for SVG files
built in; would it make any sense to search for a text/* mime type
instead (ala libfile-libmagic-perl)?

Cheers,
Matt

---

From: Axel Beckert <[email-masked]>
Date: Sun, 3 Jul 2022 03:38:21 +0200

Control: tag -1 + confirmed
Control: clone -1 -2
Control: retitle -2 lintian: very-long-line-length-in-source-file should use file/magic to distinguish text from binary files
Control: submitter -2 Matt Barry <[email-masked]>
Control: severity -2 wishlist
Control: clone -1 -3
Control: retitle -2 lintian: very-long-line-length-in-source-file should ignore lines starting with INSERT or SELECT (i.e. commonly long SQL statements)
Control: submitter -2 Peter B <[email-masked]>

Hi,

> these non-text files in the gnupg2 sources:

Thanks for this list.

>From my point of view while many of these binary files might not be in
the preferred representation (especially for the .gmo files I'd expect
a plain text file to be the source),
very-long-line-length-in-source-file should not be emitted for binary files.

> I'd prefer it if lintian instead just wouldn't flag non-text source

Correct. Currently this is handled via a blacklist of common binary
file suffixes.

>    still rather ship the upstream tarball if possible.

Yep. Do expect that there will be a future lintian tag for these kind
of files which is meant to be overriden if and only if the build
system rebuilds them at build time.

Matt Barry wrote:
> built in;

At least not at the suffix list.

> would it make any sense to search for a text/* mime type

Yes, that would probably make more sense than manually curating a list
of suffixes. Also the performance impact should be low as Lintian
seems to run "file" over nearly every file anyways.

That's nevertheless not a short term fix. Cloning this bug report into
a new one to track this separately.

> > P: gnupg2 source: very-long-line-length-in-source-file 1008 > 512

Please refrain from doing fullquotes in the Debian bug tracking system
unless really necessary. Thanks!

> > P: strawberry source: very-long-line-length-in-source-file 3435 > 512 [dist/macos/strawberry.icns:5678]

The suffix "icns" is already in the blacklist since 2.115.2. With
which version of Lintian did you generate that list?

> > P: strawberry source: very-long-line-length-in-source-file 756 > 512 [3rdparty/SPMediaKeyTap/LICENSE:8]

These are likely a valid cases.

> > P: strawberry source: very-long-line-length-in-source-file 566 > 512 [data/schema/schema-11.sql:235]

These are corner cases IMHO. Not really binary files, but also files
where long lines are very common, especially for INSERT and SELECT.

I tend to write code which explicitly ignores lines starting with
INSERT or SELECT for that.

		Regards, Axel
-- 
 ,''`.  |  Axel Beckert <[email-masked]>, https://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-    |  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

---

From: Axel Beckert <[email-masked]>
Date: Sun, 3 Jul 2022 03:51:32 +0200

retitle 1014255 lintian: very-long-line-length-in-source-file should use file/magic to distinguish text from binary files
submitter 1014255 Matt Barry <[email-masked]>
retitle 1014256 lintian: very-long-line-length-in-source-file should ignore lines starting with INSERT or SELECT (i.e. commonly long SQL statements)
submitter 1014256 Peter B <[email-masked]>
kthxbye

Sorry, just noticed that I made a copy & paste error and changed title
and submitter twice for -2 instead of using -3 the second time...

Debian Bug Tracking System wrote:
> > clone -1 -2
> Bug #1014156 [lintian] lintian: very-long-line-length-in-source-file for non-text source files
> Bug 1014156 cloned as bug 1014255
> > retitle -2 lintian: very-long-line-length-in-source-file should use file/magic to distinguish text from binary files
> Bug #1014255 [lintian] lintian: very-long-line-length-in-source-file for non-text source files
> Changed Bug title to 'lintian: very-long-line-length-in-source-file should use file/magic to distinguish text from binary files' from 'lintian: very-long-line-length-in-source-file for non-text source files'.
> > submitter -2 Matt Barry <[email-masked]>
> Bug #1014255 [lintian] lintian: very-long-line-length-in-source-file should use file/magic to distinguish text from binary files
> Changed Bug submitter to 'Matt Barry <[email-masked]>' from 'Daniel Kahn Gillmor <[email-masked]>'.
> > severity -2 wishlist
> Bug #1014255 [lintian] lintian: very-long-line-length-in-source-file should use file/magic to distinguish text from binary files
> Severity set to 'wishlist' from 'normal'
> > clone -1 -3
> Bug #1014156 [lintian] lintian: very-long-line-length-in-source-file for non-text source files
> Bug 1014156 cloned as bug 1014256
> > retitle -2 lintian: very-long-line-length-in-source-file should ignore lines starting with INSERT or SELECT (i.e. commonly long SQL statements)
> Bug #1014255 [lintian] lintian: very-long-line-length-in-source-file should use file/magic to distinguish text from binary files
> Changed Bug title to 'lintian: very-long-line-length-in-source-file should ignore lines starting with INSERT or SELECT (i.e. commonly long SQL statements)' from 'lintian: very-long-line-length-in-source-file should use file/magic to distinguish text from binary files'.
> > submitter -2 Peter B <[email-masked]>
> Bug #1014255 [lintian] lintian: very-long-line-length-in-source-file should ignore lines starting with INSERT or SELECT (i.e. commonly long SQL statements)
> Changed Bug submitter to 'Peter B <[email-masked]>' from 'Matt Barry <[email-masked]>'.

		Regards, Axel

---

From: Matt Barry <[email-masked]>
Date: Tue, 05 Jul 2022 20:19:37 -0400

tags -1 + patch
thanks

---

From: "Debian Bug Tracking System" <[email-masked]>
Date: Wed, 06 Jul 2022 00:21:02 +0000

Thank you for the additional information you have supplied regarding
this Bug report.

This is an automatically generated reply to let you know your message
has been received.

Your message has not been forwarded to the package maintainers or
other interested parties; you should ensure that the developers are
aware of the problem you have entered into the system - preferably
quoting the Bug reference number, #1014255.

If you wish to submit further information on this problem, please
send it to [email-masked].

Please do not send mail to [email-masked] unless you wish
to report a problem with the Bug-tracking system.

-- 
1014255: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1014255
Debian Bug Tracking System
Contact [email-masked] with problems

---

From: Andreas Beckmann <[email-masked]>
Date: Wed, 04 Jan 2023 13:35:37 +0100

Followup-For: Bug #1014255

tested with current git (2.115.3-66-ge45c16683):

E: pyopencl source: source-is-missing [_skbuild/linux-x86_64-3.11/cmake-install/pyopencl/_cl.cpython-311-x86_64-linux-gnu.so]
P: pyopencl source: source-contains-prebuilt-binary [_skbuild/linux-x86_64-3.11/cmake-install/pyopencl/_cl.cpython-311-x86_64-linux-gnu.so]
X: pyopencl source: very-long-line-length-in-source-file 21987 > 512 [_skbuild/linux-x86_64-3.11/cmake-install/pyopencl/_cl.cpython-311-x86_64-linux-gnu.so:21]

Just funny that lintian considers the prebuilt binary as source ...

(not going to be uploaded, already fixed in a new pyopencl upstream release)

Andreas

---

From: "Dr. Bas Wijnen" <[email-masked]>
Date: Wed, 2 Aug 2023 21:01:19 +0200

Hi,

Axel writes that README.md and LICENSE are likely valid cases. I disagree.
While I (and I expect many developers in Debian) are used to using short lines
in text files, this is not that common for other people. Many will use the
automatic word-wrap feature of text editors and write a whole paragraph on a
single line.

I don't think I should ask upstream to reformat their non-code text files; it's
not a good use of their time, and also they would likely not even do it.

Instead, I hope lintian can avoid emitting this tag for non-code text files (in
particular: *.txt, *.html, *.md). Although CMakeLists.txt is probably an
exception: that is code and it shouldn't contain long lines.

I could add overrides, but I don't think this would be an appropriate case for
that. But please let me know if it is.

Thanks,
Bas

---

From: Louis-Philippe Véronneau <[email-masked]>
Date: Wed, 2 Aug 2023 15:30:37 -0400

> that. But please let me know if it is.

Note that the 'very-long-line-length-in-source-file' was marked as 
'experimental' in this commit [1] exactly for that kind of reason.

The default lintian mode doesn't use the 'experimental' tags and I would 
advise you not to include them if this tag is currently bothering you.

[1]: 
https://salsa.debian.org/lintian/lintian/-/commit/899bd1b683c479e166ebd465ff0ad101fbd04ee2

-- 
   ⢀⣴⠾⠻⢶⣦⠀
   ⣾⠁⢠⠒⠀⣿⡁  Louis-Philippe Véronneau
   ⢿⡄⠘⠷⠚⠋   [email-masked] / veronneau.org
   ⠈⠳⣄

---

From: Russ Allbery <[email-masked]>
Date: Wed, 02 Aug 2023 13:02:01 -0700

"Dr. Bas Wijnen" <[email-masked]> writes:

> Axel writes that README.md and LICENSE are likely valid cases. I
> disagree.  While I (and I expect many developers in Debian) are used to
> using short lines in text files, this is not that common for other
> people. Many will use the automatic word-wrap feature of text editors
> and write a whole paragraph on a single line.

Or, if not a whole paragraph, there are pretty strong arguments for
following https://xkcd.com/1285/ whenever writing in any markup language,
including Markdown, which can lead to long lines for complex sentences.
That's the standard for our projects at my current employer.

-- 
Russ Allbery ([email-masked])              <https://www.eyrie.org/~eagle/>