Subject: debhelper: Does not support double build (possible violation of Policy 4.9)

From: Helge Kreutzmann <[email-masked]>
Date: Sun, 22 Oct 2023 16:58:40 +0000

Package: debhelper
Version: 13.11.4
Severity: minor
User: [email-masked]
Usertags: qa-doublebuild

Some time ago my package linuxinfo aquired a bug (#1047898) whereas
dpkg-buildpackage ; dpkg-buildpackage -S 
does not work. And if fact, it does not.

The reporter informed me that this is a potential violation of Debian
Policy section 4.9 (clean target), but that this was under discussion
at the time of the reporting. 

I first checked (my) upstream build system. Except for one stamp file
(which is *much* less than done by debhelper) the build is idempotent,
i.e.:

./configure && make && make clean

returns the sources into the state as shipped.

In the past I had an explicit debian/rules, but to ensure compliance
with the ever evolving Debian Policy I made it on purpose very simple:

include /usr/share/dpkg/architecture.mk

ifeq ($(DEB_HOST_ARCH),amd64)
        confflags += --with-pcre
else ifeq ($(DEB_HOST_ARCH),i386)
        confflags += --with-pcre
else ifeq ($(DEB_HOST_ARCH),x32)
        confflags += --with-pcre
endif

%:
        dh $@

override_dh_auto_configure:
                dh_auto_configure -- $(confflags)

As you can see, I only add (architecture dependend) configuration flags 
for linking with pcre.

dh seems to delete quite a few files shipped in the package.

For me, this is a clear bug in dh, as linuxinfo just uses it plain and
there is no "manipulation" of build files happening (on purpose).

I checked dh_clean(1) and dh(1), but could not find any mention of how 
to modify this (which I would not have expected anyhow).

If the severity of 1047898 is changed, then I will change this one (as
it is the root cause). In linuxinfo I probably could work around this,
by backing up all affected files before clean and restoring them after
clean (using an override). But this is a band aid, not a solution.

-- 
      Dr. Helge Kreutzmann                     [email-masked]
           Dipl.-Phys.                   http://www.helgefjell.de/debian.php
        64bit GNU powered                     gpg signed mail preferred
           Help keep free software "libre": http://www.ffii.de/

---

From: Niels Thykier <[email-masked]>
Date: Wed, 25 Oct 2023 15:32:47 +0200

Control: tags -1 wontfix

Helge Kreutzmann:
> [...]
> [...]

Deleting files shipped in the package is a non-issue (dpkg ignores that 
with a warning).

> there is no "manipulation" of build files happening (on purpose).

The root cause seems to be that the upstream tarball contains binary 
".gmo" files that will be regenerated when you build with dh and are not 
reset/deleted when you clean.

Most likely this is a consequence of dh_autoreconf (which is not 
debhelper but a separate package that debhelper depends on) or 
`dh_auto_clean` using `make distclean` rather than `make clean` (as you 
tested with).

> to modify this (which I would not have expected anyhow).

Both `dh_clean` and `dh` are red herrings in this case.

> clean (using an override). But this is a band aid, not a solution.

 From my point of view, running an upstream build system is running 
arbitrary code. There is no way debhelper can reliably detect and manage 
all possible variations of upstreams and how they implement 
"./configure" vs. whether they include binary .gmo files in their source 
tarball that may or may not be regenerated during build and how they 
structure their source code internally.
   As a consequence, it is my view that the root cause is not solely 
debhelper and that you will have to work around this in your package 
somehow. This could be by:

   * Disabling dh features that cause this problem, OR by

   * Telling dh_clean to purge the `*.gmo` files
     (`echo 'po/*.gmo' > debian/clean` should do), OR by

   * Repacking the upstream tarball to not include the binary files being
     mutated in the first place.

As the maintainer of debhelper, this is my principle stance on this 
matter. If you disagree, you are welcome to either:

  * provide a "small non-intrusive" patch that solves your problem
    without causing a significant number of regressions. Onus is on
    you (whoever providing the patch) to research alternatives, and, if
    the patch is controversial/likely to break other packages, provide
    archive-wide test results and as necessary show project wide
    consensus on debian-devel, OR

  * raise this issue to the tech-ctte according to constitution 6.1.1
    (AFAICT). However, if you do put this before the tech-ctte, be
    advised that they cannot do detailed design work (constitution
    6.3.5). This means for them to make a decision someone else has to
    come up with a practical technical solution that they can vote in
    over the status quo. And by doing that, that someone might as well
    provide a patch per the first bullet point because the tech-ctte
    will probably have the same questions/requirements as I laid out
    above.

Best regards,
Niels

---

From: Helge Kreutzmann <[email-masked]>
Date: Wed, 25 Oct 2023 17:11:31 +0000

Hello Niels,
thanks for your reply.

Am Wed, Oct 25, 2023 at 03:32:47PM +0200 schrieb Niels Thykier:
> Deleting files shipped in the package is a non-issue (dpkg ignores that with
> a warning).

Ok, so then #1047898 is "just" about the gmo files.

> The root cause seems to be that the upstream tarball contains binary ".gmo"
> files that will be regenerated when you build with dh and are not

Thanks. Then I check how to deal with them (possibly upstream).

> Most likely this is a consequence of dh_autoreconf (which is not debhelper
> but a separate package that debhelper depends on) or `dh_auto_clean` using
> `make distclean` rather than `make clean` (as you tested with).

Thanks for your pointer.

> Both `dh_clean` and `dh` are red herrings in this case.

Well, I'm trying to find the root cause, and as stated I just use
plain dh, so I don't know where I should have looked at. Maybe you
could introduce a pointer to better / correct documentation then?

> From my point of view, running an upstream build system is running arbitrary
> code. There is no way debhelper can reliably detect and manage all possible
> variations of upstreams and how they implement "./configure" vs. whether
> they include binary .gmo files in their source tarball that may or may not
> be regenerated during build and how they structure their source code
> internally.

This is a plain autconf system. When I took it over (as upstream), I
tried to keep as close to the (info) documentation as possible on any
extension.

Though, I admit, that I only "understand" a small part of it.

>   * Disabling dh features that cause this problem, OR by

Which I don't know, but 

>     (`echo 'po/*.gmo' > debian/clean` should do), OR by

Thanks, this is a good hint!

>     mutated in the first place.

As I'm upstream as well, I can update the upstream system to do this.

> As the maintainer of debhelper, this is my principle stance on this matter.
> If you disagree, you are welcome to either:
> 

Sorry, I'm not a "real" programmer. I just respond to #1047898 where I
was informed that an RC bug might come in. I'm currently just learning
where the problem is and I have no idae of the inner working of dh.

>    will probably have the same questions/requirements as I laid out

Well, I want to fix #1047898. And until now I (mis)understood that the
best package guidance is to use dh, as it encodes policy. If you say
this is not the case, then I mainly learned that this is not the case.

And again, thanks for the pointers.

Greetings

         Helge