Package: debian-installer
Severity: normal
Tags: d-i
Steps to reproduce:
- Install debian using recently published debian-bookworm-DI-
rc1-amd64-netinst.iso
- At first boot as a very first thing to do type "apt-get install a<Tab>" (as
root, in normal root login shell)
- You will see that bash-completion doesn't work
- If you type "apt-get install apt<Enter>", and try "apt-get install a<Tab>"
again, you will see that bash-completion works
This situation is very bad. Imagine you installed Debian. Of course, the first
thing you may try to do is install some package to it. So it is quite possible
you will type "apt-get install ..." as a first thing. And you will see that
bash-completion doesn't work!
(Don't confuse "apt-get install a<Tab>" with "apt-get insta<Tab>")
I think I know why this happens. d-i has file /usr/lib/finish-
install.d/60cleanup. This file calls "apt-get clean". "apt-get clean" removes
"/var/cache/apt/*.bin". And "/var/cache/apt/*.bin" are needed for bash-
completion to work with apt.
If I remove /usr/lib/finish-install.d/60cleanup, then this bug disappears.
So, if you want to quickly fix this bug, simply delete /usr/lib/finish-
install.d/60cleanup.
I don't know what should really be done. I can list possible solutions:
- Fix d-i. Remove /usr/lib/finish-install.d/60cleanup or replace it with
something else
- Fix "apt-get clean". Make it not remove /var/cache/apt/*.bin
- Fix "bash-completion". Make it work even if /var/cache/apt/*.bin are not
present (for example, by recreating them on-the-fly)
(Disregard "System information" below)
-- System Information:
Debian Release: 9.13
APT prefers oldoldstable
APT policy: (500, 'oldoldstable')
Architecture: amd64 (x86_64)
Kernel: Linux 5.10.0-0.deb9.21-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Acknowledgement sent
to Cyril Brulebois <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <[email protected]>.
(Thu, 20 Apr 2023 20:36:04 GMT) (full text, mbox, link).
Hi Askar,
and thanks for the report.
Askar Safin <[email protected]> (2023-04-20):
> I don't know what should really be done. I can list possible solutions:
>
> - Fix d-i. Remove /usr/lib/finish-install.d/60cleanup or replace it with
> something else
d-i is merely exposing a bug in bash-completion, so I don't think we
should change anything there.
> - Fix "apt-get clean". Make it not remove /var/cache/apt/*.bin
> - Fix "bash-completion". Make it work even if /var/cache/apt/*.bin are
> not present (for example, by recreating them on-the-fly)
Clearly one of these. While I'm no apt maintainer and can't speak for
the semantics behind “apt-get clean”, it seems to me bash-completion's
relying on files that might not be there is the crux of the problem.
Cheers,
--
Cyril Brulebois ([email protected]) <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant
Acknowledgement sent
to Askar Safin <[email protected]>:
Extra info received and forwarded to list. Copy sent to APT Development Team <[email protected]>.
(Fri, 21 Apr 2023 11:09:05 GMT) (full text, mbox, link).
Четверг, 20 апреля 2023, 23:33 +03:00 от "Cyril Brulebois" <[email protected]>:
> Clearly one of these.
It seems apt completion is located in /usr/share/bash-completion/completions/apt, which belongs to apt package.
So, anyway this belongs to apt package, so I reassigned
==
Askar Safin
Acknowledgement sent
to David Kalnischkies <[email protected]>:
Extra info received and forwarded to list. Copy sent to APT Development Team <[email protected]>.
(Thu, 04 May 2023 11:42:03 GMT) (full text, mbox, link).
Hi,
before I am playing bug ping-pong (as I don't think we can do much about
this on the apt side), let me ask a d-i question first:
On Thu, Apr 20, 2023 at 10:33:08PM +0200, Cyril Brulebois wrote:
> Askar Safin <[email protected]> (2023-04-20):
> > I don't know what should really be done. I can list possible solutions:
> >
> > - Fix d-i. Remove /usr/lib/finish-install.d/60cleanup or replace it with
> > something else
>
> d-i is merely exposing a bug in bash-completion, so I don't think we
> should change anything there.
>
> > - Fix "apt-get clean". Make it not remove /var/cache/apt/*.bin
Why is d-i calling 'apt-get clean' to begin with?
Is it perhaps just wanting to remove /var/cache/apt/archives/*.deb ?
If so, could you switch to: -o APT::Keep-Downloaded-Packages=false
(well, probably in a conf file rather than on each command line).
This instructs apt(-get) to remove the deb file right after it has
installed them with dpkg, which might even be beneficial for space
constraint installation targets. At the very least it is a much used
& tested option as it is the default with apt (but not apt-get).
d-i could also "just" run 'apt-cache gencaches' to create the caches;
but perhaps it is intended that the caches are gone (docker people e.g.
like to do that I am told, but they nuke other things as well…).
> > - Fix "bash-completion". Make it work even if /var/cache/apt/*.bin are
> > not present (for example, by recreating them on-the-fly)
The completion script explicitly disables the recreation on-the-fly as
creating the files takes a while robbing users for many seconds of their
interactivity. So, we can't just "fix" the completion script as that has
a(nother?) set of users complain as well.
Best regards
David Kalnischkies
Acknowledgement sent
to Cyril Brulebois <[email protected]>:
Extra info received and forwarded to list. Copy sent to APT Development Team <[email protected]>.
(Thu, 04 May 2023 13:27:03 GMT) (full text, mbox, link).
Hi David,
David Kalnischkies <[email protected]> (2023-05-04):
> before I am playing bug ping-pong (as I don't think we can do much
> about this on the apt side), let me ask a d-i question first:
Entirely fair, thanks. Putting debian-boot@ in copy since currently the
bug is assigned to apt… and others might have some feedback about my
proposal(s).
> Why is d-i calling 'apt-get clean' to begin with?
Checking occurrences of `apt-get clean` (I didn't check for variations)
in our packages, I see two callers.
One is pkgsel (which deals with tasksel and pulls many/most packages),
and one later on in finish-install (which as you can guess runs at the
end).
> Is it perhaps just wanting to remove /var/cache/apt/archives/*.deb ?
> If so, could you switch to: -o APT::Keep-Downloaded-Packages=false
> (well, probably in a conf file rather than on each command line).
There are various apt-get (not clean) calls, some of them wrapped, and I
have no idea what the consequences of tweaking them could be. In any
case, while deplying a conf file during the installation process could
be attempted, it's far too late to try toying with that…
We can probably keep track of the suggestion via some bug report against
debian-installer, so that someone can investigate that post-Bookworm
(probably removing the possible tweak discussed below).
> This instructs apt(-get) to remove the deb file right after it has
> installed them with dpkg, which might even be beneficial for space
> constraint installation targets. At the very least it is a much used
> & tested option as it is the default with apt (but not apt-get).
OK, thanks.
> d-i could also "just" run 'apt-cache gencaches' to create the caches;
> but perhaps it is intended that the caches are gone (docker people e.g.
> like to do that I am told, but they nuke other things as well…).
The finish-install call can be traced back to:
https://bugs.debian.org/586434
> > > - Fix "bash-completion". Make it work even if /var/cache/apt/*.bin are
> > > not present (for example, by recreating them on-the-fly)
>
> The completion script explicitly disables the recreation on-the-fly as
> creating the files takes a while robbing users for many seconds of their
> interactivity. So, we can't just "fix" the completion script as that has
> a(nother?) set of users complain as well.
With all the backstory, the interactions/dependencies between
bash-completion and apt's internal files became clear, thanks.
For Bookworm, I'm tempted to add `apt-cache gencaches` after `apt-get
clean` in finish-install, restoring the files bash-completion relies on.
That'd waste some little time, rebuilding some of the files that were
just deleted… but at least we wouldn't risk anything in the rest of the
installation process.
(Maybe making the extra call conditional to bash-completion's being
installed? Not sure about the “small optimization” vs. “different apt
files depending on the installed packages” balance.)
Cheers,
--
Cyril Brulebois ([email protected]) <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant
Acknowledgement sent
to Askar Safin <[email protected]>:
Extra info received and forwarded to list. Copy sent to APT Development Team <[email protected]>.
(Fri, 05 May 2023 17:06:03 GMT) (full text, mbox, link).
Current apt behavior also breaks official debian docker images, I
reported this here:
https://github.com/debuerreotype/debuerreotype/pull/153 . The images'
developer said this is apt completion scripts' problem
--
Askar Safin
Acknowledgement sent
to Tianon Gravi <[email protected]>:
Extra info received and forwarded to list. Copy sent to APT Development Team <[email protected]>.
(Fri, 05 May 2023 17:57:02 GMT) (full text, mbox, link).
On Fri, 5 May 2023 at 10:06, Askar Safin <[email protected]> wrote:
> Current apt behavior also breaks official debian docker images, I
> reported this here:
> https://github.com/debuerreotype/debuerreotype/pull/153 . The images'
> developer said this is apt completion scripts' problem
To be clear, there was nothing _certain_ in my response, merely that
my first impression matched Cyril's for the very similar issue on d-i.
I'd be very happy to spend some time with APT folks to figure out a
better configuration for the Docker images (both specifically on this
issue and in general!), especially as there seems to be some tension
around our current configuration that was unintentional and I'm sorry
for. 🙇
(Having APT maintainers say "you should do X to accomplish what you're
looking for" is a much more defensible change than anything I could
possibly make up alone! 💕)
♥,
- Tianon
4096R / B42F 6819 007F 00F8 8E36 4FD4 036A 9C25 BF35 7DD4
Acknowledgement sent
to Askar Safin <[email protected]>:
Extra info received and forwarded to list. Copy sent to APT Development Team <[email protected]>.
(Fri, 05 May 2023 18:12:05 GMT) (full text, mbox, link).
David Kalnischkies:
> The completion script explicitly disables the recreation on-the-fly as
> creating the files takes a while robbing users for many seconds of their
> interactivity. So, we can't just "fix" the completion script as that has
> a(nother?) set of users complain as well.
Deleting /var/cache/apt/*.bin and then running "apt-cache gencaches"
takes 0.22 on my (loaded) system.
I propose regenerate caches in completion scripts only if they are not
present. This will change situation from "not working at all" (in
default d-i install and in official docker image) to "working slowly".
Also, "working slowly" is just minor annoyance. But "not working"
means that the user doesn't know how to fix. Yes, merely calling
"apt-get install apt" will regenerate caches (in d-i case, not in
docker case), but how the user should know this? The solution is not
discoverable
--
Askar Safin
Acknowledgement sent
to Askar Safin <[email protected]>:
Extra info received and forwarded to list. Copy sent to APT Development Team <[email protected]>.
(Fri, 05 May 2023 18:15:02 GMT) (full text, mbox, link).
On Fri, May 5, 2023 at 8:53 PM Tianon Gravi <[email protected]> wrote:
> To be clear, there was nothing _certain_ in my response, merely that
> my first impression matched Cyril's for the very similar issue on d-i.
Okay, sorry for misunderstanding.
--
Askar Safin
Acknowledgement sent
to Cyril Brulebois <[email protected]>:
Extra info received and forwarded to list. Copy sent to APT Development Team <[email protected]>.
(Sat, 06 May 2023 01:03:02 GMT) (full text, mbox, link).
Control: clone -1 -2
Control: reassign -2 debian-installer
Control: retitle -2 debian-installer: investigate setting APT::Keep-Downloaded-Packages=false
Hi again,
Cyril Brulebois <[email protected]> (2023-05-04):
> David Kalnischkies <[email protected]> (2023-05-04):
[…]
> Entirely fair, thanks. Putting debian-boot@ in copy since currently
> the bug is assigned to apt… and others might have some feedback about
> my proposal(s).
No feedback in either direction at the moment, I'm going ahead as the
Bookworm clock is ticking.
> > Is it perhaps just wanting to remove /var/cache/apt/archives/*.deb ?
> > If so, could you switch to: -o APT::Keep-Downloaded-Packages=false
> > (well, probably in a conf file rather than on each command line).
>
> There are various apt-get (not clean) calls, some of them wrapped, and
> I have no idea what the consequences of tweaking them could be. In any
> case, while deplying a conf file during the installation process could
> be attempted, it's far too late to try toying with that…
>
> We can probably keep track of the suggestion via some bug report
> against debian-installer, so that someone can investigate that
> post-Bookworm (probably removing the possible tweak discussed below).
Doing so via the control commands above.
> For Bookworm, I'm tempted to add `apt-cache gencaches` after `apt-get
> clean` in finish-install, restoring the files bash-completion relies
> on.
I'll do that in finish-install and add a pointer to this bug report; I
won't add a Closes statement though, as I'm not sure what other users
will be implementing (for Docker etc.)… I'll leave it up to you to
decide what's best regarding this bug report (the -1 one).
Cheers,
--
Cyril Brulebois ([email protected]) <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant
Acknowledgement sent
to Askar Safin <[email protected]>:
Extra info received and forwarded to list. Copy sent to APT Development Team <[email protected]>.
(Sat, 06 May 2023 14:09:02 GMT) (full text, mbox, link).
I did some experiments and wrote a patch!
Okay, so, I spend some more time thinking about this. And I think the
bug is in apt's completion scripts. Because that *.bin files are
simply cache. And lack of caches should not break program! This
follows from my interpretation of FHS 3.0 5.5 (
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s05.html ). And
that *.bin files are located in /var/cache. And debian policy refers
to FHS 3.0 ( https://www.debian.org/doc/debian-policy/ch-opersys.html#file-system-hierarchy
). So, the bug is in apt's completion scripts and we should simply
remove "--no-generate" from them. And hence I attach a patch, which
does exactly that.
This will cause completion scripts to regenerate caches as needed.
This will work even if *.bin files are not present. And this will work
even if we are in unprivileged bash, I tested this.
Okay, so you can ask: what are speed implications of this? Well, I
tested this on my main system (Intel Core i7) and I feel that even
full cache regeneration takes zero time, i. e. it is absolutely
instant.
So I started qemu-system-x86_64 without -enable-kvm and tested there.
Results are this: if caches are not present, then bash-completion is
slow, it takes something like 10 seconds (in qemu in emulation
mode!!!). But if caches are present, then bash-completion feels fast.
So, it seems that apt is smart, it regenerates cache only if needed.
So, there is no need to insert special "if" in completion scripts to
check whether we should regenerate caches, apt already does this.
Also, I found that file /usr/share/bash-completion/completions/apt is
owned by apt, but file /usr/share/bash-completion/completions/apt-get
is owned by bash-completion . So, I apply patch to "apt" only. Needed
patch for "apt-get" is essentially the same.
Also, I think "apt-get" (and similar) scripts should be transferred
from "bash-completion" package to "apt".
This patch is for current apt's "main",
9da15d149f97e0f26cf5b7e32405512a5e63523c .
diff --git a/completions/bash/apt b/completions/bash/apt
index 59a8eaa94..ff3d7bd24 100644
--- a/completions/bash/apt
+++ b/completions/bash/apt
@@ -173,6 +173,7 @@ _apt()
fi
# specific command arguments
+ # we don't pass "--no-generate" to "apt-cache" because we want
completion to work even if caches are not present (#1034650)
if [[ -v command ]]; then
case $command in
remove|purge|autoremove|autopurge)
@@ -187,7 +188,7 @@ _apt()
return 0
;;
show|list|download|changelog|depends|rdepends)
- COMPREPLY=( $( apt-cache --no-generate pkgnames "$cur" \
+ COMPREPLY=( $( apt-cache pkgnames "$cur" \
2> /dev/null ) )
return 0
;;
@@ -195,7 +196,7 @@ _apt()
if [[ "$cur" == .* || "$cur" == /* || "$cur" == ~* ]]; then
_filedir "deb"
else
- COMPREPLY=( $( apt-cache --no-generate pkgnames "$cur" \
+ COMPREPLY=( $( apt-cache pkgnames "$cur" \
2> /dev/null ) )
fi
return 0
@@ -204,7 +205,7 @@ _apt()
if [[ "$command" == build-dep && ( "$cur" == .* ||
"$cur" == /* || "$cur" == ~* ) ]]; then
_filedir "dsc"
else
- COMPREPLY=( $( apt-cache --no-generate pkgnames "$cur" \
+ COMPREPLY=( $( apt-cache pkgnames "$cur" \
2> /dev/null ) $( apt-cache dumpavail | \
command grep "^Source: $cur" | sort -u | cut
-f2 -d" " ) )
fi
Acknowledgement sent
to sisto tina <[email protected]>:
Extra info received and forwarded to list. Copy sent to APT Development Team <[email protected]>.
(Wed, 09 Oct 2024 11:27:02 GMT) (full text, mbox, link).
Debbugs is free software and licensed under the terms of the GNU General
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.