Subject: Bug#960738: simple-cdd: Does not mirror non-free or contrib

From: Oliver <[email-masked]>
Date: Fri, 15 May 2020 17:39:44 -0700

Package: simple-cdd
Version: 0.6.7
Severity: normal
Tags: d-i

Dear Maintainer,

I am trying to build a CD via simple-cdd by running

build-simple-cdd --conf ./cd.conf

The contents of that conf file are:

mirror_components="non-free main contrib"
server=debian.osuosl.org
debian_mirror="http://debian.osuosl.org/debian/"
locale="en_US"
keyboard=us
dist=buster
profiles=revl
auto_profiles=revl

It does not mirror non-free or contrib. The build exits with a valid CD
but outputs:

2020-05-15 16:24:57 ERROR Last 2 lines of standard error:
2020-05-15 16:24:57 ERROR distcheck:: Fatal error in module common/input.ml:
2020-05-15 16:24:57 ERROR distcheck::  Input file /home/oliver/Documents/code/revl/configs/site-box/preseed/tmp/cd-build/buster/CD1/dists/buster/contrib/binary-amd64/Packages.gz does not exist.

It is also the case that neither `non-free` nor `contrib` is not mirrored and the
`tmp/mirror/dists/buster/X/ ../Packages.gz` file exists but is empty for
X = non-free or contrib.

I did notice that if I change `mirror_components="main contrib
non-free"` to `mirror_components="non-free main"` then I do get non-free
and main packages and it all seems to work (luckily I don't actually
need anything from contrib).

-- System Information:
Debian Release: 10.4
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.19.0-9-amd64 (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE= (charmap=UTF-8)
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages simple-cdd depends on:
ii  dctrl-tools         2.24-3
ii  debian-cd           3.1.25
ii  lsb-release         10.2019051400
ii  python3             3.7.3-1
ii  python3-simple-cdd  0.6.7
ii  reprepro            5.3.0-1
ii  rsync               3.1.3-6
ii  wget                1.20.1-1.1

Versions of packages simple-cdd recommends:
ii  dose-distcheck  5.0.1-12

Versions of packages simple-cdd suggests:
pn  qemu-system | qemu-kvm  <none>

-- no debconf information

---

From: Imran Hussain <[email-masked]>
Date: Sat, 16 May 2020 20:39:28 +0100

Hi,

I stumbled into this too, thanks to your report I was able to work out 
the following:

It seems to be the behaviour is that unless you are using packages from 
those specific components then that local component-mirror doesn't get 
created properly therefore you end up with errors, but a working CD.

The only component-mirrors that fail to get created properly are the 
ones with 0 packages anyway, so these errors are ignorable.

Thanks,
Imran

---

From: Oliver Dain <[email-masked]>
Date: Sat, 16 May 2020 13:20:29 -0700

I think I figured this out. It seems that the contents of `Packages` and
`Packages.gz` in tmp/mirror/dists/buster/non-free/binary-amd64/Packages is
the intersection of the packages available in that section and the set of
packages selected in a NAME.packages profile. If that intersection is empty
the CD build fails. When I added some packages from non-free things started
working again. The ordering issue reported in the original bug seems to be
a red herring.

On Fri, May 15, 2020 at 5:42 PM Oliver <[email-masked]> wrote:

> 2020-05-15 16:24:57 ERROR distcheck::  Input file
> /home/oliver/Documents/code/revl/configs/site-box/preseed/tmp/cd-build/buster/CD1/dists/buster/contrib/binary-amd64/Packages.gz
> does not exist.
>