[OE-core] [PATCH] package_manager.py: Explicit complementary fail

Mauno Niko niko.mauno at vaisala.com
Wed Mar 7 09:45:00 UTC 2018


On 03/02/2018 03:59 PM, Mauno Niko wrote:
> On 02/28/2018 06:58 PM, Alexander Kanavin wrote:
>> On 02/28/2018 06:20 PM, Niko Mauno wrote:
>>>
>>> Improve associated install() method behaviour in affected OpkgPM and
>>> DpkgPM classes so that a problematic state of affairs becomes directly
>>> obvious for bitbake user, resulting in shell output like:
>>
>> Does rpm need the same fix?
> 
> RpmPM's respective install() implementation deviates somewhat from it's OpkgPM, DpkgPM counterparts. It seems that rpm applies '--skip-broken' switch to dnf command which supposedly ignores just the conflicting packages in a given set, while installing all others.
> 
> The behaviour in opkg/dpkg case however seems to be, that in case of even single conflicting package in a package set, the whole package set is silently left uninstalled. I performed a test against a reference image where the issue manifests, and compared to rpm, opkg/dpkg-based SDK images lacked over half of target packages:
> 
> $ wc -l toolchain-target-manifest-*
>    317 toolchain-target-manifest-deb
>    317 toolchain-target-manifest-ipk
>    653 toolchain-target-manifest-rpm
> 

Steps to reproduce problem with current poky master (at e5f258cecef6c51a4d7ebeedbaa29d2126e566e3):

1) Add following line to build/conf/local.conf:
IMAGE_INSTALL_append = " openssh-sftp-server"

2) Perform 'bitbake -c populate_sdk core-image-sato' against each 'package_deb', 'package_rpm' and 'package_ipk' values of PACKAGE_CLASSES variable and make note of contents of each resulting build/tmp/deploy/sdk/poky-glibc-x86_64-core-image-sato-i586-toolchain-2.4+snapshot.target.manifest files.

3) Compare resulting target.manifest files and notice that vast amount of target packages (mostly *-dev, *-dbg, but others too) are missing from SDK when PACKAGE_CLASSES is set to ipk or deb:

# wc -l sdk-sato-*/*.target.manifest 
  601 sdk-sato-deb/....target.manifest
  601 sdk-sato-ipk/....target.manifest
 1413 sdk-sato-rpm/....target.manifest


In OpkgPM, DpkgPM cases, install_complementary() seems to fail the whole set (without QA warnings) if there is even one conflicting package (such as 'openssh' in this case). 

In RpmPM case, implementation deviates somewhat from it's OpkgPM, DpkgPM counterparts - a '--skip-broken' switch is applied to dnf command which effectively would seem to (silently) leave out only the conflicting 'openssh' package (without QA warnings), instead of whole set. 

Without this patch, there is no explicit notification of shortcoming to bitbake user. 

With this patch applied, warnings are printed to console during bitbake operation (full output in appendix below) in package_ipk, package_deb cases. This should be helpful to bitbake user who then can mitigate the situation (eg. by adding 'openssh' to 'PACKAGE_EXCLUDE_COMPLEMENTARY' variable).

--

Appendix 1: PACKAGE_CLASSES = "package_ipk"

WARNING: core-image-sato-1.0-r0 do_populate_sdk: Unable to install packages. Command '/home/oe/branches/upstream/master/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/recipe-sysroot-native/usr/bin/opkg --volatile-cache -f /home/oe/branches/upstream/master/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/opkg.conf -t /home/oe/branches/upstream/master/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/temp/ipktemp/ -o /home/oe/branches/upstream/master/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/sdk/image/opt/poky/2.4+snapshot/sysroots/i586-poky-linux  --force_postinstall --prefer-arch-to-version   install sato-screenshot-dbg consolekit-dbg libpopt-dev libmb-dev libgles2-mesa-dev initscripts-dev gcc-runtime-dbg packagegroup-base-wifi-dev openssh-dev matchbox-panel-2-dev iptables-dbg libglapi-dev packagegroup-base-pci-dev libtag-dev libfontenc-dbg libnss-mdns-dbg xcursor-transparent-theme-dev libmb-dbg sysvinit-inittab-dbg libstartup-notification-1-dbg libexif-dev iw-dev libnss-mdns-dev libx11-dev distcc-dbg libatk-1.0-dbg libsqlite3-dev libxinerama-dev vte-dbg bash-dev xf86-video-vmware-dev libglib-2.0-dev libdbus-glib-1-dev util-linux-dev bzip2-dev libz-dbg libxtst-dbg kmod-dbg orc-dbg ca-certificates-dev libxkbfile-dbg packagegroup-core-x11-base-dev xf86-video-cirrus-dev libfakekey-dbg libpcre-dbg libnl-3-dbg libevdev-dbg libogg-dev libdbus-glib-1-dbg netbase-dev shared-mime-info-dbg packagegroup-core-boot-dev libxxf86vm-dev xrandr-dbg packagegroup-base-zeroconf-dev pcmanfm-dev binutils-dev packagegroup-core-ssh-dropbear-dev libinput-dev libxxf86dga-dev pointercal-xinput-dev sysvinit-inittab-dev libxcursor-dbg xauth-dbg busybox-dev shadow-securetty-dev connman-conf-dbg libxcursor-dev psplash-dev libgcrypt-dev libwayland-egl-dev libfm-extra-dev libinput-dbg libtirpc-dbg libtirpc-dev libarchive-dev packagegroup-base-zeroconf-dbg gtk+3-dbg libxfixes-dev opkg-dbg gstreamer1.0-plugins-base-dev base-passwd-dev curl-dev shadow-securetty-dbg sysvinit-dev avahi-dbg libgbm-dev netbase-dbg liblzo2-dbg mini-x-session-dev modutils-initscripts-dbg libwrap-dev libstartup-notification-1-dev acl-dev udev-extraconf-dev packagegroup-base-dbg libspeex-dbg libxml2-dbg libwebp-dev openssh-dbg libusb-1.0-dev packagegroup-core-boot-dbg libnfsidmap-dbg packagegroup-base-bluetooth-dev xf86-video-fbdev-dbg libxcb-dev libdmx-dbg attr-dev packagegroup-core-x11-sato-games-dev xdpyinfo-dev libevent-dbg xserver-nodm-init-dev vte-dev alsa-state-dbg matchbox-panel-2-dbg libarchive-dbg alsa-lib-dbg formfactor-dev pointercal-xinput-dbg connman-dbg libpng16-dbg libusb-1.0-dbg libxxf86vm-dbg libxdamage-dbg wpa-supplicant-dbg opkg-arch-config-dev packagegroup-core-x11-sato-dev xmodmap-dev libmenu-cache-dbg libgudev-1.0-dbg libpixman-1-dev modutils-initscripts-dev libxi-dev libsbc-dev libxcb-dbg puzzles-dev libcroco-dbg xmodmap-dbg pciutils-dbg ncurses-dbg xinput-dbg libxv-dbg gconf-dev init-ifupdown-dbg libnl-3-dev packagegroup-base-usbgadget-dev xhost-dbg libxcb-util-dbg packagegroup-core-device-devel-dbg libdmx-dev base-files-dbg libepoxy-dev gstreamer1.0-plugins-bad-dev pulseaudio-dbg libatk-bridge-2.0-dev libxkbcommon-dbg wayland-dev wayland-dbg gst-examples-dbg libgcc-s-dbg xserver-xf86-config-dbg hicolor-icon-theme-dbg pcmanfm-dbg libxcomposite-dbg util-linux-dbg xinput-calibrator-dbg opkg-arch-config-dbg ofono-dbg l3afpad-dbg libxmu-dbg xset-dev distcc-dev libxcomposite-dev avahi-dev matchbox-keyboard-dbg libmenu-cache-dev target-sdk-provides-dummy-dbg libxfixes-dbg xinput-calibrator-dev libfm-extra-dbg mtdev-dev alsa-lib-dev libpng16-dev eudev-dev libxv-dev libfontconfig-dev nfs-export-root-dbg packagegroup-core-nfs-server-dbg libatspi-dev glibc-locale-dbg alsa-utils-dbg libcairo-dbg xinput-dev libevdev-dev libxext-dev libgnutls-dev attr-dbg flac-dev acl-dbg xz-dev libspeexdsp-dbg matchbox-terminal-dev xf86-input-libinput-dbg libwebp-dbg iptables-dev xset-dbg libxxf86dga-dbg ncurses-dev sysvinit-dbg formfactor-dbg libatspi-dbg libfreetype-dbg libgpg-error-dbg libvorbis-dbg xserver-xorg-dev libpcre2-dbg libsndfile-d
Collected errors:
 * Solver encountered 1 problem(s):
 * Problem 1/1:
 *   - package openssh-dev-7.6p1-r0.i586 requires openssh = 7.6p1-r0, but none of the providers can be installed
 * 
 * Solution 1:
 *   - allow deinstallation of packagegroup-core-ssh-dropbear-1.0-r1.all

 *   - do not ask to install a package providing dropbear-dev

 *   - do not ask to install a package providing packagegroup-core-ssh-dropbear-dev

 * Solution 2:
 *   - do not ask to install a package providing openssh-dev


Appendix 2: PACKAGE_CLASSES = "package_deb"

WARNING: core-image-sato-1.0-r0 do_populate_sdk: Unable to install packages. Command '/home/oe/branches/upstream/master/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/recipe-sysroot-native/usr/bin/apt-get  install --force-yes --allow-unauthenticated nettle-dev busybox-dbg distcc-config-dev libnfsidmap-dbg alsa-plugins-dbg attr-dbg libspeex-dev libmenu-cache-dbg libxi-dev settings-daemon-dbg wireless-tools-dbg xcursor-transparent-theme-dbg libnl-3-dev pciutils-dev libsbc-dbg libxcomposite-dev libmb-dbg libnfsidmap-dev glibc-locale-dbg gstreamer1.0-plugins-good-dev ca-certificates-dbg libdrm-dev libffi-dev libsndfile-dev orc-dev libspeexdsp-dev shutdown-desktop-dbg libnss-mdns-dev libunistring-dev xdpyinfo-dev packagegroup-core-nfs-server-dev nfs-utils-dbg openssl-dbg settings-daemon-dev libcairo-dev libgpg-error-dbg gstreamer1.0-dbg libfakekey-dbg matchbox-panel-2-dbg util-linux-dbg libtool-dev libreadline-dbg libxcb-dev packagegroup-core-ssh-dropbear-dbg libusb-1.0-dev libpng16-dbg packagegroup-base-alsa-dbg libfm-extra-dbg eudev-dev libpixman-1-dev libxtst-dev alsa-lib-dev matchbox-wm-dev libogg-dev pciutils-dbg pcmanfm-dbg packagegroup-core-x11-sato-apps-dev vte-dev libdbus-glib-1-dev libice-dev libpcre2-dev packagegroup-core-x11-utils-dev sato-screenshot-dbg xinput-dbg libxcomposite-dbg packagegroup-base-nfs-dev alsa-utils-dev sato-screenshot-dev libgdk-pixbuf-2.0-dev alsa-state-dbg flac-dbg dbus-dev shared-mime-info-dbg libatk-1.0-dev packagegroup-base-nfs-dbg sysvinit-dev packagegroup-base-bluetooth-dbg libxrandr-dbg libtag-dev wayland-dbg netbase-dev dpkg-dbg mtdev-dev sudo-dbg libgpg-error-dev libtirpc-dbg libxv-dev libxext-dev packagegroup-distro-base-dbg xset-dbg xf86-video-vmware-dbg flac-dev matchbox-theme-sato-dev perl-dev libtirpc-dev librsvg-2-dev packagegroup-machine-base-dbg libffi-dbg libxau-dev opkg-utils-dbg libxft-dbg libdmx-dbg wpa-supplicant-dev packagegroup-machine-base-dev libxau-dbg attr-dev matchbox-terminal-dbg shared-mime-info-dev libpciaccess-dbg dropbear-dbg libmb-dev libwebp-dev run-postinsts-dbg xf86-input-libinput-dbg libpng16-dev libgles2-mesa-dev gconf-dbg libxmu-dev xinput-calibrator-dev libstartup-notification-1-dbg xdpyinfo-dbg libxxf86dga-dbg packagegroup-core-x11-utils-dbg bluez5-dbg xset-dev libxdmcp-dev libexif-dev libcap-dbg libsm-dbg nfs-export-root-dbg libexpat-dbg libpcre-dbg libxcursor-dev target-sdk-provides-dummy-dev xauth-dbg libharfbuzz-dbg packagegroup-base-ipv6-dbg libgcrypt-dev xrandr-dev matchbox-keyboard-dev libxxf86dga-dev libgnutls-dev rpcbind-dev shadow-dev base-passwd-dbg nfs-utils-dev sysvinit-inittab-dev packagegroup-base-ipv6-dev libxtst-dbg libgudev-1.0-dev libevdev-dbg packagegroup-core-x11-base-dev packagegroup-base-dev libexif-dbg apt-dev libspeexdsp-dbg kmod-dbg matchbox-session-sato-dev libsqlite3-dev libxxf86misc-dev xhost-dev libxi-dbg consolekit-dbg psplash-dbg pulseaudio-client-conf-sato-dev libxkbcommon-dev mobile-broadband-provider-info-dev udev-extraconf-dbg packagegroup-core-nfs-server-dbg l3afpad-dbg packagegroup-core-ssh-dropbear-dev libgmp-dbg sudo-dev libpopt-dev bluez5-dev libxinerama-dev libxshmfence-dev v86d-dbg libdbus-glib-1-dbg xserver-xorg-dbg busybox-dev libevent-dbg packagegroup-core-x11-xserver-dev shadow-securetty-dev packagegroup-distro-base-dev bash-dev libcairo-dbg openssl-dev nfs-export-root-dev bzip2-dev xserver-xf86-config-dev connman-conf-dbg connman-dev libevdev-dev libxfont2-dev libfontconfig-dev libinput-dbg libfontenc-dev ca-certificates-dev connman-dbg gstreamer1.0-plugins-good-dbg matchbox-terminal-dev shadow-dbg libxdmcp-dbg shutdown-desktop-dev packagegroup-core-boot-dbg libdrm-dbg xinput-calibrator-dbg alsa-utils-dbg libxdamage-dev xcursor-transparent-theme-dev libstartup-notification-1-dev alsa-state-dev packagegroup-base-zeroconf-dbg connman-gnome-dev glib-networking-dbg packagegroup-core-x11-sato-games-dev packagegroup-base-usbgadget-dbg libjpeg-dbg xhost-dbg util-linux-dev matchbox-wm-dbg mini-x-session-dbg packagegroup-base-zeroconf-dev packagegroup-core-boot-dev xmodmap-dbg libxinerama-dbg vte-dbg 
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 openssh-dev : Depends: openssh (= 7.6p1-r0) but it is not going to be installed
               Recommends: libcrypto-dev but it is not installable
               Recommends: openssh-keygen-dev but it is not installable
               Recommends: openssh-misc-dev but it is not installable
               Recommends: openssh-scp-dev but it is not installable
               Recommends: openssh-sftp-dev but it is not installable
               Recommends: openssh-sftp-server-dev but it is not installable
               Recommends: openssh-ssh-dev but it is not installable
               Recommends: openssh-sshd-dev but it is not installable
               Recommends: openssl10-dev but it is not installable
               Recommends: shadow-sysroot-dev but it is not installable
               Recommends: update-alternatives-opkg-dev but it is not installable
E: Unable to correct problems, you have held broken packages.




More information about the Openembedded-core mailing list