[Openembedded-architecture] Changes that switching from smart to dnf will cause

[Alexander Kanavin]

Hello,

I've posted the patchset for replacing the smart package manager with 
dnf package manager to oe-core list, and wanted to write a separate 
email here, laying out what are the changes this will cause and why 
they're necessary or good to have. Please refer to the patchset if you 
want to see implementation details.

1. Smart package manager is replaced by Dnf package manager. Smart has 
become unmaintained upstream, is not ported to Python 3.x and we need to 
find a replacement. Dnf is the only feasible candidate.

The change in functionality is that the on-target package management 
from remote package feeds is now done with a different tool that has a 
different set of command line options. If you have scripts that call the 
tool directly, or use its api (not sure if smart has an api), they need 
to be fixed.

2. Rpm 5.x is replaced with rpm 4.x. This is done for two major reasons:

- DNF is API incompatible with Rpm 5.x and porting it and maintaining 
the port is non-trivial.

- rpm 5.x itself is more or less unmaintained both in the upstream and 
in Yocto. There is no community around it, and Yocto is the sole 
remaining user. I think this is a stronger argument for moving away from 
it, than any technically superior features rpm 5.x may have over 4.x 
(not sure what those would be).

3. Berkeley db 6.x is removed; Berkeley db 5.x becomes default

Version 6.x of Berkeley DB has been rejected by open source community
due to its hostile AGPLv3 license; both Fedora and Debian are sticking 
with db 5.x - and by extension,all the open source projects are still 
developed and tested with db 5.x

In oe-core the only thing that was requiring db 6.x was rpm 5.x, and so
there's no reason to continue carrying db 6.x in oe-core. If someone 
needs API features that are only available in db 6.x, it can be re-added 
to meta-oe.

4. Createrepo is replaced with createrepo_c

createrepo_c is the current incarnation of the tool that generates 
remote repository metadata; it's written in C (createrepo was in 
Python), is faster, and is maintained.

5. architecture-independent .rpm packages are "noarch" instead of "all"

Too many places in dnf/rpm4 stack make that assumption; let's not fight 
against it. It's only about the filenames and the architecture tag in 
them; nothing else has changed in oe-core system, particularly 
allarch.bbclass is untouched.

6. Things that need to run on target during package installation should 
go into pkg_postinst_ontarget()

Previously, the way to achieve this was:

pkg_postinst_PACKAGENAME() {
      if [ x"$D" = "x" ]; then
           # Actions to carry out on the device go here
      else
           exit 1
      fi
}

The new way is simply:

pkg_postinst_ontarget_PACKAGENAME() {
      # Actions to carry out on the device go here
}

The old way still works, but is deprecated and will produce a warning. I 
understand this change is orthogonal to dnf, but the current design is 
flawed and now is a chance to fix it. The reasons are:

1) Code is hard to read; it is not obvious that 'if D is defined then 
exit 1' means 'defer what follows to first boot during package 
cross-install'.

2) Worse, this hides actual errors in the scriptlets; there is no 
difference between scriptlet failing because it's intended to be run on 
target and scriptlet failing because there's a bug or a regression 
somewhere. I've already found broken scriptlets where the brokenness was 
hidden this way (in meta-selftest/recipes-test/postinst/postinst_1.0.bb)

Plain pkg_postinst() without special tricks works exactly as before.

7. There are various rpm-related features that have not been ported over 
yet.

The reason is that they are all disabled by default and don't have a 
test case - I don't want to figure out what they're supposed to do in 
the absence of a clear way to verify them - there's already more than 
enough changes to test and review. Specifically:

- configuring remote package feeds with PACKAGE_FEED_URIS (bug to create 
a test case has been filed)
- signing of remote package feeds using PACKAGE_FEED_SIGN (also has a 
bug to create a test case)
- signing of rpm packages using RPM_SIGN_PACKAGES (this one I will fix 
in the next few days, as I actually did find a testcase)
- RPM_PREFER_ELF_ARCH (have no idea about what this does at all).

... and possibly other features - there's a lot of obscure corners in 
oe-core which I might have missed.

Regards,
Alex