[OE-core] ptest with BBCLASSEXTEND
Richard Purdie
richard.purdie at linuxfoundation.org
Thu May 16 10:54:09 UTC 2019
We have a need to check the dependencies of our ptest packages. The
best way to do that is to take a minimal image and run them under that
in isolation. That would be a bit tedious and boring.
It got me thinking about creative ways to do that and I thought I'd
share the result.
I created an image recipe:
$ cat p-t-image.bb
require recipes-core/images/core-image-minimal.bb
require conf/distro/include/ptest-packagelists.inc
PTESTS = "${PTESTS_SLOW} ${PTESTS_FAST}"
ALL_PTESTS = "${@' '.join(['p-t-image-'+x for x in d.getVar('PTESTS').split()])}"
IMAGE_INSTALL_append = " ${MCNAME} openssh"
BBCLASSEXTEND = "${@' '.join(['mcextend:'+x for x in d.getVar('PTESTS').split()])}"
IMAGE_OVERHEAD_FACTOR = "1.0"
IMAGE_ROOTFS_EXTRA_SPACE = "1324288"
# ptests need more memory than standard to avoid the OOM killer
QB_MEM = "-m 1024"
Its not fast to parse this creates about 59 variants of the image. Its
(ab)using the mcextend class to an image and add the ptest to that
image for each ptest in the PTESTS_SLOW and PTESTS_FAST variables. It
also adds openssh since we need an ssh client in the images for
testimage to work. I also had to add extra space and memory to the
image config so all the tests could work.
You can get a list of the target images with:
bitbake p-t-image-elfutils-ptest -e | grep ALL_PTESTS=
then run:
bitbake <targets>
and then
bitbake <targets> -c testimage
which batch build and then runs all the images in parallel.
The results also get aggregated automatically into the json file so if
you delete it before starting, the json file is equavalient from a
ptest perspective to if they'd all been run in the same image.
The results I got from this are:
--------------------------------------------------------------------------------------------------------------
Recipe | Passed | Failed | Skipped | Time(s)
--------------------------------------------------------------------------------------------------------------
acl | 1 | 1 | 0 | 0
apr | 0 | 0 | 0 | -
apr-util | 4 | 0 | 0 | -
attr | 0 | 0 | 0 | 0
bash | 36 | 8 | 0 | 107 T
bluez5 | 23 | 2 | 0 | 5
busybox | 528 | 4 | 76 | 4
bzip2 | 0 | 0 | 0 | 1
dbus-test | 0 | 0 | 0 | 91 T
diffstat | 439 | 0 | 0 | 4
diffutils | 19 | 0 | 2 | 2
e2fsprogs | 0 | 0 | 0 | 78
elfutils | 172 | 24 | 7 | 21
ethtool | 2 | 0 | 0 | 0
flex | 0 | 0 | 0 | 1
gawk | 342 | 0 | 0 | 2
gdbm | 24 | 0 | 0 | 1
gdk-pixbuf | 20 | 0 | 0 | 39
gettext | 0 | 0 | 0 | 1
glib-2.0 | 242 | 3 | 0 | 344
gstreamer1.0 | 103 | 1 | 0 | 105
gzip | 0 | 0 | 0 | 1
json-glib | 14 | 0 | 0 | 1
kbd | 5 | 2 | 0 | 0
libconvert-asn1-perl | 0 | 22 | 0 | 0
liberror-perl | 0 | 15 | 0 | 1
libevent | 1 | 0 | 0 | 86
libpcre | 0 | 0 | 0 | 0
libtest-needs-perl | 0 | 2 | 0 | 0
libtimedate-perl | 3 | 2 | 0 | 0
liburi-perl | 0 | 39 | 2 | 0
libusb1 | 4 | 0 | 0 | 6
libxml-namespacesupport-perl | 0 | 1 | 0 | 0
libxml-parser-perl | 0 | 15 | 0 | 0
libxml-perl | 0 | 6 | 0 | 0
libxml-sax-base-perl | 0 | 19 | 0 | 1
libxml-sax-perl | 0 | 14 | 0 | 0
libxml-simple-perl | 0 | 14 | 0 | 0
libxml2 | 134 | 0 | 0 | 16
lttng-tools | 3397 | 626 | 1 | 348 T
lzo | 5 | 0 | 0 | 2
m4 | 0 | 0 | 0 | 1
nettle | 97 | 0 | 2 | 10
openssh | 0 | 0 | 1 | 1
openssl | 146 | 1 | 8 | 78
opkg | 0 | 0 | 0 | 11
pango | 12 | 0 | 0 | 1
parted | 0 | 0 | 0 | 0
perl | 2423 | 19 | 238 | 703
python | 302 | 5 | 41 | 330
python3 | 30277 | 2 | 1043 | 1005
quilt | 0 | 57 | 0 | 1
sed | 86 | 0 | 0 | 4
slang | 0 | 0 | 0 | 1
strace | 466 | 18 | 147 | 179
tcl | 149 | 0 | 0 | 183
util-linux | 336 | 2 | 80 | 40
valgrind | 149 | 1 | 9 | 537
zlib | 1 | 0 | 0 | 1
which can be compared against:
https://autobuilder.yocto.io/pub/non-release/20190514-14/testresults/testresult-report.txt
which means we have a fair number of ptest recipes with dependency problems:
--------------------------------------------------------------------------------------------------------------
Recipe | Passed | Failed | Skipped | Time(s)
--------------------------------------------------------------------------------------------------------------
acl | 1 | 1 | 0 | 0
apr | 0 | 0 | 0 | -
apr-util | 4 | 0 | 0 | -
attr | 0 | 0 | 0 | 0
bash | 36 | 8 | 0 | 107 T
bluez5 | 23 | 2 | 0 | 5
bzip2 | 0 | 0 | 0 | 1
dbus-test | 0 | 0 | 0 | 91 T
diffutils | 19 | 0 | 2 | 2
e2fsprogs | 0 | 0 | 0 | 78
elfutils | 172 | 24 | 7 | 21
ethtool | 2 | 0 | 0 | 0
flex | 0 | 0 | 0 | 1
gettext | 0 | 0 | 0 | 1
glib-2.0 | 242 | 3 | 0 | 344
gzip | 0 | 0 | 0 | 1
kbd | 5 | 2 | 0 | 0
libconvert-asn1-perl | 0 | 22 | 0 | 0
liberror-perl | 0 | 15 | 0 | 1
libpcre | 0 | 0 | 0 | 0
libtest-needs-perl | 0 | 2 | 0 | 0
libtimedate-perl | 3 | 2 | 0 | 0
liburi-perl | 0 | 39 | 2 | 0
libxml-namespacesupport-perl | 0 | 1 | 0 | 0
libxml-parser-perl | 0 | 15 | 0 | 0
libxml-perl | 0 | 6 | 0 | 0
libxml-sax-base-perl | 0 | 19 | 0 | 1
libxml-sax-perl | 0 | 14 | 0 | 0
libxml-simple-perl | 0 | 14 | 0 | 0
lttng-tools | 3397 | 626 | 1 | 348 T
m4 | 0 | 0 | 0 | 1
openssh | 0 | 0 | 1 | 1
opkg | 0 | 0 | 0 | 11
parted | 0 | 0 | 0 | 0
python | 302 | 5 | 41 | 330
python3 | 30277 | 2 | 1043 | 1005
quilt | 0 | 57 | 0 | 1
slang | 0 | 0 | 0 | 1
util-linux | 336 | 2 | 80 | 40
The logs from this are available here:
https://www.rpsys.net/wp/rp/testresults.json.gz
(can be extracted with resulttool log)
Anyhow, I thought I'd share this slightly crazy but IMO fairly neat
trick :)
Cheers,
Richard
More information about the Openembedded-core
mailing list