[OE-core] [PATCH 3/3] oeqa/runtime: Automatic test for ptest

[Richard Purdie]

On Fri, 2014-08-29 at 16:51 +0300, Lucian Musat wrote:
> For images without ptest the packages are automatically installed alongside ptest-runner. Log results are saved in ./results folder.
> No cleanup is done for packages after the test is finished.
> 
> Signed-off-by: Stefan Stanacar <stefanx.stanacar at intel.com>
> Signed-off-by: Lucian Musat <georgex.l.musat at intel.com>
> ---

[...]

> +    def get_ptest_packages(self):
> +        pkgs = set()
> +        image_pkgtype = self.tc.d.getVar('IMAGE_PKGTYPE', True)
> +        pkgarchs = self.tc.d.getVar('PACKAGE_ARCHS', True).replace("-","_").split()
> +        for arch in pkgarchs:
> +            folder = self.repo_server.root_dir+"/"+image_pkgtype+'/'+arch
> +            if (os.path.isdir(folder)):
> +                for fil in os.listdir(folder):
> +                    if ("ptest" in str(fil) and "ptest-runner" not in str(fil)):
> +                        #get all the packages with -ptest in the name and remove ptest from them for future comparisons 
> +                        filez = "".join(str(fil).split("-ptest"))
> +                        rootfilez = ".".join(filez.split(".")[:-2])
> +                        filex = str(fil).split("-ptest")[0]
> +                        #compare with all the packages installed on the board and get a list of potential matches
> +                        if filex in self.packagelist:
> +                            i = 0
> +                            matches = []
> +                            while i<len(self.pkglist):
> +                                if filex in self.pkglist[i]:
> +                                    matches.append(self.pkglist[i])
> +                                i +=1
> +                            for i in matches:
> +                                #sometimes package names differ from corresponding ptest package names (ex. libacl1 != acl-ptest) so we use the Source field from smart info to compare
> +                                (status, result) = self.target.run("smart info "+i,0)
> +                                rootpkg = re.search("(.*Source:.*)", result).group(1).split(":")[1][1:]
> +                                #even source package names mai differ a little so we do a fuzzy string match (ex. libz-1.2.8-r0 -> zlib-1.2.8-r0)
> +                                m = SequenceMatcher(None, rootpkg, rootfilez)
> +                                if (m.ratio > 0.9):
> +                                    filey = "".join(str(fil).split("-ptest")[0])+"-ptest"
> +                                    pkgs.add(filey)
> +        if str(pkgs) == "set([])":
> +            raise AssertionError("Cannot get ptest packages to install!")
> +        pkgs.add("ptest-runner")
> +        return pkgs

At a quick glance, I wondered if it would be possible to use oe-pkg-util
here so we have one code path for finding the list of ptest packages
available? You can see example usage at  lib/oe/package_manager.py, the
install_complementary function.

I did hear a comment that there may be some issues with that function,
if there are we need to figure out what they are and fix them.

I'm pleased to see progress on this btw!

Cheers,

Richard