Difference between revisions of "Testing"

From Openembedded.org
Jump to: navigation, search
(update testing process based on list feedback)
Line 1: Line 1:
The OpenEmbedded Testing branch is a git branch of the OE metadata with the goal of providing a provide a recent snapshot of OE that is known to be build-able for a subset of distros, machines, images, and host workstations.  The goal is to provide users with a somewhat tested version of OE that will likely build if they use one of the known working combinations.
+
The OpenEmbedded Testing branch is a git branch of the OE metadata with the goal of providing a provide a recent snapshot of OE that is known to be build-able for a subset of distros, machines, images, and host workstations.  The goal of this effort is twofold:
 +
# the testing branch represents a reasonably stable version of OE that builds for most tested combinations
 +
# for all tested combinations, we list the last known-good-build tag so that users can always start with something that will build
  
The tested combinations tested are:
+
The test combinations are:
  
 
{| border="1"
 
{| border="1"
!'''machine'''  !!'''distro'''      !!'''target'''                          !!'''workstation'''    !!'''tester'''    !!'''last successful build'''
+
!'''machine'''  !!'''distro'''      !!'''target'''                          !!'''workstation'''    !!'''tester'''    !!'''last successful build'''  !!'''Current issues'''
 
|-
 
|-
|beagleboard    ||angstrom-2008.1  ||beagleboard-linuxtag2010-demo-image  ||Ubuntu 10.04 64-bit  ||[[User:Cbrake]]      ||testing_2010-08-23
+
|beagleboard    ||angstrom-2008.1  ||beagleboard-linuxtag2010-demo-image  ||Ubuntu 10.04 64-bit  ||[[User:Cbrake]]      ||testing_2010-08-23||
 
|-
 
|-
|beagleboard    ||angstrom-2008.1  ||console-image                        ||Ubuntu 10.04 64-bit  ||[[User:Cbrake]]      ||testing_2010-08-23
+
|beagleboard    ||angstrom-2008.1  ||console-image                        ||Ubuntu 10.04 64-bit  ||[[User:Cbrake]]      ||testing_2010-08-23||
 
|-
 
|-
|bug20    ||angstrom-2008.1  ||openjdk-6  ||Debian Unstable 64-bit  ||[[Stefan]]      ||testing_2010-08-23            
+
|bug20    ||angstrom-2008.1  ||openjdk-6  ||Debian Unstable 64-bit  ||[[Stefan]]      ||testing_2010-08-23||
 
|-
 
|-
|efikamx    ||minimal  ||native-sdk-image  ||ubuntu 10.10(Alpha) 64-bit  ||[[User:khem]]      ||testing_2010-08-23
+
|efikamx    ||minimal  ||native-sdk-image  ||ubuntu 10.10(Alpha) 64-bit  ||[[User:khem]]      ||testing_2010-08-23||
 
|-
 
|-
|omap5912osk    ||minimal-uclibc  ||native-sdk-image  ||Debian Unstable 32-bit  ||[[User:khem]]      ||testing_2010-08-23
+
|omap5912osk    ||minimal-uclibc  ||native-sdk-image  ||Debian Unstable 32-bit  ||[[User:khem]]      ||testing_2010-08-23||
 
|-
 
|-
|qemuarm    ||minimal  ||native-sdk-image  ||ubuntu 10.10(Alpha) 64-bit  ||[[User:khem]]      ||testing_2010-08-23
+
|qemuarm    ||minimal  ||native-sdk-image  ||ubuntu 10.10(Alpha) 64-bit  ||[[User:khem]]      ||testing_2010-08-23||
 
|-
 
|-
|qemumips    ||minimal  ||native-sdk-image  ||ubuntu 10.10(Alpha) 64-bit  ||[[User:khem]]      ||testing_2010-08-23
+
|qemumips    ||minimal  ||native-sdk-image  ||ubuntu 10.10(Alpha) 64-bit  ||[[User:khem]]      ||testing_2010-08-23||
 
|-
 
|-
|qemuppc    ||minimal  ||native-sdk-image  ||ubuntu 10.10(Alpha) 64-bit  ||[[User:khem]]      ||fails to build sed
+
|qemuppc    ||minimal  ||native-sdk-image  ||ubuntu 10.10(Alpha) 64-bit  ||[[User:khem]]      ||None ||fails to build sed
 
|-
 
|-
|qemux86    ||minimal  ||native-sdk-image  ||ubuntu 10.10(Alpha) 64-bit  ||[[User:khem]]      ||Fails to build acpid
+
|qemux86    ||minimal  ||native-sdk-image  ||ubuntu 10.10(Alpha) 64-bit  ||[[User:khem]]      ||None ||Fails to build acpid
 
|-
 
|-
|qemumipsel    ||minimal  ||x11-image  ||Slackware 13.1 64-bit  ||[[User:grg]]      ||testing_2010-08-23
+
|qemumipsel    ||minimal  ||x11-image  ||Slackware 13.1 64-bit  ||[[User:grg]]      ||testing_2010-08-23||
 
|-
 
|-
|qemuarm    ||angstrom-2010.x  ||x11-gpe-image  ||Fedora 12 32-bit  ||[[User:gthomas]]      ||testing_2010-08-23
+
|qemuarm    ||angstrom-2010.x  ||x11-gpe-image  ||Fedora 12 32-bit  ||[[User:gthomas]]      ||testing_2010-08-23||
 
|-
 
|-
|beagleboard    ||angstrom-2010.x  ||beagleboard-linuxtag2010-demo-image  ||Fedora 12 32-bit  ||[[User:gthomas]]      ||fails to build ti-dsplink
+
|beagleboard    ||angstrom-2010.x  ||beagleboard-linuxtag2010-demo-image  ||Fedora 12 32-bit  ||[[User:gthomas]]      ||None ||fails to build ti-dsplink
 
|}
 
|}
  
 
The general process for managing the testing branch is:
 
The general process for managing the testing branch is:
# starting on Monday of every week, the dev.openembedded.org branch is branched to the '''testing-testing''' branch.
+
# Starting on Monday of every week, the dev.openembedded.org branch is branched to the '''testing-testing''' branch.
# whoever creates the testing-testing branch sends an email to the oe-devel list with a subject of "testing branch YYYY-MM-DD".  All volunteers start clean builds for the combinations they test, and report status as replies to the above email.
+
# Whoever creates the testing-testing branch sends an email to the oe-devel list with a subject of "testing branch YYYY-MM-DD".  All volunteers start a clean build and build the combinations they test, update the above chart, and report status/issues as replies to the above email.
# if no issues are reported by the following Monday, the testing-testing branch is merged to the '''testing''' branch, and the testing_YYYY-MM-DD tag is applied.  An annotated tag is used with a message that includes a text copy of the above table with variations that have been tested working for this cycle.  This way information about what was tested with each tagged version is captured in the repository.
+
# If most combinations build by following Monday, the testing-testing branch is merged to the '''testing''' branch.  Regardless, the repository is tagged with the testing_YYYY-MM-DD tag.  An annotated tag is used with a message that includes a text copy of the above table with variations that have been tested building for this cycle.  The reason we tag after testing is complete is so that we can include status in the annotated tag.  This way information about what was tested with each tagged version is captured in the repository.
# if a build fails, then issues are reported, and the testing-testing branch is moved ahead to capture fixes made in the dev branch.  Once the issue is fixed, all combinations are retested.
+
# if a build fails, then issues are reported, and we try to get issues fixed before the next weeks testing cycle.
# Not every combination is tested on every cycle, due to availability of testers, so we list the last known version in the above chart.  Testers are responsible for updating the above chart.
+
# Not every combination is tested on every cycle, due to availability of testers, so we list the last known built version in the above chart.  Testers are responsible for updating the above chart.
  
 
We strongly encourage chip and SBC/Module vendors to become involved in this effort to ensure OE works for your platforms.
 
We strongly encourage chip and SBC/Module vendors to become involved in this effort to ensure OE works for your platforms.

Revision as of 14:44, 27 August 2010

The OpenEmbedded Testing branch is a git branch of the OE metadata with the goal of providing a provide a recent snapshot of OE that is known to be build-able for a subset of distros, machines, images, and host workstations. The goal of this effort is twofold:

  1. the testing branch represents a reasonably stable version of OE that builds for most tested combinations
  2. for all tested combinations, we list the last known-good-build tag so that users can always start with something that will build

The test combinations are:

machine distro target workstation tester last successful build Current issues
beagleboard angstrom-2008.1 beagleboard-linuxtag2010-demo-image Ubuntu 10.04 64-bit User:Cbrake testing_2010-08-23
beagleboard angstrom-2008.1 console-image Ubuntu 10.04 64-bit User:Cbrake testing_2010-08-23
bug20 angstrom-2008.1 openjdk-6 Debian Unstable 64-bit Stefan testing_2010-08-23
efikamx minimal native-sdk-image ubuntu 10.10(Alpha) 64-bit User:khem testing_2010-08-23
omap5912osk minimal-uclibc native-sdk-image Debian Unstable 32-bit User:khem testing_2010-08-23
qemuarm minimal native-sdk-image ubuntu 10.10(Alpha) 64-bit User:khem testing_2010-08-23
qemumips minimal native-sdk-image ubuntu 10.10(Alpha) 64-bit User:khem testing_2010-08-23
qemuppc minimal native-sdk-image ubuntu 10.10(Alpha) 64-bit User:khem None fails to build sed
qemux86 minimal native-sdk-image ubuntu 10.10(Alpha) 64-bit User:khem None Fails to build acpid
qemumipsel minimal x11-image Slackware 13.1 64-bit User:grg testing_2010-08-23
qemuarm angstrom-2010.x x11-gpe-image Fedora 12 32-bit User:gthomas testing_2010-08-23
beagleboard angstrom-2010.x beagleboard-linuxtag2010-demo-image Fedora 12 32-bit User:gthomas None fails to build ti-dsplink

The general process for managing the testing branch is:

  1. Starting on Monday of every week, the dev.openembedded.org branch is branched to the testing-testing branch.
  2. Whoever creates the testing-testing branch sends an email to the oe-devel list with a subject of "testing branch YYYY-MM-DD". All volunteers start a clean build and build the combinations they test, update the above chart, and report status/issues as replies to the above email.
  3. If most combinations build by following Monday, the testing-testing branch is merged to the testing branch. Regardless, the repository is tagged with the testing_YYYY-MM-DD tag. An annotated tag is used with a message that includes a text copy of the above table with variations that have been tested building for this cycle. The reason we tag after testing is complete is so that we can include status in the annotated tag. This way information about what was tested with each tagged version is captured in the repository.
  4. if a build fails, then issues are reported, and we try to get issues fixed before the next weeks testing cycle.
  5. Not every combination is tested on every cycle, due to availability of testers, so we list the last known built version in the above chart. Testers are responsible for updating the above chart.

We strongly encourage chip and SBC/Module vendors to become involved in this effort to ensure OE works for your platforms.