TestingScript - Revision history

Sledz: cat+

2011-01-05T10:58:29Z

cat+

← Older revision		Revision as of 10:58, 5 January 2011
Line 204:		Line 204:
	Feel free to update it with improvements, better solutions etc.		Feel free to update it with improvements, better solutions etc.

	~~--- eFfeM, nov 4, 2010~~		[[Category:Quality Assurance]]

Eric: Undo revision 3275 by Yjytalamago (Talk)

2010-11-24T07:54:04Z

Undo revision 3275 by Yjytalamago (Talk)

← Older revision	Revision as of 07:54, 24 November 2010
(No difference)

EFfeM: /* Testing Script */

2010-11-05T08:39:08Z

Testing Script

← Older revision		Revision as of 08:39, 5 November 2010
Line 2:		Line 2:

	In order to run the OE testing builds a small script has been created. It is felt that this might be useful for others so I am putting and documenting it here.		In order to run the OE testing builds a small script has been created. It is felt that this might be useful for others so I am putting and documenting it here.
			The latest version of the script can also be found in the openembedded tree in contrib/testing/testscript.sh

	== The conf file ==		== The conf file ==

EFfeM: /* A full script */

2010-11-05T07:59:05Z

A full script

← Older revision		Revision as of 07:59, 5 November 2010
Line 129:		Line 129:
	</pre>		</pre>

	This script uses the external env variables MACHINE, DISTRO and TARGET_RECIPE.		This script uses the external env variables MACHINE, DISTRO and TARGET_RECIPE.<br>
			You might of course add these to the beginning of the script. E.g.:
			<pre>
			MACHINE="beagleboard"
			DISTRO="minimal"
			TARGET_RECIPE="console-image"
			</pre>

	== Getting things on Hudson autobuilder (1) ==		== Getting things on Hudson autobuilder (1) ==

Trini: /* A full script */ Mention python SimpleHTTPServer here too

2010-11-04T22:04:42Z

A full script: Mention python SimpleHTTPServer here too

← Older revision		Revision as of 22:04, 4 November 2010
Line 75:		Line 75:
	cat > ${TOPDIR}/openembedded/conf/local.conf << EOF		cat > ${TOPDIR}/openembedded/conf/local.conf << EOF
	# fix next line if you want to use your own mirror, then remove the # for the next two lines		# fix next line if you want to use your own mirror, then remove the # for the next two lines
	#SOURCE_MIRROR_URL = "~~ftp~~://~~your-mirror~~/directory"		# You can start a trivial server with 'python -m SimpleHTTPServer'
			#SOURCE_MIRROR_URL = "http://localhost:8000/directory"
	#INHERIT += "own-mirrors"		#INHERIT += "own-mirrors"

Trini: /* The conf file */ Mention python SimpleHTTPServer module for hosting a mirror

2010-11-04T22:04:19Z

The conf file: Mention python SimpleHTTPServer module for hosting a mirror

← Older revision		Revision as of 22:04, 4 November 2010
Line 7:		Line 7:
	<pre>		<pre>
	# fix next line if you want to use your own mirror, then remove the # for the next two lines		# fix next line if you want to use your own mirror, then remove the # for the next two lines
	#SOURCE_MIRROR_URL = "~~ftp~~://~~your-mirror~~/directory"		# You can start a trivial server with 'python -m SimpleHTTPServer'
			#SOURCE_MIRROR_URL = "http://localhost:8000/directory"
	#INHERIT += "own-mirrors"		#INHERIT += "own-mirrors"

Trini: /* A full script */ Update to bitbake 1.10.1

2010-11-04T22:00:19Z

A full script: Update to bitbake 1.10.1

← Older revision		Revision as of 22:00, 4 November 2010
Line 60:		Line 60:

	# test if bitbake exist; if not; fetch it and untar it		# test if bitbake exist; if not; fetch it and untar it
	if [ ! -d bitbake-1.10.0 ]		if [ ! -d bitbake-1.10.1 ]
	then		then
	(wget http://download.berlios.de/bitbake/bitbake-1.10.0.tar.gz; tar xf bitbake-1.10.0.tar.gz; rm bitbake-1.10.0.tar.gz)		(wget http://download.berlios.de/bitbake/bitbake-1.10.1.tar.gz; tar xf bitbake-1.10.1.tar.gz; rm bitbake-1.10.0.tar.gz)
	fi		fi

Line 69:		Line 69:

	# add bitbake to the path		# add bitbake to the path
	export PATH=${TOPDIR}/bitbake-1.10.0/bin:$PATH		export PATH=${TOPDIR}/bitbake-1.10.1/bin:$PATH

	# create a local.conf by using a here document		# create a local.conf by using a here document

EFfeM: added build trigger

2010-11-04T09:11:07Z

added build trigger

← Older revision		Revision as of 09:11, 4 November 2010
Line 182:		Line 182:
	as the start of the script.		as the start of the script.
	This causes hudson to build everything two levels up.		This causes hudson to build everything two levels up.

			And if you want to run things periodically create a build trigger for it.
			I use:
			<pre>
			0 18 * * 5
			</pre>
			Which starts the script every friday (5) at 18.00 local time, so hopefully there are results after the weekend.

	== Conclusion ==		== Conclusion ==

EFfeM: created

2010-11-04T09:05:37Z

created

New page

= Testing Script =

In order to run the OE testing builds a small script has been created. It is felt that this might be useful for others so I am putting and documenting it here.

== The conf file ==
The local.conf used:
<pre>
# fix next line if you want to use your own mirror, then remove the # for the next two lines
#SOURCE_MIRROR_URL = "ftp://your-mirror/directory"
#INHERIT += "own-mirrors"

DL_DIR = "${TOPDIR}/downloads"

# if you want to keep tmp dirs for different builds you might want to set TMPDIR to e.g. ${TOPDIR}/tmp_${MACHINE}_${DISTRO}
TMPDIR = "${TOPDIR}/tmp"
BBFILES = "${TOPDIR}/openembedded/recipes/*/*.bb"
ENABLE_BINARY_LOCALE_GENERATION = "0"

# Which files do we want to parse:
BBMASK = ""

# ccache always overfill $HOME....
CCACHE=""

# What kind of images do we want?
IMAGE_FSTYPES = "tar.gz "

# Make use of my SMP box
PARALLEL_MAKE="-j4"
BB_NUMBER_THREADS = "2"

OE_STRICT_CHECKSUMS = "1"

# if you are low on disk space you can remove the next #, disadvantage, nastier debugging in case of failures
#INHERIT += "rm_work"

# if you want to report build results (recommended) you need to edit OESTATS_BUILDER and add your name or nick in it, then uncomment the next 3 lines.
#INHERIT += "oestats-client"
#OESTATS_SERVER = "tinderbox.openembedded.net"
#OESTATS_BUILDER = "your nick goes here"
</pre>
Note that in order to use the above conf you need to have:
<pre>
BB_ENV_EXTRAWHITE="MACHINE DISTRO TOPDIR"
export BB_ENV_EXTRAWHITE MACHINE DISTRO TOPDIR
</pre>

== A full script ==
I have wrapped things together with a small shell script to checkout oe if needed, and to checkout the testing branch:
<pre>
# test if we have an openembedded dir, clone it if it does not exist
if [ ! -d openembedded ]
then
(git clone git://git.openembedded.org/openembedded)
(cd openembedded; git checkout -b testing-next origin/testing-next)
fi

# switch to the testing branch
(cd openembedded; git checkout testing-next)

# test if bitbake exist; if not; fetch it and untar it
if [ ! -d bitbake-1.10.0 ]
then
(wget http://download.berlios.de/bitbake/bitbake-1.10.0.tar.gz; tar xf bitbake-1.10.0.tar.gz; rm bitbake-1.10.0.tar.gz)
fi

# TOPDIR is where we are now
TOPDIR=`pwd`

# add bitbake to the path
export PATH=${TOPDIR}/bitbake-1.10.0/bin:$PATH

# create a local.conf by using a here document
cat > ${TOPDIR}/openembedded/conf/local.conf << EOF
# fix next line if you want to use your own mirror, then remove the # for the next two lines
#SOURCE_MIRROR_URL = "ftp://your-mirror/directory"
#INHERIT += "own-mirrors"

DL_DIR = "${TOPDIR}/downloads"

# if you want to keep tmp dirs for different builds you might want to set TMPDIR to e.g. ${TOPDIR}/tmp_${MACHINE}_${DISTRO}
TMPDIR = "${TOPDIR}/tmp"
BBFILES = "${TOPDIR}/openembedded/recipes/*/*.bb"
ENABLE_BINARY_LOCALE_GENERATION = "0"

# Which files do we want to parse:
BBMASK = ""

# ccache always overfill $HOME....
CCACHE=""

# What kind of images do we want?
IMAGE_FSTYPES = "tar.gz "

# Make use of my SMP box
PARALLEL_MAKE="-j4"
BB_NUMBER_THREADS = "2"

OE_STRICT_CHECKSUMS = "1"

# if you are low on disk space you can remove the next #, disadvantage, nastier debugging in case of failures
#INHERIT += "rm_work"

# if you want to report build results (recommended) you need to edit OESTATS_BUILDER and add your name or nick in it, then uncomment the next 3 lines.
#INHERIT += "oestats-client"
#OESTATS_SERVER = "tinderbox.openembedded.net"
#OESTATS_BUILDER = "your nick goes here"

EOF

# smake sure BB_ENV_EXTRAWHIT is correct, and export the needed vars
BB_ENV_EXTRAWHITE="MACHINE DISTRO TOPDIR"
export BB_ENV_EXTRAWHITE MACHINE DISTRO TOPDIR
export BBPATH=${TOPDIR}/openembedded

# pull the current branch; in case a stale lock exists remove it
(cd openembedded; rm -f .git/index.lock;git pull)

# clean tmp; I want to start with a clean build; if you changed TMPDIR in the conf file better change it here too.
rm -rf ${TOPDIR}/tmp

# add an echo about the vars so we can see what has been done in a log file
echo $MACHINE $DISTRO $TARGET_RECIPE

# and do the actual work.
bitbake ${TARGET_RECIPE}
</pre>

This script uses the external env variables MACHINE, DISTRO and TARGET_RECIPE.

== Getting things on Hudson autobuilder (1) ==

I use hudson (www.hudson.org) as autobuilder. If you also want to do that install hudson and create a small project.<br>
The first option is to create a job with one build step and use the script above as build step. In order to get the env vars working I checked "This build is parametrized" and defined 3 enum parameters named MACHINE, DISTRO and TARGET_RECIPE. Make sure the parameters are of the enum type. The value you choose when running the build will be passed as environment variable.

== Getting things on Hudson autobuilder (2) ==
A more advanced mechanism is to build a multiconfiguration with hudson.

This is done by selecting "Build multi-configuration project" when creating the job. Note that you cannot change a regular project in a multi-configuration project.
After having done that you can select your configuration matrix. This can have multiple axes. When building hudson, by default it will compile all permutations. However mechanisms exist to limit the # of permutations.

I have been a little bit lazy here. I only created one axe (MACHINE) and as I only wanted to build one distro and image for each machine I prepended the script above with a little bit of selection code:
<pre>
if [ $MACHINE = "neek" ]
then
DISTRO="minimal"
TARGET_RECIPE="console-image"
fi
if [ $MACHINE = "nslu2le" ]
then
DISTRO="slugos"
TARGET_RECIPE="slugos-image"
fi
if [ $MACHINE = "nslu2be" ]
then
DISTRO="slugos"
TARGET_RECIPE="slugos-image"
fi
if [ $MACHINE = "calamari" ]
then
DISTRO="minimal"
TARGET_RECIPE="console-image"
fi
if [ $MACHINE = "mpc8313e-rdb" ]
then
DISTRO="minimal"
TARGET_RECIPE="console-image"
fi
if [ $MACHINE = "sheevaplug" ]
then
DISTRO="minimal"
TARGET_RECIPE="console-image"
fi
</pre>
You get the idea.

One more small remark: multi-configuration will build each config in its own dir. The disadvantage is that each dir has its own copy of oe, bitbake, downloads etc.
I have avoided this by simply doing a
<pre>
cd ../..
</pre>
as the start of the script.
This causes hudson to build everything two levels up.

== Conclusion ==

This terminates my wrap up of my build script.
Feel free to update it with improvements, better solutions etc.

--- eFfeM, nov 4, 2010