[bitbake-devel] python function used in variable expansion called multiple times per recipe

[Daniel Lazzari]

>> I think immediate expansion is what I’ll need to use, thanks for the tip.
>> That at least brings the invocation down to once per run. Unfortunately, the
>> variable doesn’t seem to get filled in when I change to immediate
>expansion.
>>
>> def testFunc(bb, d):
>>                 print( bb.data.getVar("PN", d, True) )
>>                 return "e1"
>>
>> EXTERNPV := "${@testFunc(bb, d)}"
>>
>> Now only prints the package name once, but using –e, I don’t see the
>> variable EXTERNPV at all.
>
>Don't know what to tell you. It does work, so it's something wrong on
>your end. Either it's returning an empty string, which will result in
>-e only showing the commented out definition, or it's raising an
>exception, or something else in your setup. Aside: you don't need to
>pass 'bb' in. That namespace is always accessible, even from def'd
>python functions.
>--
>Christopher Larson

Just thought I'd provide an update real quick on this. I discovered, to my dismay, that even doing an immediate expansion still ran the code before every task for that recipe. I switched to using an event handler, but the RecipeParsed event occurs before every task for that recipe. Eventually I discovered that the variable BB_WORKERCONTEXT is set for RecipeParsed events when they are occurring before a task, but not for the initial parse. Also, I discovered that setting a variable in the event handler would not necessarily keep it set through building the recipe. Thus I devised the below code to run the time intensive code during initial parse and cache the value in a file so it can be picked up again in later events. I'm leaving this here in case it helps someone else.

addhandler svn_extern_eventhandler
python svn_extern_eventhandler() {
	from bb.event import getName
	from bb import data
	
	if getName(e) == "RecipeParsed":
		extern_cache_dir = os.path.join( data.getVar('TMPDIR', e.data, True),
		                             'stamps',
		                             data.getVar('MULTIMACH_TARGET_SYS', e.data, True) )
		extern_cache = os.path.join( extern_cache_dir,
		                             data.getVar('PN', e.data, True) + '.extern_cache' )
		#If BB_WORKERCONTEXT is "1", this is NOT the initial recipe parse
		if data.getVar('BB_WORKERCONTEXT', e.data, True) == "1":
			#Pull cached copy
			try:
				cache_file = open(extern_cache, 'r')
				externpv = cache_file.read()
				cache_file.close()
			except IOError:
				#Failed to get cached copy, do it the hard way
				externpv = getExternRevs(e.data)
		else:
			externpv = getExternRevs(e.data)
			try:
				os.makedirs(extern_cache_dir)
			except OSError:
				pass
			#Generate the cache for later use
			cache_file = open(extern_cache, 'w')
			cache_file.write(externpv)
			cache_file.close()

		data.setVar('EXTERNPV', externpv, e.data)
}