[OE-core] Overriding SDE_FILE
Joshua Watt
jpewhacker at gmail.com
Fri Feb 28 14:18:40 UTC 2020
On 2/28/20 7:41 AM, Jacob Kroon wrote:
> On 2/28/20 2:39 PM, Jacob Kroon wrote:
>> On 2/27/20 11:35 PM, Douglas Royds via Openembedded-core wrote:
>>> On 28/02/20 11:34 am, Joshua Watt wrote:
>>>
>>>>
>>>> On 2/27/20 4:29 PM, Douglas Royds wrote:
>>>>>
>>>>> On 28/02/20 10:49 am, Joshua Watt wrote:
>>>>>
>>>>>>
>>>>>> On 2/27/20 3:22 PM, Douglas Royds wrote:
>>>>>>>
>>>>>>> On 28/02/20 5:45 am, Joshua Watt wrote:
>>>>>>>
>>>>>>>> On 2/27/20 9:01 AM, Joshua Watt wrote:
>>>>>>>>> On 2/26/20 11:46 PM, Douglas Royds wrote:
>>>>>>>>>>
>>>>>>>>>> On 26/02/20 4:53 am, Jacob Kroon wrote:
>>>>>>>>>>
>>>>>>>>>>> On 2/24/20 8:25 AM, Jacob Kroon wrote:
>>>>>>>>>>>> Hi Douglas,
>>>>>>>>>>>>
>>>>>>>>>>>> You updated a comment in reproducible_build.bbclass, commit
>>>>>>>>>>>> e7b891b76954c784f5a93bd0a1c91315673ce40d:
>>>>>>>>>>>>
>>>>>>>>>>>>> -# Once the value of SOURCE_DATE_EPOCH is determined, it
>>>>>>>>>>>>> is stored in the recipe's ${SDE_FILE}.
>>>>>>>>>>>>> +# Once the value of SOURCE_DATE_EPOCH is determined, it
>>>>>>>>>>>>> is stored in the recipe's SDE_FILE.
>>>>>>>>>>>>> +# If none of these mechanisms are suitable, replace the
>>>>>>>>>>>>> do_deploy_source_date_epoch task
>>>>>>>>>>>>> +# with recipe-specific functionality to write the
>>>>>>>>>>>>> appropriate SOURCE_DATE_EPOCH into the SDE_FILE.
>>>>>>>>>>>>> +#
>>>>>>>>>>>>
>>>>>>>>>>>> But I can't really get this to work. What did work for me
>>>>>>>>>>>> was to replace "do_create_source_date_epoch_stamp()" in my
>>>>>>>>>>>> recipe:
>>>>>>>>>>>>
>>>>>>>>>>>> do_create_source_date_epoch_stamp() {
>>>>>>>>>>>> mkdir -p ${SDE_DIR}
>>>>>>>>>>>> date -d "1981-03-03" "+%s" > ${SDE_FILE}
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> What is the intended way to achieve the thing I'm trying to
>>>>>>>>>>>> do here ?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> FYI, JPEW has a proposed patch here
>>>>>>>>>>>
>>>>>>>>>>> http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=jpew/reproducible&id=d091d2aa53ea417f70c10f5ce89151820c3db9ce
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> for allowing a recipe to just set SOURCE_DATE_EPOCH directly.
>>>>>>>>>>>
>>>>>>>>>>> But maybe that currently is at odds with SOURCE_DATE_EPOCH
>>>>>>>>>>> being in BB_HASHBASE_WHITELIST ?
>>>>>>>>>>>
>>>>>>>>>>> /Jacob
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On the surface of it, my comment appears to be just wrong: It
>>>>>>>>>> does make sense to replace
>>>>>>>>>> do_create_source_date_epoch_stamp() as you suggest.
>>>>>>>>>>
>>>>>>>>>> Joshua's proposed patch looks promising:
>>>>>>>>>>
>>>>>>>>>> * Should the new function not be called first, so that it
>>>>>>>>>> takes priority over the git, known files, and youngest file
>>>>>>>>>> functions? If someone has explicitly set SOURCE_DATE_EPOCH,
>>>>>>>>>> then they want it to take priority.
>>>>>>>>>>
>>>>>>>>> Having that be the first option makes sense. The only case in
>>>>>>>>> which that might not work, is if a recipe does something like:
>>>>>>>>>
>>>>>>>>> SOURCE_DATE_EPOCH = "${@my_awesome_sde_calculation(d)}"
>>>>>>>>>
>>>>>>>>> e.g. uses a function to get the SDE instead of setting to a
>>>>>>>>> fixed value, but that's probably going to be extremely rare.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> *
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> * As you observe, SOURCE_DATE_EPOCH would need to be removed
>>>>>>>>>> from BB_HASHBASE_WHITELIST. I'm not sure why it was in the
>>>>>>>>>> whitelist in the first place.
>>>>>>>>>>
>>>>>>>>> I'm not sure why exactly it is whitelisted; I didn't write the
>>>>>>>>> original code that whitelisted it, but I've CC'd Juro in case
>>>>>>>>> he happens to remember.
>>>>>>>>>
>>>>>>>> After a discussion with Richard, we figured out why
>>>>>>>> SOURCE_DATE_EPOCH has to be whitelisted. The value of the
>>>>>>>> variable *must* be calculable at parse time before any task is
>>>>>>>> ran, but in practice it's value is only available once the
>>>>>>>> __source_date_epoch.txt file is present, which is after
>>>>>>>> parsing. This causes the taskhash to be calculated differently
>>>>>>>> during parsing and task execution which causes taskhash
>>>>>>>> mismatch errors.
>>>>>>>>
>>>>>>>
>>>>>>> True.
>>>>>>>
>>>>>>> What to do? Would it work to use a different non-whitelisted
>>>>>>> variable in the recipe, eg. SOURCE_DATE_EPOCH_FIXED?
>>>>>>>
>>>>>>
>>>>>> Ya, that would work. You'd have to figure out how to get the
>>>>>> variable to be included in each taskhash even though it's not
>>>>>> directly referenced, but I'm sure that's possible.
>>>>>>
>>>>>>
>>>>>> Another option that's at lot more "magic" would be something like
>>>>>> this:
>>>>>> http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=jpew/reproducible&id=2b524916cf35238ff3deea34017e8a4cd73926cd
>>>>>>
>>>>>>
>>>>>> That's really weird, and I'm not sure I like it, but worth a
>>>>>> thought.
>>>>>>
>>>>> If it works, it's good. I like the fact that the user can just set
>>>>> SOURCE_DATE_EPOCH directly in their recipe.
>>>>>
>>>>> How about ...
>>>>>
>>>>> BB_HASHBASE_WHITELIST += "${@'SOURCE_DATE_EPOCH' if not
>>>>> source_date_epoch_var(d) else ''}"
>>>>>
>>>> Ya, that works. The only reason I chose "is None" was in case a
>>>> user really wanted to do
>>>>
>>>> SOURCE_DATE_EPOCH = "0"
>>>>
>>> True, best to allow for that.
>>>
>>
>> Joshua, I just tried the two top patches in your reproducible branch;
>> I can see that SOURCE_DATE_EPOCH is not in BB_HASHBASE_WHITELIST,
>> however, changing its value in my recipe doesn't trigger rebuilding
>> for some reason..
>
> In case it makes a difference, I only reference ${SOURCE_DATE_EPOCH}
> in my do_install() task.
I don't think it matters. My understanding is that an export variable
(like SOURCE_DATE_EPOCH) is included in the taskhash of all tasks unless
it is specifically whitelisted. It's possible there is some bug I
haven't throughly tested it yet.... or it just can't work that way at
all for some reason. You might be able to dump the task signature for
your do_install task to see if SOURCE_DATE_EPOCH is being included.
More information about the Openembedded-core
mailing list