Openembedded.org - User contributions [en]

OEDAM 2018

2018-03-11T17:47:07Z

Mhatle: /* Planning to Attend */

== Location and Time ==

Sunday, 11 March 2018 (before ELC [March 12-14]) 
Board meeting to happen before the event
Doors open at 10am
Developer's meeting to start at 10:30
Expectation is to finish by 1800

For those who want to commute with Armin, Please meet in the Lobby in the Hilton
where ELC is hosted at 9:30 am. I can only take 15.

Lunch and coffee are provided by the Yocto Project.

Location: Mentor Graphics Headquarters
8105 Boeckman Road, Wilsonville, OR
Evergreen Building, E2411 Training room

Lunch location is McMenamins Old Church & Pub, 30340 SW Boones Ferry Rd, Wilsonville.
Lunch will be at about 12:45.

Carpooling is encouraged, see below

== Logistics ==

Yes, we know it is Sunday. Best we could do this year.

'''Daylight Savings Time starts tomorrow''', so be sure to roll your clocks FORWARD by an hour - so 10am is actually logical 9am.

The planned start is 10:30. I am planning to arrive at 10am with coffee. It is roughly 1/2 hour drive from downtown, so please plan accordingly.

Carpooling is highly encouraged. Armin has a van rented that can take up to 15. Jefro is already picking up 2 and has room for 1-2 more. If anyone else has a vehicle and can take people, please pipe up on the email thread or send email to jefro@jefro.net. Note that an uber or lyft will be a $35 trip from downtown.

The day will need to end by 6pm. Jefro will be going back up to the Hilton at top speed to set up the Yocto booth.

== Remote conference link ==

If you are not able to be there in person, you can join this conference remotely over zoom.us:

Join URL: https://zoom.us/j/347593211

== Planning to Attend ==

Carpool note: Please add a {#} after your name if you plan to drive and can take # of people in your car other than yourself

* Jeff Osier-Mixon "Jefro" {4}
* Philip Balister (Crofton)
* Denys Dmytriyenko (denix)
* Sean Hudson (darknighte)
* Manjukumar Matha (manju)
* Joshua Watt (JPEWhacker)
* Zachary Booth
* Jim Lodes
* Armin Kuster (Armpit){15}
* Scott Murray (smurray)
* Tim Orling (moto-timo)
* Alejandro Hernandez (aehs29)
* Christopher Clark
* Behan Webster (behanw)
* Marek Vasut (Marex)
* Taras Kondratiuk
* Bill Mills
* Ricardo Salveti (rsalveti)
* Khem Raj (khem)
* Tom King (ka6sox)
* Rich Persaud
* Michael Halstead (halstead)
* Stephano Cetola (stephano) {1 + truck bed }
* Richard Purdie (RP)
* Marco Cavallini (mckoan)
* Mark Hatle (fray) {1}

== Agenda Items ==

=== OE Board ===
E.v status

=== OE Developers Meeting ===
# State of the Union
## Discuss meta-oe release process
## World build monitoring and automatic notifications
# Topics from GA
## Introduction
### Quick agenda overview (Sean)
### Refer to https://www.openembedded.org/wiki/2018_02_General_Meeting
## Call for Topics at OEDAM in 2018-03 in Portland
####Refer to https://www.openembedded.org/wiki/OEDAM_2018
###Layer Quality (hatle)
###Splitting meta-oe if not resolved prior to meeting (armpit)
####Patch submissions (all OE components)
###REQUEST: Identify areas where we need maintainers to help
####ptest additions to meta-oe (timo)
###REQUEST: Could someone run through a simple example to help others learn to do this correctly
####OEQA testing
###Clear need for a quickstart guide (one-page recipe to get folks started quickly)
###Running tests (individual, groups, all)
###Connecting to a Machine ( h/w)(armpit)
###FOSDEM feedback (paulbarker)
###Auto Updater ( AUH)(armpit)???
# Logistics for OEDAM in 2018-03 in Portland
## Board will facilitate putting folks needing rides and folks with spots in their cars together. (see OEDAM wiki)
## Moped balance in question… Carpooling recommended...

== Minutes ==

SHARED MINUTES AT
https://docs.google.com/document/d/1h_otwcH-6ej4URB5vER-7FpaPthrji3TpJWf5jAKRIk/edit?usp=sharing

=== Prior minutes ===

Minutes from Prague, Oct 2017:
https://docs.google.com/document/d/1R_pV_PumhZ9a8_aRDTuW7wBjqNwIDuau7FGr6sYYRu0/edit?usp=sharing

Minutes from Portland, Feb 2017:
https://docs.google.com/document/d/1ECCfBwdLUaW3I23RATWCU3kGnSh04Kv0DkH1_KfiLVA/edit?usp=sharing

OEDEM 2017

2017-09-15T16:24:20Z

Mhatle:

== Location and Time ==

Sunday, 22 October 2017 
Start at 0900

(before ELC-E [Oct 23-25])

Yes, we know it is Sunday. Best we could do this year.

NOTE: venue has changed, we are now at:

JURY'S INN Prague
Sokolovská 204/11, 186 00
Praha 8-Florenc, Czechia

This hotel is across the street from the Hilton (ELCE venue). If you have any difficulty please let [mailto:jefro@jefro.net jefro] know.

== Attendees ==

* Jeff Osier-Mixon "Jefro"
* Behan Webster (behanw)
* Philip Balister (Crofton)
* Martin Jansa (JaMa)
* Scott Murray (smurray)
* Peter Kjellerstedt (Saur)
* Armin Kuster (Armpit)
* Marco Cavallini (mckoan)
* Paul Barker (paulbarker)
* Myunghun Chun
* Changhyeok Bae (chbae)
* Adrian Ratiu (aratiu)
* Josef Holzmayr (LetoThe2nd)
* Jan Leupold
* Anders Darander
* Sean Hudson (darknighte)
* Tim Orling (moto-timo)
* Ricardo Ribalda (ribalda)
* Nicolas Dechesne (ndec)
* Richard Purdie (RP)
* Andrei Gherzan (agherzan)
* Denys Dmytriyenko (denix)
* Anton Gerasimov
* Rich Persaud
* Koen Kooi
* Marek Vasut (marex)
* Ruslan Bilovol
* Grygorii Tertychnyi
* Bruce Ashfield (zeddii)
* Mark Hatle (fray)

NOTE: we will make every effort to provide a web/phone link for remote attendees

== Agenda Items ==

# Stack Overflow report (From someone who regularly watches stackoverflow)
# Demos for FOSDEM and other events. Good chance to (discretely) show your companies work. (Philip)
# Long-term releases support (more than one year)
# Automated CVE checking, issues with cve-check-tool, alternative tools

== Minutes ==

=== OEDEM: ===

TSC

2017-05-03T13:23:20Z

Mhatle: /* Members */

=OE Technical Steering Committee=

The TSC consists of 5 people who are elected by, and ultimately be answerable to, the members of the OpenEmbedded organisation.

Here is the [[TSCCharter | working copy]] of the new TSC charter.

The TSC meets on an irregular basis, but is happy to hold meetings on request if there are issues to discuss.

The TSC discusses technical issues for the direction of OpenEmbedded. Note that the TSC is NOT chartered with actually doing all of the work discussed, but they do make every effort to locate resources to do that work.

= Members =

In order by election:

* Martin Jansa (JaMa) - re-elected May 2017
* Richard Purdie (RP) - re-elected May 2017
* Khem Raj (khem) - re-elected May 2017
* Paul Eggleton (bluelightning) - re-elected May 2017
* Mark Hatle (fray) - re-elected May 2017

Each member serves a two-year term. During election time, one member may stand for election every two months.

= Responsibilities =

On a day-to-day basis the TSC is responsible for making
tactical policy decisions, resolving disputes between contributors,
administering access control to the git tree, and generally promoting
good development practice.

= Decision Making =

Decisions are made where necessary by majority vote. Once a decision is made by
the technical steering committee, the decision should be respected as a democratic decision.

= TSC issue escalation procedure =

The best way to agendize an issue for the TSC is to send email to one of the TSC members or to [mailto:jefro@jefro.net Jefro], the curator. The issue will be added to the agenda for the next meeting. If the issue is particularly urgent, a specific TSC meeting may be called.

= TSC mailing list archive =

http://lists.openembedded.org/pipermail/tsc/

= Meeting Minutes =

== 2013 Minutes ==

* [http://lists.openembedded.org/pipermail/tsc/2013-February/000362.html 29 January 2013]
* [http://lists.openembedded.org/pipermail/tsc/2013-February/000363.html 12 February 2013]
* [http://lists.openembedded.org/pipermail/tsc/2013-March/000365.html 26 February 2013]
* [http://lists.openembedded.org/pipermail/tsc/2013-March/000366.html 19 March 2013]
* [http://lists.openembedded.org/pipermail/tsc/2013-April/000367.html 8 April 2013]
* [http://lists.openembedded.org/pipermail/tsc/2013-May/000368.html 23 April 2013]
* [http://lists.openembedded.org/pipermail/tsc/2013-May/00037.html 7 May 2013]
* [http://lists.openembedded.org/pipermail/tsc/2013-June/000372.html 21 May 2013]
* [http://lists.openembedded.org/pipermail/tsc/2013-June/000373.html 4 June 2013]
* [http://lists.openembedded.org/pipermail/tsc/2013-June/000381.html 18 June 2013]
* [http://lists.openembedded.org/pipermail/tsc/2013-August/000393.html 2 July 2013]
* 16 July 2013
* 30 July 2013 (public meeting)
* 13 August 2013 (cancelled)
* 27 August 2013

== 2012 Minutes ==

* [http://lists.linuxtogo.org/pipermail/tsc/2012-January/000330.html 3-Jan-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-January/000331.html 17-Jan-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-February/000332.html 30-Jan-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-March/000334.html 15-Feb-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-March/000335.html 28-Feb-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-March/000336.html 13-Mar-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-April/000337.html 27-Mar-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-April/000340.html 10-Apr-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-May/000342.html 24-Apr-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-May/000341.html 08-May-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-June/000343.html 22-May-2012]
* 5-Jun-2012 (no meeting)
* [http://lists.linuxtogo.org/pipermail/tsc/2012-June/000344.html 19-Jun-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-August/000346.html 3-Jul-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-August/000347.html 17-Jul-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-August/000348.html 31-Jul-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-August/000353.html 14-Aug-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-September/000354.html 28-Aug-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-September/000355.html 11-Sep-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-October/000356.html 25-Sep-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-October/000357.html 9-Oct-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-November/000358.html 23-Oct-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2012-December/000359.html 6-Nov-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2013-January/000360.html 20-Nov-2012]
* [http://lists.linuxtogo.org/pipermail/tsc/2013-January/000361.html 4-Dec-2012]

== 2011 Minutes ==

* [http://lists.linuxtogo.org/pipermail/openembedded-devel/2011-February/029974.html 14-Feb-2011]
* [http://lists.linuxtogo.org/pipermail/openembedded-devel/2011-February/030355.html 21-Feb-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-March/000113.html 28-Feb-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-March/000192.html 10-Mar-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-March/000193.html 17-Mar-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-March/000208.html 25-Mar-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-April/000224.html 31-Mar-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-April/000232.html 10-Apr-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-May/000241.html 28-Apr-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-May/000240.html 05-May-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-June/000245.html 12-May-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-June/000246.html 19-May-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-June/000249.html 26-May-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-June/000250.html 02-Jun-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-June/000251.html 09-Jun-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-June/000258.html 16-Jun-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-June/000266.html 23-Jun-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-July/000270.html 30-Jun-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-July/000272.html 14-Jul-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-August/000283.html 21-Jul-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-August/000285.html 04-Aug-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-September/000298.html 11-Aug-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-October/000301.html 01-Sep-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-September/000297.html 16-Sep-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-September/000299.html 29-Sep-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-October/000302.html 06-Oct-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-October/000303.html 20-Oct-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-November/000304.html 03-Nov-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-November/000322.html 17-Nov-2011]
* [http://lists.linuxtogo.org/pipermail/tsc/2011-November/000323.html 22-Nov-2011]
* [http://lists.linuxtogo.org/pipermail/openembedded-core/2011-December/014563.html 13-Dec-2011]

== 2010 Minutes ==

* [http://lists.linuxtogo.org/pipermail/openembedded-devel/2010-June/020655.html June 2010]
* [http://lists.linuxtogo.org/pipermail/openembedded-devel/2010-May/020047.html May 2010]
* [http://lists.linuxtogo.org/pipermail/openembedded-devel/2010-March/017876.html March 2010]
* [http://lists.linuxtogo.org/pipermail/openembedded-devel/2010-February/017094.html February 2010]
* [http://lists.linuxtogo.org/pipermail/openembedded-devel/2010-January/015931.html January 2010]

[[Category:Policy]]

2017 General Meeting

2017-04-19T23:49:57Z

Mhatle: /* Planning to attend */

== Purpose ==
The new bylaws of the organization require '''at least''' one general meeting per year. This will fulfill that requirement. Also, we plan to discuss topics of interest to the community and the project.

== Date and Time ==
'''!!!Tentative!!!''' 2017-05-03 @ 8am US-CDT(UTC-06:00)/3pm CET(UTC+01:00)

Here's a link to the time:
https://www.timeanddate.com/worldclock/fixedtime.html?msg=OpenEmbedded+2017+General+Meeting&iso=20170503T08&p1=24&ah=1

Countdown link:
https://www.timeanddate.com/countdown/generic?p0=24&iso=20170503T08&msg=OpenEmbedded%202017%20General%20Meeting

== Conference Information ==

Meeting will be held as a teleconference with a IRC channel to allow for background discussion and trouble shooting of audio.

IRC: #oe-meeting on freenode

'''Dial-in Conference # 8964521'''

{| class="wikitable"
!colspan="2"|Dial-in Information
|-
| United States (USA)
|917-210-2607
|-
|DENMARK (DNK)
|36910500
|-
|FINLAND (FIN)
|923194205
|-
|GERMANY (DEU)
|69710448206
|-
|IRELAND (IRL)
|14360203
|-
|MEXICO
|52-5547772297
|-
|NETHERLANDS (NLD)
|207940366
|-
|PAKISTAN
|4238108701
|-
|ROMANIA
|215291724
|-
|RUSSIA
|7 4959952645
|-
|SPAIN (ESP)
|917911851
|-
|SWEDEN (SWE)
|114501530
|-
|UNITED KINGDOM (GBR)
|2070840301
|-
|}

== Tentative Agenda ==
# Meeting Schedule
## The board would like to propose a quarterly meeting schedule with alternating online and face-to-face meetings.
# Membership
## Grandfathered all existing members in good standing
## New members
# TSC
## Grandfathered from old organization
# Online voting
# Elections
# Future plans/purpose for the organization
# Technical topics

== Planning to attend ==
* Sean Hudson (irc:darknighte)
* Denys Dmytriyenko (denix)
* Bruce Ashfield (zeddii*)
* Armin Kuster (armpit)
* Mark Hatle (fray)

OEDAM 2017

2017-02-20T06:00:23Z

Mhatle: /* Attendees */

== Location and Time ==

Monday February 20 2017

Location: 
Mentor Graphics 
8005 Boeckman Rd 
Wilsonville, OR 97070 
Phone: (800) 547-3000 
9am-5pm 

(Before ELC [Feb 21-23] and Yocto Project Developer Day [Feb 24])

Current transportation plan is meet in the Hilton Lobby at 0800. From there we will use Lyft/Uber/other in as efficient way as possible.

(fray) I've got room for 2, maybe 3 people in my car.

== Attendees ==

* Armin Kuster (armpit)
* Philip Balister (Crofton)
* Trevor Woerner (tlwoerner)
* Denys Dmytriyenko (denix)
* Stephano Cetola
* Matthew McClintock
* Tim Orling (moto-timo)
* James Perkins (jamesp)
* Patrick Ohly (pohly)
* Richard Purdie (RP)
* Mark Hatle (fray)
* Jan-Simon Möller (dl9pf)
* Derek Straka
* Scott Murray (scottm)
* Ken Sharp
* Behan Webster (behanw)
* Brian Avery (bavery)
* Saul Wold (sgw)
* David Reyna

== Agenda Items ==

* Review items from last meeting in Berlin (Please add remarks)
* Proposal (Patrick): "production" vs. "development" builds and features
* Proposal (Patrick): stateless distro
* Proposal (Patrick): GPLv3 handling - remove recipes for obsolete components, replace with adaptive recipe and image configurations (like disabling readline support)
* Proposal (Patrick): remove openembedded-devel@lists.openembedded.org Reply-To?
* Discuss recruiting new layer maintainers for meta-oe and what a maintainer does.

=== Berlin AR's ===

==== LTS ====

Yocto-Compatibility requires pushing patches upstream, but Yocto doesn't handle QA for old releases
A lot of work for the software vendors
rp: have new/separate LTS tree for older releases
crofton: how to coordinate between ppl who want this, collect interest in the wiki
add maintainer information to the layer repos?
rp: write a proposal?
→ enough interest to set this up

action: Armin will start conversation on the Architecture list
'''Not completed
'''

==== Windriver ‘setup’ Demo ====
Conclusion: We want it in OE instead of Yocto (unanimous)
Mark: Once able to contribute, talk to halstead for new repo.
'''Repo published. OE has not taken over'''

Notes: Layer Index was recently updated and this was required before progress can be made on promoting this tool.

I would like to talk about the next steps at the OEDAM, now that everything is public.

==== Devtool and other stuff (10:04) ====
Sysroot-Contamination
rp: solution “sysroot per recipe”

==== Suggestion for improvement of how to handle site and user configuration. ====
Conclusion: Saur will implement the BBPATH_EXTRA and discuss it on the list.

==== BSPs and layer name recommendations ====
RP: We can do this by … For Compatible v2 we want to raise the bar. Sanity check for things that keep breaking.
Conclusion: Jan-Simon will start a script check the basic stuff, RP should add the checksum checks.

==== OTA ====
RP: This discussion needs to continue beyond OEDEM

==== Layer Quality ====
Conclusion: We need to get the word out and get suggestions.

==== Make perl and python distro features? [Saur] ====
Conclusion: No change.

==== Support for meson build system? [Saur] ====
Conclusion: Patches welcome, talk to Ross.

==== The use of ${COREBASE}/LICENSE in LIC_FILES_CHKSUM. [Saur] ====
Conclusion: RP: This needs to go to the ML

==== Discuss merging duplicate classes and recipes (metadata) ====
Conclusion: Duplicate recipes and machines need to be brought up with the maintainers.

Conclusion: Mark, RP, Chris and others should look to create a proposal about resolving the confusion over layer order/priority/etc.

==== Changes to deploy_image ====
Conclusion: Document

==== Discussion on mesa and splitting libgbm ====
Conclusion: RP: Topic should be discussed with Ross on mailing list.

==== Discussion of recipe maintainers for oe-core and beyond ====
RP: This list is a Yocto initiative. They try to get more involvement from the members. There is an incentive to get more ppl involved (recipe reporting system):

== Minutes ==

OEDAM 2017

2017-02-20T05:59:50Z

Mhatle: /* Location and Time */

== Location and Time ==

Monday February 20 2017

Location: 
Mentor Graphics 
8005 Boeckman Rd 
Wilsonville, OR 97070 
Phone: (800) 547-3000 
9am-5pm 

(Before ELC [Feb 21-23] and Yocto Project Developer Day [Feb 24])

Current transportation plan is meet in the Hilton Lobby at 0800. From there we will use Lyft/Uber/other in as efficient way as possible.

(fray) I've got room for 2, maybe 3 people in my car.

== Attendees ==

* Armin Kuster (armpit)
* Philip Balister (Crofton)
* Trevor Woerner (tlwoerner)
* Denys Dmytriyenko (denix)
* Stephano Cetola
* Matthew McClintock
* Tim Orling (moto-timo)
* James Perkins (jamesp)
* Patrick Ohly (pohly)
* Richard Purdie (RP)
* Mark Hatle (fray)
* Jan-Simon Möller (dl9pf)
* Derek Straka
* Scott Murray (scottm)
* Ken Sharp
* Behan Webster (behanw)
* Brian Avery (bavery)
* Saul Wold (sgw)

== Agenda Items ==

* Review items from last meeting in Berlin (Please add remarks)
* Proposal (Patrick): "production" vs. "development" builds and features
* Proposal (Patrick): stateless distro
* Proposal (Patrick): GPLv3 handling - remove recipes for obsolete components, replace with adaptive recipe and image configurations (like disabling readline support)
* Proposal (Patrick): remove openembedded-devel@lists.openembedded.org Reply-To?
* Discuss recruiting new layer maintainers for meta-oe and what a maintainer does.

=== Berlin AR's ===

==== LTS ====

Yocto-Compatibility requires pushing patches upstream, but Yocto doesn't handle QA for old releases
A lot of work for the software vendors
rp: have new/separate LTS tree for older releases
crofton: how to coordinate between ppl who want this, collect interest in the wiki
add maintainer information to the layer repos?
rp: write a proposal?
→ enough interest to set this up

action: Armin will start conversation on the Architecture list
'''Not completed
'''

==== Windriver ‘setup’ Demo ====
Conclusion: We want it in OE instead of Yocto (unanimous)
Mark: Once able to contribute, talk to halstead for new repo.
'''Repo published. OE has not taken over'''

Notes: Layer Index was recently updated and this was required before progress can be made on promoting this tool.

I would like to talk about the next steps at the OEDAM, now that everything is public.

==== Devtool and other stuff (10:04) ====
Sysroot-Contamination
rp: solution “sysroot per recipe”

==== Suggestion for improvement of how to handle site and user configuration. ====
Conclusion: Saur will implement the BBPATH_EXTRA and discuss it on the list.

==== BSPs and layer name recommendations ====
RP: We can do this by … For Compatible v2 we want to raise the bar. Sanity check for things that keep breaking.
Conclusion: Jan-Simon will start a script check the basic stuff, RP should add the checksum checks.

==== OTA ====
RP: This discussion needs to continue beyond OEDEM

==== Layer Quality ====
Conclusion: We need to get the word out and get suggestions.

==== Make perl and python distro features? [Saur] ====
Conclusion: No change.

==== Support for meson build system? [Saur] ====
Conclusion: Patches welcome, talk to Ross.

==== The use of ${COREBASE}/LICENSE in LIC_FILES_CHKSUM. [Saur] ====
Conclusion: RP: This needs to go to the ML

==== Discuss merging duplicate classes and recipes (metadata) ====
Conclusion: Duplicate recipes and machines need to be brought up with the maintainers.

Conclusion: Mark, RP, Chris and others should look to create a proposal about resolving the confusion over layer order/priority/etc.

==== Changes to deploy_image ====
Conclusion: Document

==== Discussion on mesa and splitting libgbm ====
Conclusion: RP: Topic should be discussed with Ross on mailing list.

==== Discussion of recipe maintainers for oe-core and beyond ====
RP: This list is a Yocto initiative. They try to get more involvement from the members. There is an incentive to get more ppl involved (recipe reporting system):

== Minutes ==

OEDAM 2017

2017-01-19T19:33:47Z

Mhatle: /* Windriver ‘setup’ Demo */

== Location and Time ==

Monday February 20 2017

Location TBA

(Before ELC [Feb 21-23] and Yocto Project Developer Day [Feb 24])

== Attendees ==

* Armin Kuster (armpit)
* Philip Balister (Crofton)
* Trevor Woerner (tlwoerner)
* Denys Dmytriyenko (denix)
* Stephano Cetola
* Matthew McClintock
* Tim Orling (moto-timo)
* James Perkins (jamesp)
* Patrick Ohly (pohly)
* Richard Purdie (RP)
* Mark Hatle (fray)

== Agenda Items ==

* Review items from last meeting in Berlin (Please add remarks)

=== Berlin AR's ===

==== LTS ====

Yocto-Compatibility requires pushing patches upstream, but Yocto doesn't handle QA for old releases
A lot of work for the software vendors
rp: have new/separate LTS tree for older releases
crofton: how to coordinate between ppl who want this, collect interest in the wiki
add maintainer information to the layer repos?
rp: write a proposal?
→ enough interest to set this up

action: Armin will start conversation on the Architecture list
'''Not completed
'''

==== Windriver ‘setup’ Demo ====
Conclusion: We want it in OE instead of Yocto (unanimous)
Mark: Once able to contribute, talk to halstead for new repo.
'''Repo published. OE has not taken over'''

Notes: Layer Index was recently updated and this was required before progress can be made on promoting this tool.

I would like to talk about the next steps at the OEDAM, now that everything is public.

==== Devtool and other stuff (10:04) ====
Sysroot-Contamination
rp: solution “sysroot per recipe”

==== Suggestion for improvement of how to handle site and user configuration. ====
Conclusion: Saur will implement the BBPATH_EXTRA and discuss it on the list.

==== BSPs and layer name recommendations ====
RP: We can do this by … For Compatible v2 we want to raise the bar. Sanity check for things that keep breaking.
Conclusion: Jan-Simon will start a script check the basic stuff, RP should add the checksum checks.

==== OTA ====
RP: This discussion needs to continue beyond OEDEM

==== Layer Quality ====
Conclusion: We need to get the word out and get suggestions.

==== Make perl and python distro features? [Saur] ====
Conclusion: No change.

==== Support for meson build system? [Saur] ====
Conclusion: Patches welcome, talk to Ross.

==== The use of ${COREBASE}/LICENSE in LIC_FILES_CHKSUM. [Saur] ====
Conclusion: RP: This needs to go to the ML

==== Discuss merging duplicate classes and recipes (metadata) ====
Conclusion: Duplicate recipes and machines need to be brought up with the maintainers.

Conclusion: Mark, RP, Chris and others should look to create a proposal about resolving the confusion over layer order/priority/etc.

==== Changes to deploy_image ====
Conclusion: Document

==== Discussion on mesa and splitting libgbm ====
Conclusion: RP: Topic should be discussed with Ross on mailing list.

==== Discussion of recipe maintainers for oe-core and beyond ====
RP: This list is a Yocto initiative. They try to get more involvement from the members. There is an incentive to get more ppl involved (recipe reporting system):

== Minutes ==

OEDAM 2017

2017-01-19T19:32:54Z

Mhatle: /* Attendees */

== Location and Time ==

Monday February 20 2017

Location TBA

(Before ELC [Feb 21-23] and Yocto Project Developer Day [Feb 24])

== Attendees ==

* Armin Kuster (armpit)
* Philip Balister (Crofton)
* Trevor Woerner (tlwoerner)
* Denys Dmytriyenko (denix)
* Stephano Cetola
* Matthew McClintock
* Tim Orling (moto-timo)
* James Perkins (jamesp)
* Patrick Ohly (pohly)
* Richard Purdie (RP)
* Mark Hatle (fray)

== Agenda Items ==

* Review items from last meeting in Berlin (Please add remarks)

=== Berlin AR's ===

==== LTS ====

Yocto-Compatibility requires pushing patches upstream, but Yocto doesn't handle QA for old releases
A lot of work for the software vendors
rp: have new/separate LTS tree for older releases
crofton: how to coordinate between ppl who want this, collect interest in the wiki
add maintainer information to the layer repos?
rp: write a proposal?
→ enough interest to set this up

action: Armin will start conversation on the Architecture list
'''Not completed
'''

==== Windriver ‘setup’ Demo ====
Conclusion: We want it in OE instead of Yocto (unanimous)
Mark: Once able to contribute, talk to halstead for new repo.
'''Repo published. OE has not taken over'''

==== Devtool and other stuff (10:04) ====
Sysroot-Contamination
rp: solution “sysroot per recipe”

==== Suggestion for improvement of how to handle site and user configuration. ====
Conclusion: Saur will implement the BBPATH_EXTRA and discuss it on the list.

==== BSPs and layer name recommendations ====
RP: We can do this by … For Compatible v2 we want to raise the bar. Sanity check for things that keep breaking.
Conclusion: Jan-Simon will start a script check the basic stuff, RP should add the checksum checks.

==== OTA ====
RP: This discussion needs to continue beyond OEDEM

==== Layer Quality ====
Conclusion: We need to get the word out and get suggestions.

==== Make perl and python distro features? [Saur] ====
Conclusion: No change.

==== Support for meson build system? [Saur] ====
Conclusion: Patches welcome, talk to Ross.

==== The use of ${COREBASE}/LICENSE in LIC_FILES_CHKSUM. [Saur] ====
Conclusion: RP: This needs to go to the ML

==== Discuss merging duplicate classes and recipes (metadata) ====
Conclusion: Duplicate recipes and machines need to be brought up with the maintainers.

Conclusion: Mark, RP, Chris and others should look to create a proposal about resolving the confusion over layer order/priority/etc.

==== Changes to deploy_image ====
Conclusion: Document

==== Discussion on mesa and splitting libgbm ====
Conclusion: RP: Topic should be discussed with Ross on mailing list.

==== Discussion of recipe maintainers for oe-core and beyond ====
RP: This list is a Yocto initiative. They try to get more involvement from the members. There is an incentive to get more ppl involved (recipe reporting system):

== Minutes ==

Commit Patch Message Guidelines

2016-11-08T15:49:08Z

Mhatle:

This set of guidelines is intended to help both the developer and reviewers of
changes determine reasonable patch headers and change commit messages. Often
when working with the code, you forget that not everyone is as familiar with the
problem and/or fix as you are. Often the next person in the code doesn't
understand what or why something is done so they quickly look at patch header
and commit messages. Unless these messages are clear it will be difficult to
understand the relevance of a given change and how future changes may impact
previous decisions.

This policy refers both to patches that are being applied by recipes as well as
commit messages that are part of the source control system, usually git. A patch
file needs a header in order to describe the specific changes that are made
within that patch, while a commit message describes one or more changes to the
overall project or system. Both the patch headers and commit messages require
the same attention and basic details, however the purposes of the messages are
slightly different. A commit message documents all of the changes made as part
of a commit, while a patch header documents items specific to a single patch. In
many cases the patch header can also be used as the commit message.

This policy does not cover the testing of the changes, the technical criteria
for accepting a patch, nor the style requirements for the technical changes. See
the [http://www.openembedded.org/wiki/Styleguide Styleguide] for more information on style requirements.

By following these guidelines we will have a better record of the problems and
solutions made over the course of development. It will also help establish a
clear provenance of all of the code and changes.

Draft version, changes from last approved version:
* Revised General Information section to correct broken link
* Clarify the ''Developer's Certificate of Origin''

== General Information ==

While specific to the Linux kernel development, the following could also be
considered a general guide for any Open Source development:

https://www.linux.com/publications/how-participate-linux-community

Many of the guidelines in this document are related to the items referenced in
the link above.

Pay particular attention to section 5.3 that talks about patch preparation.
The key thing to remember is to break up your changes into logical sections.
Otherwise you run the risk of not being able to even explain the purpose of a
change in the patch headers!

In addition section 5.4 explains the purpose of the Signed-off-by: tag line. The
signed-off-by: tag line is the ''Developer's Certificate of Origin''. The full text
of which can be found in the Linux kernel's Documentation/SubmittingPatches.

Pay particular attention to section 11 of
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/SubmittingPatches

Due to the importance of the ''Developer's Certificate of Origin'', part of section 11
is copied below:

The sign-off is a simple line at the end of the explanation for the
patch, which certifies that you wrote it or otherwise have the right to
pass it on as an open-source patch. The rules are pretty simple: if you
can certify the below:

Developer's Certificate of Origin 1.1
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

then you just add a line saying::

Signed-off-by: Random J Developer <random@developer.example.org>

using your real name (sorry, no pseudonyms or anonymous contributions.)

== Patch Headers and Commit Messages ==

There seems to always be a question or two surrounding what a person
should put in a patch header, or commit message.

The general rules always apply, those being that there is a single line
short log or summary of the change (think of this as the subject when the patch
is e-mailed), and then the more detailed long log, and closure with tag lines
such as "Signed-off-by:".

== New Development ==

A minimal patch or commit message would be of the format:

Short log / Statement of what needed to be changed.

(Optional pointers to external resources, such as defect tracking)

The intent of your change.

(Optional, if it's not clear from above) how your change resolves the
issues in the first part.

Tag line(s) at the end.

For example:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The minimal commit message is good for new code development and simple changes.

The single short log message indicates what needed to be changed. It should
begin with an indicator as to the primary item changed by this work,
followed by a short summary of the change. In the above case we're indicating
that we've changed the "foobar" item, by "adjusting the foo setting in bar".

The single short log message is analogous to the git "commit summary". While no
maximum line length is specified by this policy, it is suggested that it remains
under 78 characters wherever possible.

Optionally, you may include pointers to defects this change corrects. Unless
the defect format is specified by the component you are modifying, it is
suggested that you use a full URL to specify the reference to the defect
information. Generally these pointers will precede any long description, but as
an optional item it may be after the long description.

For example, you might refer to an open source defect in the RPM project:

rpm: Adjusted the foo setting in bar

<nowiki>[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5</nowiki>

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

You must then have a full description of the change. Specifying the intent of
your change and if necessary how the change resolves the issue.

As mentioned above this is intended to answer the "what were you thinking"
questions down the road and to know what other impacts are likely to
result of the change needs to be reverted. It also allows for better
solutions if someone comes along later and agrees with paragraph 1
(the problem statement) and either disagrees with paragraph 2
(the design) or paragraph 3 (the implementation).

Finally one or more tag lines should exist. Each developer responsible
for working on the patch is responsible for adding a Signed-off-by: tag line.

It is not acceptable to have an empty or non-existent header, or just a single
line message. The summary and description is required for all changes.

== Importing from Elsewhere ==

If you are importing work from somewhere else, such as a cherry-pick from
another repository or a code patch pulled from a mailing list, the minimum patch
header or commit message is not enough. It does not clearly establish the
provenance of the code.

The following specifies the additional guidelines required for importing changes
from elsewhere.

By default you should keep the original author's summary and description,
along with any tag lines such as, "Signed-off-by:". If the original change that
was imported does not have a summary and/or commit message from the original
author, it is still your responsibility to add the summary and commit message to
the patch header. Just as if you wrote the code, you should be able to clearly
explain what the change does. It is also necessary to document the original
author of the change. You should indicate the original author by simply stating
"written by" or "posted to the ... mailing list by". You must not add a
Signed-off-by: tag line with their name, only the original author may add their
own Signed-off-by: tag line.

It is also required that the origin of the work be fully documented. The origin
should be specified as part of the commit message in a way that an average
developer would be able to track down the original code. URLs should reference
original authoritative sources and not mirrors.

If changes were required to resolve conflicts, they should be documented as
well. When incorporating a commit or patch from an external source, changes to
the functionality not related to resolving conflicts should be made in a
second commit or patch. This preserves the original external commit, and makes
the modifications clearly visible, and prevents confusion over the author of the
code.

Finally you must add a "Signed-off-by:" tag line to indicate that you worked
with the patch.

=== Example: Importing from Elsewhere Unmodified ===

For example, if you were to pull in the patch from the above example unmodified:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The patch was imported from the OpenEmbedded git server
(<nowiki>git://git.openembedded.org/openembedded</nowiki>) as of commit id
b65a0e0c84cf489bfa00d6aa6c48abc5a237100f.

Signed-off-by: Your Name <your.name@openembedded.org>

The above example shows a clear way for a developer to find the original source
of the change. It indicates that the OpenEmbedded developer simply integrated
the change into the system without making any further modifications.

=== Example: Importing from Elsewhere Modified ===

When importing a patch, occasionally changes have to be made in order to resolve
conflicts. The following is an example of the commit message when the item needed
to be modified:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The patch was imported from the OpenEmbedded git server
(<nowiki>git://git.openembedded.org/openembedded</nowiki>) as of commit id
b65a0e0c84cf489bfa00d6aa6c48abc5a237100f.

A previous change had modified the memory threshold calculation,
causing a conflict. The conflict was resolved by preserving the
memory threshold calculation from the existing implementation.

Signed-off-by: Your Name <your.name@openembedded.org>

== Patch Header Recommendations ==

In order to keep track of patches and ultimately reduce the number of patches
that are required to be maintained, we need to track the status of the patches
that are created. The following items are specific to patches applied by recipes.

In addition to the items mentioned above, we also want to track if it's
appropriate to get this particular patch into the upstream project. Since we
want to track this information, we need a consistent tag and status indicator
that can be searched.

While adding this tracking information to the patch headers is currently
optional, it is highly recommended and some maintainers may require it. It is
optional at this time so that it can be evaluated as to its usefulness over
time. Existing patches will be updated with the tag as they are modified.

As mentioned in the above, be sure to include any URL to bug tracking, mailing
lists or other reference material pertaining to the patch. Then add a new tag
"Upstream-Status:" which contains one of the following items:

'''Pending'''
- No determination has been made yet or not yet submitted to upstream

'''Submitted''' [where]
- Submitted to upstream, waiting approval
- Optionally include where it was submitted, such as the author, mailing
list, etc.

'''Accepted'''
- Accepted in upstream, expect it to be removed at next update, include
expected version info

'''Backport'''
- Backported from new upstream version, because we are at a fixed version,
include upstream version info

'''Denied'''
- Not accepted by upstream, include reason in patch

'''Inappropriate [reason]'''
- The patch is not appropriate for upstream, include a brief reason on the
same line enclosed with []
reason can be:
not author (You are not the author and do not intend to upstream this,
source must be listed in the comments)
native
licensing
configuration
enable feature
disable feature
bugfix (add bug URL here)
embedded specific
no upstream (the upstream is no longer available -- dead project)
other (give details in comments)

The various "Inappropriate [reason]" status items are meant to indicate that the
person responsible for adding this patch to the system does not intend to
upstream the patch for a specific reason. Unless otherwise noted, another
person could submit this patch to the upstream, if they do the status should be
changed to "Submitted [where]", and an additional Signed-off-by: line added to
the patch by the person claiming responsibility for upstreaming.

For example, if the patch has been submitted upstream:

rpm: Adjusted the foo setting in bar

<nowiki>[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5</nowiki>

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Upstream-Status: Submitted [rpm5-devel@rpm5.org]

Signed-off-by: Joe Developer <joe.developer@example.com>

A future commit can change the value to "Accepted" or "Denied" as appropriate.

=== CVE Patches ===

In order to have a better control of vulnerabilities, patches that fix CVEs must contain the "CVE:" tag. This tag list all CVEs fixed by the patch. If more than one CVE is fixed, separate them using spaces.

==== Example: CVE patch header ====

This should be the header of patch that fixes grub2 CVE-2015-8370:

grub2: Fix CVE-2015-8370

<nowiki>[No upstream tracking] -- https://bugzilla.redhat.com/show_bug.cgi?id=1286966</nowiki>

Back to 28; Grub2 Authentication

Two functions suffer from integer underflow fault; the grub_username_get() and grub_password_get()located in
grub-core/normal/auth.c and lib/crypto.c respectively. This can be exploited to obtain a Grub rescue shell.

<nowiki>Upstream-Status: Accepted [http://git.savannah.gnu.org/cgit/grub.git/commit/?id=451d80e52d851432e109771bb8febafca7a5f1f2]</nowiki>
CVE: CVE-2015-8370
Signed-off-by: Joe Developer <joe.developer@example.com>

== Common Errors in Patch and Commit Messages ==

- Short log does not start with the file or component being modified. Such as
"foo: Update to new upstream version 5.8.9"

- Avoid starting the summary line with a capital letter, unless the component
being referred to also begins with a capital letter.

- Don't have one huge patch, split your change into logical subparts. It's
easier to track down problems afterward using tools such as git bisect. It also
makes it easy for people to cherry-pick changes into things like stable branches.

- Don't simply translate your change into English for a commit log. The log
"Change compare from zero to one" is bad because it describes only the code
change in the patch; we can see that from reading the patch itself. Let the code
tell the story of the mechanics of the change (as much as possible), and let
your comment tell the other details -- i.e. what the problem was, how it
manifested itself (symptoms), and if necessary, the justification for why the
fix was done in manner that it was. In other words, the long commit message
must describe *why* the change was needed (instead of what has changed).

- Don't start your commit log with "This patch..." -- we already know it is a patch.

- Don't repeat your short log in the long log. If you really really don't have
anything new and informational to add in as a long log, then don't put a long
log at all. This should be uncommon -- i.e. the only acceptable cases for no
long log would be something like "Documentation/README: Fix spelling mistakes".

- Don't put absolute paths to source files that are specific to your site, i.e.
if you get a compile error on "fs/exec.c" then tidy up the parts of the compile
output to just show that portion. We don't need to see that it was
/home/wally/src/git/oe-core/meta/classes/insane.bbclass or similar - that full
path has no value to others. Always reduce the path to something more
meaningful, such as oe-core/meta/classes/insane.bbclass.

- Always use the most significant ramification of the change in the words of
your subject/shortlog. For example, don't say "fix compile warning in foo" when
the compiler warning was really telling us that we were dereferencing complete
garbage off in the weeds that could in theory cause an OOPS under some
circumstances. When people are choosing commits for backports to stable or
distro kernels, the shortlog will be what they use for an initial sorting
selection. If they see "Fix possible OOPS in...." then these people will look
closer, whereas they most likely will skip over simple warning fixes.

- Don't put in the full 20 or more lines of a backtrace when really it is just
the last 5 or so function calls that are relevant to the crash/fix. If the entry
point, or start of the trace is relevant (i.e. a syscall or similar), you can
leave that, and then replace a chunk of the intermediate calls in the middle
with a single [...]

- Don't include timestamps or other unnecessary information, unless they are
relevant to the situation (i.e. indicating an unacceptable delay in a driver
initialization etc.)

- Don't use links to temporary resources like pastebin and friends. The commit
message may be read long after this resource timed out.

- Don't reference mirrors, but instead always reference original authoritative
sources.

- Avoid punctuation in the short log.

[[Category:Policy]]

OEDEM 2016

2016-09-28T14:46:43Z

Mhatle: /* Agenda Items */

NOTE: for the OpenEmbedded General Assembly 2016, held the same day at the same location, see the page [[Berlin, 2016]]

== Location and Time ==

Friday, October 14 2016 
(after ELC-E [Oct 11-13])

Specific location in Berlin will be announced later

== Attendees ==

* Andrei Gherzan
* Khem Raj "khem"
* Armin Kuster "Armpit"
* Marco Cavallini "mckoan"
* Pierre-Jean Texier "pjtexier"
* Koen Kooi "koen"
* Anders Darander
* Sean Hudson "darknighte"
* Changhyeok Bae "chbae"
* Peter Kjellerstedt "Saur"
* Josef Holzmayr "LetoThe2nd"
* Vicentiu Neagoe
* Nicolas Dechesne "ndec"
* Denys Dmytriyenko "denix"
* Philip Balister "Crofton"
* Jan Lübbe "shoragan"
* Enrico Jörns
* Jeff Osier-Mixon "Jefro"
* Andrea Galbusera "gizero"
* Richard Purdie "RP"
* Changhyeok bae "chbae"
* Scott Murray "smurray"
* Behan Webster "behanw"
* Jan-Simon Moeller "dl9pf"
* Bruce Ashfield "zediii"
* Marcin Juszkiewicz "hrw"
* Mark Hatle "fray"

== Agenda Items ==

OpenEmbedded eV General Assembly [[Berlin,_2016]]

# LTS
# BSPs and layer name recommendations
# Make perl and python distro features? [Saur]
# Support for meson build system? [Saur]
# Can devtool be made standalone, or how to use latest devtool with older version of OE? [Saur]
# Show off Wind River 'setup' tool (response to discussions from last year and this spring) [Fray]

== Minutes ==

OEDEM 2016

2016-09-28T14:45:37Z

Mhatle: /* Attendees */

OEDAM 2016

2016-03-10T22:47:46Z

Mhatle: /* Attendees */

== Location and Time ==

Friday April 8 2016 (after ELC [Apr 4-6] and Yocto Project Developer Day [Apr 7])

== Attendees ==
Armin Kuster (Armpit) 
Behan Webster (behanw) 
Denys Dmytriyenko (denix) 
Jan-Simon Möller (dl9pf) 
Philip Balister (Crofton|work) 
Sean Hudson (darknighte) 
Tim Orling (Moto-timo) 
Trevor Woerner 
Tom King (ka6sox) 
Mark Hatle (fray) 

== Agenda Items ==

* Review items from last meeting in Dublin (Please add remarks)

===BSPs & Layer Index===
* oe driving toward machine readable files, can also drive from layer
test (yp compat) side - KK
* publicize defs of each list? discuss on members list - where to
overall architecture discussions belong
* TSC to add new mailing list specifically about project
architecture, no patches, and determine list name
* KK bring discussion to new mailing list

===BSP standards - standards exist,coming soon are methods for comparing===
a given layer with the standard
* need wiki documentation for TSC to ratify
* document distro vs. machine policies, other best practices

===BSP layer availability===
* encourage board vendors to provide a BSP, make it easier to do so
* ask yp ab to talk to linux.com [PB] - community issue
* look at digi manual (i.e. "just add bsp changes in local.conf")

===meta-yocto to meta-poky===
* RP to take action on the YP side, possibly not for 2.0
* formalize lowercase as mandatory requirement for overrides, look
for other options in the future

===Layer Index feature requests discussed with layer index build feedback>>===
* ask for cycles from employers to work on these issues
* make people aware, help with triage, provide resources

===Security===
* need a method for tracking CVs

===Systemd===
complaints: more package config, comments; look at packaging to have
minimal systemd even with options available
* need someone with time available to work on it
* need to get systemd feedback, people using systemd talk to package maintainer
* file systemctl wrapper bug

===Advocacy===
* board to discuss advocacy/community, invite jefro to meetings
Potential OE Day at ELCE next year
* board: do a proper cfp

== Minutes ==

OEDEM 2015

2015-08-15T15:18:16Z

Mhatle: /* Attendees */

The 2015 OpenEmbedded Developer's European Meeting will take place Friday, Oct 9, 2015 in Dublin, Ireland. Part of this meeting will also constitute the annual OE General Assembly, and some corporation/eV matters will be discussed.

== Location and Time ==

[http://www.thegibsonhotel.ie/ The Gibson Hotel] 
Point Village 
Dublin 1, Ireland 
+353 1 681 5000

9am - 5pm (note: times may change)

== OpenEmbedded eV General Assembly ==

We have a short administrative meeting before starting the Developer meeting. All are welcome for this.

* Proposal to dissolve the current e.V. and re-constituting it in the U.S. to reduce the administrative burden for the project.
* Define a method for identifying lapsed memberships.
* Proxy form [[File:Proxy_instructions-oe.pdf]]

== Agenda ==

* Standards for BSP behavior. (Crofton)
* Tentative: User/group definitions. Distribution wide (uid/gid) vs layer wide (primary group, path, description, etc for users). (Saur)
* LayerIndex feature requests (tlwoerner)
** build feedback (autobuilders and off-site)
** master branch parsability
* Systemd packaging. As new things appear, they are lumped into one package. Update split? (Crofton relaying info)
* meta-yocto -> meta-poky. People are still spending too much time explaining reality. (Crofton)

== Attendees ==

If you plan to attend, please list your name below.

# Jeff Osier-Mixon (jefro)
# Philip Balister (Crofton)
# Trevor Woerner (tlwoerner)
# Anders Darander (andersd)
# Marco Cavallini (mckoan)
# Koen Kooi (koen)
# Alexandre Belloni (abelloni)
# Armin Kuster (armpit)
# Belén Barros Pena (belen) (tentative)
# Peter Kjellerstedt (Saur)
# Alejandro del Castillo (adelcast)
# Denys Dmytriyenko (denix)
# Sean Hudson (darknighte)
# Nathan Rossi (nrossi)
# Mark Hatle (fray)

OEDAM 2015

2015-03-23T20:47:06Z

Mhatle: /* Attendees */

== Location and Time ==

March 27-28 (after ELC and Yocto Project Developer Day)

2315 North 1st Street, San Jose, CA 95131

https://goo.gl/maps/1pRGR

Now that people are already claiming to attend by Google Hangout, I will announce our intention to also webcast via Google Hangout ;)

== Attendees ==

* Philip Balister (Crofton)
* Denys Dmytriyenko (denix)
* Armin Kuster (Armpit)
* Martin Jansa (JaMa) - 50%
* Michael Halstead (halstead) - Friday only
* Jeff Osier-Mixon (jefro)
* Paul Eggleton (bluelightning)
* Steve Arnold (nerdboy/mr_science)
* Stephanie Lockwood-Childs (wormo/sjl) - via hangout
* Ron Lockwood-Childs (speedy1) - via hangout
* Herb Kuta
* Tim Orling (moto-timo)
* Sean Hudson (darknighte)
* Bruce Ashfield (zedd) - probably
* Mark Hatle (fray)
* Sona Sarmadi
* Changhyeok Bae
* Belén Barros Pena (belen) - via hangout

== Agenda Items ==

* Progress on items identified during OEDAM 2014
* Development cycle
* Patch review tools
* Tests
* Removing the oe-core and meta-oe repos on github. These repos are now called openembedded-core and meta-openembedded/
* People still use OE-classic from time to time. Should we remove the repo?
* Security processes
* Community Development / Team Building (please add more examples/ideas... almond flour cake is always good...)
** Intro / crash course events
** Coordinated bug day sprints
** Other group/meetup activities

OEDAM 2015

2015-03-13T17:07:28Z

Mhatle: /* Attendees */

== Location and Time ==

March 27-28 (after ELC and Yocto Project Developer Day)

2315 North 1st Street, San Jose, CA 95131

https://goo.gl/maps/1pRGR

Now that people are already claiming to attend by Google Hangout, I will announce our intention to also webcast via Google Hangout ;)

== Attendees ==

* Philip Balister (Crofton)
* Denys Dmytriyenko (denix)
* Armin Kuster (Armpit)
* Martin Jansa (JaMa) - 50%
* Michael Halstead (halstead) - Friday only
* Jeff Osier-Mixon (jefro)
* Paul Eggleton (bluelightning)
* Steve Arnold (nerdboy/mr_science)
* Stephanie Lockwood-Childs (wormo/sjl) - via hangout
* Ron Lockwood-Childs (speedy1) - via hangout
* Herb Kuta
* Tim Orling (moto-timo)
* Sean Hudson (darknighte)
* Bruce Ashfield (zedd) - probably
* Mark Hatle (fray) - probably

== Agenda Items ==

* Progress on items identified during OEDAM 2014
* Development cycle
* Removing the oe-core and meta-oe repos. These repos are now called openembedded-core and meta-openembedded/
* People still use OE-classic from time to time. Should we remove the repo?
* Community Development / Team Building (please add more examples/ideas... almond flour cake is always good...)
** Intro / crash course events
** Coordinated bug day sprints
** Other group/meetup activities

Dusseldorf, 2014

2014-09-05T22:37:57Z

Mhatle: /* Attendance */

==Location ==

Congress Centre Düsseldorf, Düsseldorf, Germany

October 14, 2014 at 1300 - 1500 in room 10.

== Agenda ==

* Financial report
* Election of new members

== Forms ==

* Proxy form [[File:Proxy_instructions-oe.pdf]]

== Attendance ==
* Martin 'JaMa' Jansa
* Jeff 'Jefro' Osier-Mixon
* Denys 'denix' Dmytriyenko
* Dave "davest" Stewart (likely)
* Philip "Crofton" Balister
* Ross "rburton" Burton
* 'Khem' Raj
* Mark 'fray' Hatle

== Meeting notes ==

== Action Items ==

OEDAM 2014

2014-05-13T18:40:21Z

Mhatle: /* Why is embedded still hard */

= OpenEmbedded Developers America Meeting =

The OpenEmbedded Project held a developers meeting May 2-3, 2014, in Santa Clara, CA. This meeting was co-located with the Embedded Linux Conference North America. All active OpenEmbedded developers were invited to attend.

Contact [mailto:jefro@jefro.net Jefro] with any questions.

== Location & Time ==

May 2-3, 2014 
9am - 5pm

[https://www.google.com/maps/preview#!q=4600+Patrick+Henry+Dr+Santa+Clara%2C+CA+95054+&data=!1m4!1m3!1d9724!2d-121.985724!3d37.3978139!4m15!2m14!1m13!1s0x808fc9d0bdf38417%3A0x8ca7d9c558968760!3m8!1m3!1d2439!2d-80.4013806!3d37.1440125!3m2!1i1366!2i589!4f13.1!4m2!3d37.3978139!4d-121.985724 Ettus Research / National Instruments] 
4600 Patrick Henry Drive 
Santa Clara, CA 95054 USA

== Goals ==

* Have fun.
* Get useful work done that benefits from high bandwidth interactions.
* Get more people involved with the project at a higher level.

== Working Agenda ==

This agenda is flexible, and meant to reflect the important issues in the OE community. Please feel free to contribute agenda items. We will finalize the agenda after introduction to maximize use of people's time.

* Introductions. Be prepared to tell us who you are and how you use OpenEmbedded (and the Yocto Project)
* The Yocto Project is supposed to make Embedded easy. What is still hard?
* bug scrub (also bug collecting/wrangling)
* ongoing role of the OE TSC
* wiki/website organization
* online voting
* increasing the amount of hardware that works out of the box with oe-core + layers
* next + 1 release
* quality of meta-oe and what to do with it: http://lists.openembedded.org/pipermail/openembedded-devel/2014-March/094893.html
* lune to replace sato?
* developer community - outreach/recruiting/mentoring, new developer documentation, process and QA tools
* image deployment/update best practices (shared yocto issue)
* feedback on Toaster, the web interface for BitBake (some background in this thread https://lists.yoctoproject.org/pipermail/yocto/2014-April/018956.html)
* infrastructure sync

== Registered Attendees ==

* Philip Balister (Crofton)
* <strike>Richard Purdie (RP)</strike>
* Mark Hatle (fray)
* Khem Raj (khem)
* Martin Jansa (jama)
* Armin Kuster (akuster)
* Jeff Osier-Mixon (jefro)
* Alejandro del Castillo
* Tom King (ka6sox)
* Steve Arnold (mr_science/nerdboy)
* Herb Kuta
* Trevor Woerner (tlwoerner)
* Sean Hudson (darknighte)
* Denys Dmytriyenko (denix)
* Toby Flynn
* Adam Bell
* Belen Barros Pena (belen)
* Brian Hutchinson
* Tim Orling (moto-timo)
* Michael Halstead (halstead)
* <your name here>

= Minutes =

== Actual Attendance ==

* armin kuster
* tim orling
* herb kuta
* martin jansa
* toby flynn
* steve arnold
* adam bell
* mark hatle
* sean hudson
* philip balister
* denys dmitriyenko
* alan bennett
* trevor woerner
* michael halstead
* belen barros pena
* bill mills
* khem raj
* alejandro del castillo
* jefro

on call:
* richard purdie
* paul eggleton
* saul wold
* bruce ashfield

== Agenda ==

We worked out a detailed agenda as the first task on Friday morning.

Fri am:
* introductions
* Lava
* ongoing role of the TSC
* Why is embedded still hard? & RP's email

Fri pm:
* unfinished business from this morning
* Lune to replace Sato
* increasing BSP quantity & quality
* next+1 release
* online voting
* best practices - image deployment, default config

Sat:
* unfinished business from Friday
* wiki/website organization
* meta-oe quality
* developer community outreach
* bug scrub

offline:
* toaster feedback
* infrastructure sync

== Lava ==

no board farm, possible to set up distributed board farm 
worried about documentation 
long term aggregate many, for now just a handful of hardware 
linaro can help define lava piece 
define what to certify, then push results 
simple data: configuration and red/green (yellow?) on layer+config+hw basis 

3 stages: 
* building
* testing (potentially lava)
* aggregation & publishing data (toaster?)

lava has ways to report results, suspect regressions 
drilldown capability 
lava difficult to connect inside/outside firewalls

error reporting tool in 1.6 
standard output push to bug reporting tool like this 
like a failure pastebin 
can mine data & look for trends

porting 1.6 requires resources 
2k tests in 1.6, more tests less necessary than integration

parallel work, not for 1.7 
nice to have simplified install complete with lava, hw pieces, additional test pieces 
interface for aggregation - porting mechanism 
built on error reporting system

action items:
* autobuilder assets tested in lava, apply to ab output (alan -> beth, michael)
* take some tests performed in 1.6 and bring into lava (ti/bill, steve nerdboy)
* define requirements for aggregation system (sean)

== Ongoing role of the TSC ==

needed someone to set direction 
fulfilled original charter 
board & happy with TSC - responsive 
still needed? 
new public meeting + as needed format is not much work 
hardest thing is topics 
* jefro to help with topics
need to do more? less? 
denys - people would like to see more diversity 
heavy on yocto project 
members have worked to isolate roles 
community requests yes keep going, yes keep elections 
if contentious issue, group that can vote 
like RP not being overwhelmed & provide contrast 
community meetings very useful 
some concern about uniformity/affiliations but trust exists 
mark: call us on it if you think we are deciding for intel/wr/whoever 

board to do these things within the next month:
* ask community if anyone else wants to step in or if anyone wants to step down
* vote everyone together

mark asked about discussion re dissolving ev, moving it elsewhere, etc 
no movement planned right now 
have account for donations

== Why is embedded still hard ==

i.e. best practices

learning curve 
oe and yp trying to make embedded easier 
many tasks involved 
finding changes/errors involves many steps 
locked sstate coming 
just documentation issue? no, but that is where to begin 
many people don't want to know about build systems, they want magic to happen 
system integrateors are very happy with yp 
app developers don't care 
- app developers don't (or shouldn't) care about using all of bitbake/oe 
- they should care about compiling, debugging, and even deployment of their application 

figure out steps, maybe yp does - white papers, best practices, disseminate 
what do i hav eto do day to day to get it done 
encapsulate problem 
* capture daily workflow, write email

aggregate - yp tech writer

external source class stuff 
narcissus good at making images, but can make images that don't work 
much discussion about details 
* document personal workflows as best we can & categorize
* DEPENDS overloaded? khem

* document workflows based on rp's email
* based on workflows, determine how extermal toolchains to be used
* dependency tree
* ? feed based assembly for the yocto autobuilders
* install built toolchain as toolchain (steve)
* generate RPMs - external source workflow
* eclipse

* take to list

== Increasing layer quality ==

have a lot of layers, some better maintained than others 
monitor maintainers 
where to file bugs 
MAINTAINER file should include where to send patches, report problems 
eg on github use github issues system or to mailing list 
possibly on yp bugzilla?

RP: 
no new bugs in yp bugzilla, policy is not supporting bugs for outside projects 
unless someone from that project joins the triage team 

quality of any layer, esp oe layers, bring it up to the tsc & main maintainer if one exists 
document: 
1. maintainer makes right technical decisions for layer - rp is the boss 
2. must respond on ml within reasonable time, longer than 2 weeks is too long 
3. if cant do maintainership along with other paid work, find someone 

maintainer = quality

* philip: for yp users: best practice read martin's emails about broken recipes
disable broken layers? 
has to be a way to tell people who don't read emails 
move or remove on 2nd iteration 
blacklist recipe inside itself 
add to default inherit

* increase pain for stuff that just fails
highlight qa fails, cheerlead & encourage people to look at 
more review comments helps 
collect stats 
explain what using stats for 

advertise e.g. meta-selinux

* extra fields on layer index (maintainers, readme file)
* philip to send to list re updates to README files, paul to see if he can parse things out of it
* place to put results if attached to autobuilder: dont have to advocate right away, best practices on what to bulid against - 1.4, 1.5, master, etc
* guidance to new maintainers

Philip:
* working demos & starting points, e.g. qt demo 
cinematic 
working on a set of hardware, not just one 
also document how I made the demo - no reason for handwaving

devday - splitting by function, auto/iot/medical 
getting a number of vendors interested in helping with

* pb to agitate on list for demo with recipes, no bsps - non arch specific

== lune ==

came from openwebos 
rewritten to be in qt5 
great for phones, tablets, internet of 5 
need qt5 in oe-core 
many demos 
html5 
luna is webos specific. this is lune os

general consensus - yes, worth making it work for oe demos 
bmills would love to see solutoin that does not require hw acceleration 
good demo that can build 
not good for qemu

2 things 
need sane demo that looks better than sato 
better demo for high end hardware

node.js is another option 
lune uses v8 or node

* jama to do initial work, will post for help when needed
* not to replace completely & put into oe-core, don't want to upstream due to requiring change in qt

== next+1 release ==

khem: meta-oe in yocto (1.8? no, 4.04) 
poky, oe-core YP compatible

improve deployment 
currently board specific, maybe my board is different 
also installer - config for 1st boot

common pieces - same sd card formatting several different products 
so spit out sd image small then enlarged, awesome 
solves 80% of installation needs 
deployment config file 
no sudo 
WIC 
target: 1.7

upgrade methods that don't use package feeds 
autobuilder & tester feeds to upgrade 
can't rely on distros to alert 
khem - images want upgradability 
eg internal storage and an SD card 
dual identical partitions for failover 

best practices, industry examples 
need good samples 
=> please document this problem if you have solved it 
in-service updates 
other option - atomic package feeds

plans for clang on oe 
fray's secondary toolchain layer 
layer availalbe, not in oe-core

html5 applications to become first class citizens 
oe? oe-core? bill: layer, but well supported 
also for processors without hw accel 
scale down to dumb framebuffers

lava 
steve: deployed jenkins to apache

cryptographically signed sstate cache 
secure development life cycles

== online voting & other board issues ==

drupal site for yocto project 
registered members can vote 
code is built already, like surveymonkey

Philip: any sane system is great 
* sean to drive voting system

more regular developer meetings 
build a sense of community 
need advance notice - 2-3 months 
find space

devday - trainers, overlap with plumbers

funding discussion - ongoing

= Mark Hatle's notes =

Complaints about local.conf files from 3rd parties...

— local.conf generation, comment it! 

solution use format/comments similar to existing local.conf and local.conf.extended so new developers can translate their work.

----

Friday Morning discussion, couple of key points to remember as we move forward with oe development:

— Document workflows (based on RPs email) 
— Reuse SDK toolchain 
— External SRC workflow 
— manual adjust (opt-out) of dep change 
— feed based assembly for app dev 
— Generate a ‘package’ from the SDK, add to feed and deploy 

----

Friday Afternoon discussion, making embedded easier. Few key things we can do..

— Git/SCM workflow/best practices 
— SCM rate of change is an indication of quality 
— OE Stats/Recipe usage stats 
— layer index — maintainer info, submit patches, auto builder, test results, … 
— working demos 

----

Saturday Next+1 Future discussion...

— deployment (wic — partitions… , network fs) 
— heterogeneous systems (GPU, DSP, multi arch) 
— HTML5 apps become first class citizens

OEDAM 2014

2014-05-13T15:04:47Z

Mhatle: /* Mark Hatle's notes */

= OpenEmbedded Developers America Meeting =

The OpenEmbedded Project held a developers meeting May 2-3, 2014, in Santa Clara, CA. This meeting was co-located with the Embedded Linux Conference North America. All active OpenEmbedded developers were invited to attend.

Contact [mailto:jefro@jefro.net Jefro] with any questions.

== Location & Time ==

May 2-3, 2014 
9am - 5pm

[https://www.google.com/maps/preview#!q=4600+Patrick+Henry+Dr+Santa+Clara%2C+CA+95054+&data=!1m4!1m3!1d9724!2d-121.985724!3d37.3978139!4m15!2m14!1m13!1s0x808fc9d0bdf38417%3A0x8ca7d9c558968760!3m8!1m3!1d2439!2d-80.4013806!3d37.1440125!3m2!1i1366!2i589!4f13.1!4m2!3d37.3978139!4d-121.985724 Ettus Research / National Instruments] 
4600 Patrick Henry Drive 
Santa Clara, CA 95054 USA

== Goals ==

* Have fun.
* Get useful work done that benefits from high bandwidth interactions.
* Get more people involved with the project at a higher level.

== Working Agenda ==

This agenda is flexible, and meant to reflect the important issues in the OE community. Please feel free to contribute agenda items. We will finalize the agenda after introduction to maximize use of people's time.

* Introductions. Be prepared to tell us who you are and how you use OpenEmbedded (and the Yocto Project)
* The Yocto Project is supposed to make Embedded easy. What is still hard?
* bug scrub (also bug collecting/wrangling)
* ongoing role of the OE TSC
* wiki/website organization
* online voting
* increasing the amount of hardware that works out of the box with oe-core + layers
* next + 1 release
* quality of meta-oe and what to do with it: http://lists.openembedded.org/pipermail/openembedded-devel/2014-March/094893.html
* lune to replace sato?
* developer community - outreach/recruiting/mentoring, new developer documentation, process and QA tools
* image deployment/update best practices (shared yocto issue)
* feedback on Toaster, the web interface for BitBake (some background in this thread https://lists.yoctoproject.org/pipermail/yocto/2014-April/018956.html)
* infrastructure sync

== Registered Attendees ==

* Philip Balister (Crofton)
* <strike>Richard Purdie (RP)</strike>
* Mark Hatle (fray)
* Khem Raj (khem)
* Martin Jansa (jama)
* Armin Kuster (akuster)
* Jeff Osier-Mixon (jefro)
* Alejandro del Castillo
* Tom King (ka6sox)
* Steve Arnold (mr_science/nerdboy)
* Herb Kuta
* Trevor Woerner (tlwoerner)
* Sean Hudson (darknighte)
* Denys Dmytriyenko (denix)
* Toby Flynn
* Adam Bell
* Belen Barros Pena (belen)
* Brian Hutchinson
* Tim Orling (moto-timo)
* Michael Halstead (halstead)
* <your name here>

= Minutes =

== Actual Attendance ==

* armin kuster
* tim orling
* herb kuta
* martin jansa
* toby flynn
* steve arnold
* adam bell
* mark hatle
* sean hudson
* philip balister
* denys dmitriyenko
* alan bennett
* trevor woerner
* michael halstead
* belen barros pena
* bill mills
* khem raj
* alejandro del castillo
* jefro

on call:
* richard purdie
* paul eggleton
* saul wold
* bruce ashfield

== Agenda ==

We worked out a detailed agenda as the first task on Friday morning.

Fri am:
* introductions
* Lava
* ongoing role of the TSC
* Why is embedded still hard? & RP's email

Fri pm:
* unfinished business from this morning
* Lune to replace Sato
* increasing BSP quantity & quality
* next+1 release
* online voting
* best practices - image deployment, default config

Sat:
* unfinished business from Friday
* wiki/website organization
* meta-oe quality
* developer community outreach
* bug scrub

offline:
* toaster feedback
* infrastructure sync

== Lava ==

no board farm, possible to set up distributed board farm 
worried about documentation 
long term aggregate many, for now just a handful of hardware 
linaro can help define lava piece 
define what to certify, then push results 
simple data: configuration and red/green (yellow?) on layer+config+hw basis 

3 stages: 
* building
* testing (potentially lava)
* aggregation & publishing data (toaster?)

lava has ways to report results, suspect regressions 
drilldown capability 
lava difficult to connect inside/outside firewalls

error reporting tool in 1.6 
standard output push to bug reporting tool like this 
like a failure pastebin 
can mine data & look for trends

porting 1.6 requires resources 
2k tests in 1.6, more tests less necessary than integration

parallel work, not for 1.7 
nice to have simplified install complete with lava, hw pieces, additional test pieces 
interface for aggregation - porting mechanism 
built on error reporting system

action items:
* autobuilder assets tested in lava, apply to ab output (alan -> beth, michael)
* take some tests performed in 1.6 and bring into lava (ti/bill, steve nerdboy)
* define requirements for aggregation system (sean)

== Ongoing role of the TSC ==

needed someone to set direction 
fulfilled original charter 
board & happy with TSC - responsive 
still needed? 
new public meeting + as needed format is not much work 
hardest thing is topics 
* jefro to help with topics
need to do more? less? 
denys - people would like to see more diversity 
heavy on yocto project 
members have worked to isolate roles 
community requests yes keep going, yes keep elections 
if contentious issue, group that can vote 
like RP not being overwhelmed & provide contrast 
community meetings very useful 
some concern about uniformity/affiliations but trust exists 
mark: call us on it if you think we are deciding for intel/wr/whoever 

board to do these things within the next month:
* ask community if anyone else wants to step in or if anyone wants to step down
* vote everyone together

mark asked about discussion re dissolving ev, moving it elsewhere, etc 
no movement planned right now 
have account for donations

== Why is embedded still hard ==

i.e. best practices

learning curve 
oe and yp trying to make embedded easier 
many tasks involved 
finding changes/errors involves many steps 
locked sstate coming 
just documentation issue? no, but that is where to begin 
many people don't want to know about build systems, they want magic to happen 
system integrateors are very happy with yp 
app developers don't care 
figure out steps, maybe yp does - white papers, best practices, disseminate 
what do i hav eto do day to day to get it done 
encapsulate problem 
* capture daily workflow, write email

aggregate - yp tech writer

external source class stuff 
narcissus good at making images, but can make images that don't work 
much discussion about details 
* document personal workflows as best we can & categorize
* DEPENDS overloaded? khem

* document workflows based on rp's email
* based on workflows, determine how extermal toolchains to be used
* dependency tree
* ? feed based assembly for the yocto autobuilders
* install built toolchain as toolchain (steve)
* generate RPMs - external source workflow
* eclipse

* take to list

== Increasing layer quality ==

have a lot of layers, some better maintained than others 
monitor maintainers 
where to file bugs 
MAINTAINER file should include where to send patches, report problems 
eg on github use github issues system or to mailing list 
possibly on yp bugzilla?

RP: 
no new bugs in yp bugzilla, policy is not supporting bugs for outside projects 
unless someone from that project joins the triage team 

quality of any layer, esp oe layers, bring it up to the tsc & main maintainer if one exists 
document: 
1. maintainer makes right technical decisions for layer - rp is the boss 
2. must respond on ml within reasonable time, longer than 2 weeks is too long 
3. if cant do maintainership along with other paid work, find someone 

maintainer = quality

* philip: for yp users: best practice read martin's emails about broken recipes
disable broken layers? 
has to be a way to tell people who don't read emails 
move or remove on 2nd iteration 
blacklist recipe inside itself 
add to default inherit

* increase pain for stuff that just fails
highlight qa fails, cheerlead & encourage people to look at 
more review comments helps 
collect stats 
explain what using stats for 

advertise e.g. meta-selinux

* extra fields on layer index (maintainers, readme file)
* philip to send to list re updates to README files, paul to see if he can parse things out of it
* place to put results if attached to autobuilder: dont have to advocate right away, best practices on what to bulid against - 1.4, 1.5, master, etc
* guidance to new maintainers

Philip:
* working demos & starting points, e.g. qt demo 
cinematic 
working on a set of hardware, not just one 
also document how I made the demo - no reason for handwaving

devday - splitting by function, auto/iot/medical 
getting a number of vendors interested in helping with

* pb to agitate on list for demo with recipes, no bsps - non arch specific

== lune ==

came from openwebos 
rewritten to be in qt5 
great for phones, tablets, internet of 5 
need qt5 in oe-core 
many demos 
html5 
luna is webos specific. this is lune os

general consensus - yes, worth making it work for oe demos 
bmills would love to see solutoin that does not require hw acceleration 
good demo that can build 
not good for qemu

2 things 
need sane demo that looks better than sato 
better demo for high end hardware

node.js is another option 
lune uses v8 or node

* jama to do initial work, will post for help when needed
* not to replace completely & put into oe-core, don't want to upstream due to requiring change in qt

== next+1 release ==

khem: meta-oe in yocto (1.8? no, 4.04) 
poky, oe-core YP compatible

improve deployment 
currently board specific, maybe my board is different 
also installer - config for 1st boot

common pieces - same sd card formatting several different products 
so spit out sd image small then enlarged, awesome 
solves 80% of installation needs 
deployment config file 
no sudo 
WIC 
target: 1.7

upgrade methods that don't use package feeds 
autobuilder & tester feeds to upgrade 
can't rely on distros to alert 
khem - images want upgradability 
eg internal storage and an SD card 
dual identical partitions for failover 

best practices, industry examples 
need good samples 
=> please document this problem if you have solved it 
in-service updates 
other option - atomic package feeds

plans for clang on oe 
fray's secondary toolchain layer 
layer availalbe, not in oe-core

html5 applications to become first class citizens 
oe? oe-core? bill: layer, but well supported 
also for processors without hw accel 
scale down to dumb framebuffers

lava 
steve: deployed jenkins to apache

cryptographically signed sstate cache 
secure development life cycles

== online voting & other board issues ==

drupal site for yocto project 
registered members can vote 
code is built already, like surveymonkey

Philip: any sane system is great 
* sean to drive voting system

more regular developer meetings 
build a sense of community 
need advance notice - 2-3 months 
find space

devday - trainers, overlap with plumbers

funding discussion - ongoing

= Mark Hatle's notes =

Complaints about local.conf files from 3rd parties...

— local.conf generation, comment it! 

solution use format/comments similar to existing local.conf and local.conf.extended so new developers can translate their work.

----

Friday Morning discussion, couple of key points to remember as we move forward with oe development:

— Document workflows (based on RPs email) 
— Reuse SDK toolchain 
— External SRC workflow 
— manual adjust (opt-out) of dep change 
— feed based assembly for app dev 
— Generate a ‘package’ from the SDK, add to feed and deploy 

----

Friday Afternoon discussion, making embedded easier. Few key things we can do..

— Git/SCM workflow/best practices 
— SCM rate of change is an indication of quality 
— OE Stats/Recipe usage stats 
— layer index — maintainer info, submit patches, auto builder, test results, … 
— working demos 

----

Saturday Next+1 Future discussion...

— deployment (wic — partitions… , network fs) 
— heterogeneous systems (GPU, DSP, multi arch) 
— HTML5 apps become first class citizens

Edinburgh, 2013

2013-10-14T14:46:59Z

Mhatle: /* Attendence Record */

==Location ==

Edinburgh International Conference Centre, Edinburgh, UK

October 25, 2013 at 1830 - 1930

== Agenda ==

* Financial report

== Forms ==

* Proxy form [[File:Proxy_instructions-oe.pdf]]

== Attendence Record ==

(Proxies are in brackets)

* Marco Cavallini 'mckoan' (proxying for Eric Bénard 'ericben')
* Paul Eggleton 'bluelightning' (proxying for Andrea Adami 'ant')
* Philip Balister 'Crofton'
* Richard Purdie 'RP'
* Martin Jansa 'JaMa'
* Denys Dmytriyenko 'denix'
* Koen Kooi 'koen' ( Khem Raj 'khem')
* Anders Darander
* Mark Hatle 'fray' (Otavio Salvador 'otavio')

Edinburgh, 2013

2013-10-14T14:46:44Z

Mhatle: /* Attendence Record */

Adding a secondary toolchain

2013-05-21T22:59:10Z

Mhatle: /* Further comments and information */

== Background ==

In OpenEmbedded-Core there is an included toolchain that is built up of GNU binutils, gcc, and eglibc. In addition related components such as gdb, cross-prelink, mklibs and others provider additional toolchain functionality. In this document we focus on explaining best practices for including an additional toolchain that is capable of assembling, compiling and linking code. This document does not cover replacing the existing toolchain or toolchain components with external elements.

Many companies have commercial / highly optimized toolchains for specific purposes, such as ICC from Intel, LLVM, ARM Ltd toolchains, etc.. In some cases, these highly optimized toolchains may result in better performance for some applications, however they are often not generally applicable as they can not compile all of the system software. So instead of replacing the included GNU toolchain, a mechanism for providing an alternative/secondary toolchain is desired.

== Requirements ==

'''Secondary Toolchains should be implemented in a layer'''

In order to facilitate re-use and make it easier to completely disable the secondary toolchain, a layer must be used to implement the secondary toolchain.

'''Secondary Toolchain must be generally compatible with GNU toolchain'''

The secondary toolchain must be generally compatible with the idea of sysroots, linking to the defined libc, and various GNU conventions. This compatibility may be implemented directly by the secondary toolchain or via a wrapper of some type.

'''Per recipe selection of toolchain usage'''

A mechanism is needed to activate this alternative toolchain on a per-recipe basis. Using variables it should be possible to specify on a per-recipe basis which toolchain should be used, leaving the default up to the user. (It's assumed if the user does not select a default, the system GNU toolchain will be defaulted.)

'''Blacklist/Whitelist'''

Even with the requirement to be generally compatible, we can not expect all of the possible recipes to build properly with the secondary toolchain. As a consequence of this, it will be necessary to implement a blacklist (or whitelist) of specific recipes that are known to work. This way only supported components can be build by the user, avoid numerous complaints and unresolvable defects.

'''SDK'''

The SDK generated by OpenEmbedded-Core also includes the included toolchain components. It would be useful to also provide support for the secondary toolchain within the SDK environment.

== Implementation ==

=== Bitbake variables ===

A set of common bitbake variables, defined in meta/conf/bitbake.conf, are used to define how to run the toolchain components and the proper arguments.

A number of key items may need to be changed, depending on the toolchain being provided. These items may include:

* TARGET_CPPFLAGS
* TARGET_CFLAGS
* TARGET_CXXFLAGS
* TARGET_LDFLAGS
* CC
* CXX
* F77
* CPP
* LD
* CCLD
* AR
* AS
* RANLIB
* STRIP
* OBJCOPY
* OBJDUMP
* NM
* SELECTED_OPTIMIZATION
* FULL_OPTIMIZATION
* DEBUG_OPTIMIZATION

Each of the above items has a default definition within the meta/conf/bitbake.conf file. In addition, many of the components are based on specific tune variables such as:

* TUNE_CCARGS
* TUNE_LDARGS
* TUNE_ASARGS

Deciding which items need to be overridden is up to the author of the layer. It is expected that only a subset of the above referenced variables will need secondary toolchain specific versions for most implementations.

It is suggested that the secondary toolchain values be defined in a single configuration file. Within the example this is "conf/tc-example.conf". The standard name for the file should be 'conf/tc-<toolchain>.conf'.

=== Override ===

In order to allow for the layer to selectively override the variables listed above, a standard override syntax is needed.

The standard override syntax will be 'toolchain-${TOOLCHAIN}'. This will allow us to specify multiple alternative and primary toolchain combinations as necessary.

Each of the toolchain related variables can then be overridden using the standard override syntax: <var>_toolchain-<toolchain>, such as:

CC_toolchain-icc = "icc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"

In order for an individual recipe to use the secondary toolchain, the user would define: TOOLCHAIN_pn-<recipe> = 'toolchain', such as:

TOOLCHAIN_pn-bash = "icc"

The OVERRIDE is implemented in the tc-secondary.bbclass in the example.

=== Blacklist/Whitelist ===

A standard blacklist/whitelist facility is to be implemented. This facility is independent of the existing blacklist class that is part of OpenEmbedded-Core.

Similar to the existing blacklist class, you will be able to define a blacklist, per toolchain type, such as:

TCBLACKLIST[pn] = "toolchain"

or alternatively create a whitelist facility using:

TCBLACKLIST = "toolchain"
TCWHITELIST[pn] = "toolchain"

Using the second approach will allow you to blacklist all recipes, and only enable those known to work with the alternative toolchain.

The blacklist items should be defined within conf/tc-<toolchain>-blacklist.conf. The validation of the blacklist should use the standard tc-blacklist.bbclass implementation available within the example.

=== SDK ===

In order to make it easier to use the alternative toolchain within the SDK. An additional environment-* file should be created. The purpose of this file is to simply override the settings in the main environment file.

See the example section for an implementation that will create this file automatically.

== Implementation Example ==

An example implementation has been created in the layer "meta-tc-example". The file format follows:

meta-tc-example/
meta-tc-example/bin
meta-tc-example/bin/example-toolchain
meta-tc-example/conf
meta-tc-example/conf/layer.conf
meta-tc-example/conf/tc-example.conf
meta-tc-example/conf/tc-example-blacklist.conf
meta-tc-example/classes
meta-tc-example/classes/tc-secondary.bbclass
meta-tc-example/classes/tc-blacklist.bbclass

The following example is available at: [http://gate.crashing.org/~fray/yocto/meta-tc-example.tar.bz2 meta-tc-example.tar.bz2]

=== conf/layer.conf ===

# We have a conf and classes directory, append to BBPATH
BBPATH .= ":${LAYERDIR}"

BBFILE_COLLECTIONS += "tc-example"

# If we have a recipes directory, add to BBFILES
#BBFILES += "${LAYERDIR}/recipes-*/*/*.bb ${LAYERDIR}/*/recipes-*/*/*.bbappend"
#BBFILE_COLLECTIONS += "tc-example"
#BBFILE_PATTERN_tc-example := "^${LAYERDIR}/"
#BBFILE_PRIORITY_tc-example = "5"

Above is standard layer configuration. Note, the commented out sections should be re-enabled if the secondary toolchain includes recipes and/or bbappend files.

PATH := "${PATH}:${LAYERDIR}/bin"

Add to the path a standard system path, a layer specific path. This is the way you can add a secondary toolchain to the path.

# Define the alternative toolchain
SECONDARYTC = 'example'

# Enable the secondary toolchain
INHERIT += 'tc-secondary tc-blacklist'

Define the alternative toolchain name in SECONDARYTC. The inherit causes the tc-secondary.bbclass and tc-blacklist.bbclass to be loaded later. Eventually this class used SECONDARYTC to load in the secondary toolchain configuration files.

=== conf/tc-example.conf ===

# Define default system toolchain (default to built-in)
TOOLCHAIN = 'gnu'

Define a reasonable default such as gnu. The purpose of this default is to allow someone to specify the default or provide default override items for the built-in toolchain.

# Define a secondary toolchain called 'example'
# the 'example' toolchain can be activated using:
#
# Per package (preferred):
# TOOLCHAIN_pn-bash = 'example'
#
# Globally:
# TOOLCHAIN = 'example'
# TOOLCHAIN_pn-eglibc = 'gnu'

The above is an example of how to use the alternative toolchain.

# When the secondary toolchain is used, we add an automatic depend...
DEPENDS_append_toolchain-example = " virtual/TOOLCHAIN-EXAMPLE"

# ...the depend can be provided by a recipe or as an ASSUME_PROVIDED
ASSUME_PROVIDED_append = " virtual/TOOLCHAIN-EXAMPLE"

Use the above as an example on how to add additional dependencies to specific packages. This can be used to trigger a recipe to build.

# Default values from bitbake.conf...
# Variables with a '*' are things likely to need to be overridden:
#
# FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
# DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe"
# SELECTED_OPTIMIZATION = "${@d.getVar(['FULL_OPTIMIZATION', 'DEBUG_OPTIMIZATION'][d.getVar('DEBUG_BUILD', True) == '1'], True)}"
#
# * TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
#
# * TARGET_CPPFLAGS = ""
# * TARGET_CFLAGS = "${TARGET_CPPFLAGS} ${SELECTED_OPTIMIZATION}"
# * TARGET CXXFLAGS = "${TARGET_CFLAGS} -fpermissive"
# * TARGET_LDFLAGS = "-Wl,-O1 ${TARGET_LINK_HASH_STYLE}"
#
# TARGET_CC_ARCH = "${TUNE_CCARGS}"
# TARGET_LD_ARCH = "${TUNE_LDARGS}"
# TARGET_AS_ARCH = "${TUNE_ASARGS}"
#
# HOST_CC_ARCH = "${TARGET_CC_ARCH}"
# HOST_LD_ARCH = "${TARGET_LD_ARCH}"
# HOST_AS_ARCH = "${TARGET_AS_ARCH}"
#
# * CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CXX = "${CCACHE}${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * F77 = "${CCACHE}${HOST_PREFIX}g77 ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CPP = "${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
# * LD = "${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"
# * CCLD = "${CC}"
# AR = "${HOST_PREFIX}ar"
# AS = "${HOST_PREFIX}as ${HOST_AS_ARCH}"
# RANLIB = "${HOST_PREFIX}ranlib"
# STRIP = "${HOST_PREFIX}strip"
# OBJCOPY = "${HOST_PREFIX}objcopy"
# OBJDUMP = "${HOST_PREFIX}objdump"
# NM = "${HOST_PREFIX}nm"

# Override values:
CC_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CXX_toolchain-example = "example-toolchain ${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CPP_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
LD_toolchain-example = "example-toolchain ${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"

The above implements the necessary overrides for the example toolchain.

=== conf/tc-example-blacklist.conf ===

Example configuration file that enables the blacklist and defines the defaults.

# Blacklist certain items...
TCBLACKLIST[eglibc] = 'example'
TCBLACKLIST[bash] = 'example'

#
# or
#

# Switch to a global blacklist and whitelist situation...
# TCBLACKLIST = 'example'
# TCWHITELIST[bash] = 'example'

=== classes/tc-secondary.bbclass ===

This is the standard secondary toolchain class. It should be the same in all secondary toolchain layers.

# In order to add overrides, we need to do it later then in the layers.conf
# the only standard way is using an inherit, which requires a bbclass

# Add the necessary override
TOOLCHAINOVERRIDES = ":toolchain-${TOOLCHAIN}"
TOOLCHAINOVERRIDES[vardepsexclude] = "TOOLCHAIN"

OVERRIDES .= "${TOOLCHAINOVERRIDES}"
OVERRIDES[vardepsexclude] += "TOOLCHAINOVERRIDES"

require conf/tc-${SECONDARYTC}.conf

The above code configures the basic overrides and loads the configuration file. There is one limitation to the above however. It will only load one secondary toolchain. A simple way around this is to rename the inherit and change to using something other then SECONDARYTC. This likely will not be a problem.

toolchain_create_sdk_env_script_alt () {
set -x

# Create environment setup script (secondary toolchain)
orig_script=${1:-${SDK_OUTPUT}/${SDKPATH}/environment-setup-${REAL_MULTIMACH_TARGET_SYS}}
script="$orig_script""-${SECONDARYTC}"
rm -f $script
touch $script
echo ". $orig_script" | sed -e "s:${SDK_OUTPUT}::" >> $script
if [ '${CC_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CC_toolchain-${SECONDARYTC}}' ]; then
echo 'export CC="${CC_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CXX_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CXX_toolchain-${SECONDARYTC}}' ]; then
echo 'export CXX="${CXX_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CPP_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CPP_toolchain-${SECONDARYTC}}' ]; then
echo 'export CPP="${CPP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${AS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>AS_toolchain-${SECONDARYTC}}' ]; then
echo 'export AS="${AS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${LD_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>LD_toolchain-${SECONDARYTC}}' ]; then
echo 'export LD="${LD_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${RANLIB_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>RANLIB_toolchain-${SECONDARYTC}}' ]; then
echo 'export RANLIB="${RANLIB_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${STRIP_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>STRIP_toolchain-${SECONDARYTC}}' ]; then
echo 'export STRIP="${STRIP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${OBJCOPY_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>OBJCOPY_toolchain-${SECONDARYTC}}' ]; then
echo 'export OBJCOPY="${OBJCOPY_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${OBJDUMP_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>OBJDUMP_toolchain-${SECONDARYTC}}' ]; then
echo 'export OBJDUMP="${OBJDUMP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${NM_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>NM_toolchain-${SECONDARYTC}}' ]; then
echo 'export NM="${NM_toolchain-${SECONDARYTC}}"' >> $script
fi

if [ '${CFLAGS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CFLAGS="${CFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CXXFLAGS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CXXFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CXXFLAGS="${CXXFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${LDFLAGS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>LDFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export LDFLAGS="${LDFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CPPFLAGS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CPPFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CPPFLAGS="${CPPFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi

# Fixup any sysroot references
sed -i -e "s:${STAGING_DIR_TARGET}:${SDKTARGETSYSROOT}:g" $script

# All environment scripts require OECORE_NATIVE_SYSROOT
echo 'export OECORE_NATIVE_SYSROOT="${SDKPATHNATIVE}"' >> $script
}

populate_sdk_image_append() {
toolchain_create_sdk_env_script_alt ${SDK_OUTPUT}/${SDKPATH}/environment-setup-${REAL_MULTIMACH_TARGET_SYS}
}

The above code will add an additional environment-setup-*-<toolchain> file. This can be used to enable the right environment to use the alternative toolchain. Note, it does not add anything to the path so the user will still need to configure the path to the alternative toolchain. (The layer creator can always provide the necessary 'nativesdk' or similar package that provides the alternative toolchain as part of the SDK as well.)

=== classes/tc-blacklist.bbclass ===

This is the standard secondary toolchain blacklist class. It should be the same in all secondary toolchain layers.

# Implement blacklist/whitelist functionality for alternative toolchains
# Based on the oe-core blacklist bclass
#
# To add a blacklisted package:
# TCBLACKLIST[pn] = 'toolchain'
#
# To blacklist everything, and whitelist a package:
# TCBLACKLIST = 'toolchain'
# TCWHITELIST[pn] = 'toolchain'

include conf/tc-${SECONDARYTC}-blacklist.conf

python () {
tc = d.getVar('TOOLCHAIN', True)
pn = d.getVar('PN', True)

blacklist = d.getVarFlag('TCBLACKLIST', pn, True)

if blacklist and blacklist == tc:
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))

blacklist = d.getVar('TCBLACKLIST', True)
whitelist = d.getVarFlag('TCWHITELIST', pn, True)

if blacklist and (not whitelist or whitelist != tc):
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))
}

== Further comments and information ==

For questions, comments or bug reports please contact the author at markDOThatleATwindriverDOTcom, also be sure to CC the openembedded-devel mailing list.

[[Category:Dev]]
[[Category:Layers]]

Adding a secondary toolchain

2013-01-08T21:38:43Z

Mhatle:

Commit Patch Message Guidelines

2012-12-04T17:46:25Z

Mhatle:

This set of guidelines is intended to help both the developer and reviewers of
changes determine reasonable patch headers and change commit messages. Often
when working with the code, you forget that not everyone is as familiar with the
problem and/or fix as you are. Often the next person in the code doesn't
understand what or why something is done so they quickly look at patch header
and commit messages. Unless these messages are clear it will be difficult to
understand the relevance of a given change and how future changes may impact
previous decisions.

This policy refers both to patches that are being applied by recipes as well as
commit messages that are part of the source control system, usually git. A patch
file needs a header in order to describe the specific changes that are made
within that patch, while a commit message describes one or more changes to the
overall project or system. Both the patch headers and commit messages require
the same attention and basic details, however the purposes of the messages are
slightly different. A commit message documents all of the changes made as part
of a commit, while a patch header documents items specific to a single patch. In
many cases the patch header can also be used as the commit message.

This policy does not cover the testing of the changes, the technical criteria
for accepting a patch, nor the style requirements for the technical changes. See
the [http://www.openembedded.org/wiki/Styleguide Styleguide] for more information on style requirements.

By following these guidelines we will have a better record of the problems and
solutions made over the course of development. It will also help establish a
clear provenance of all of the code and changes.

''Approved by the Technical Steering Committee 2011/06/09.''

== General Information ==

While specific to the Linux kernel development, the following could also be
considered a general guide for any Open Source development:

http://ldn.linuxfoundation.org/book/how-participate-linux-community

Many of the guidelines in this document are related to the items in that
information.

Pay particular attention to section 5.3 that talks about patch preparation.
The key thing to remember is to break up your changes into logical sections.
Otherwise you run the risk of not being able to even explain the purpose of a
change in the patch headers!

In addition section 5.4 explains the purpose of the Signed-off-by: tag line as
discussed in later parts of this document. Due to the importance of the
Signed-off-by: tag line a copy of the description follows:

Signed-off-by: this is a developer's certification that he or she has
the right to submit the patch for inclusion into the [project]. It is
an agreement to the Developer's Certificate of Origin, the full text of
which can be found in [Linux Kernel] Documentation/SubmittingPatches.
Code without a proper signoff cannot be merged into the mainline.

For more information on the Developer's Certificate of Origin and the Linux
Kernel Documentation/SubmittingPatches, see:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/SubmittingPatches

== Patch Headers and Commit Messages ==

There seems to always be a question or two surrounding what a person
should put in a patch header, or commit message.

The general rules always apply, those being that there is a single line
short log or summary of the change (think of this as the subject when the patch
is e-mailed), and then the more detailed long log, and closure with tag lines
such as "Signed-off-by:".

== New Development ==

A minimal patch or commit message would be of the format:

Short log / Statement of what needed to be changed.

(Optional pointers to external resources, such as defect tracking)

The intent of your change.

(Optional, if it's not clear from above) how your change resolves the
issues in the first part.

Tag line(s) at the end.

For example:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The minimal commit message is good for new code development and simple changes.

The single short log message indicates what needed to be changed. It should
begin with an indicator as to the primary item changed by this work,
followed by a short summary of the change. In the above case we're indicating
that we've changed the "foobar" item, by "adjusting the foo setting in bar".

The single short log message is analogous to the git "commit summary". While no
maximum line length is specified by this policy, it is suggested that it remains
under 78 characters wherever possible.

Optionally, you may include pointers to defects this change corrects. Unless
the defect format is specified by the component you are modifying, it is
suggested that you use a full URL to specify the reference to the defect
information. Generally these pointers will precede any long description, but as
an optional item it may be after the long description.

For example, you might refer to an open source defect in the RPM project:

rpm: Adjusted the foo setting in bar

<nowiki>[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5</nowiki>

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

You must then have a full description of the change. Specifying the intent of
your change and if necessary how the change resolves the issue.

As mentioned above this is intended to answer the "what were you thinking"
questions down the road and to know what other impacts are likely to
result of the change needs to be reverted. It also allows for better
solutions if someone comes along later and agrees with paragraph 1
(the problem statement) and either disagrees with paragraph 2
(the design) or paragraph 3 (the implementation).

Finally one or more tag lines should exist. Each developer responsible
for working on the patch is responsible for adding a Signed-off-by: tag line.

It is not acceptable to have an empty or non-existent header, or just a single
line message. The summary and description is required for all changes.

== Importing from Elsewhere ==

If you are importing work from somewhere else, such as a cherry-pick from
another repository or a code patch pulled from a mailing list, the minimum patch
header or commit message is not enough. It does not clearly establish the
provenance of the code.

The following specifies the additional guidelines required for importing changes
from elsewhere.

By default you should keep the original author's summary and description,
along with any tag lines such as, "Signed-off-by:". If the original change that
was imported does not have a summary and/or commit message from the original
author, it is still your responsibility to add the summary and commit message to
the patch header. Just as if you wrote the code, you should be able to clearly
explain what the change does. It is also necessary to document the original
author of the change. You should indicate the original author by simply stating
"written by" or "posted to the ... mailing list by". You must not add a
Signed-off-by: tag line with their name, only the original author may add their
own Signed-off-by: tag line.

It is also required that the origin of the work be fully documented. The origin
should be specified as part of the commit message in a way that an average
developer would be able to track down the original code. URLs should reference
original authoritative sources and not mirrors.

If changes were required to resolve conflicts, they should be documented as
well. When incorporating a commit or patch from an external source, changes to
the functionality not related to resolving conflicts should be made in a
second commit or patch. This preserves the original external commit, and makes
the modifications clearly visible, and prevents confusion over the author of the
code.

Finally you must add a "Signed-off-by:" tag line to indicate that you worked
with the patch.

=== Example: Importing from Elsewhere Unmodified ===

For example, if you were to pull in the patch from the above example unmodified:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The patch was imported from the OpenEmbedded git server
(<nowiki>git://git.openembedded.org/openembedded</nowiki>) as of commit id
b65a0e0c84cf489bfa00d6aa6c48abc5a237100f.

Signed-off-by: Your Name <your.name@openembedded.org>

The above example shows a clear way for a developer to find the original source
of the change. It indicates that the OpenEmbedded developer simply integrated
the change into the system without making any further modifications.

=== Example: Importing from Elsewhere Modified ===

When importing a patch, occasionally changes have to be made in order to resolve
conflicts. The following is an example of the commit message when the item needed
to be modified:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The patch was imported from the OpenEmbedded git server
(<nowiki>git://git.openembedded.org/openembedded</nowiki>) as of commit id
b65a0e0c84cf489bfa00d6aa6c48abc5a237100f.

A previous change had modified the memory threshold calculation,
causing a conflict. The conflict was resolved by preserving the
memory threshold calculation from the existing implementation.

Signed-off-by: Your Name <your.name@openembedded.org>

== Patch Header Recommendations ==

In order to keep track of patches and ultimately reduce the number of patches
that are required to be maintained, we need to track the status of the patches
that are created. The following items are specific to patches applied by recipes.

In addition to the items mentioned above, we also want to track if it's
appropriate to get this particular patch into the upstream project. Since we
want to track this information, we need a consistent tag and status indicator
that can be searched.

While adding this tracking information to the patch headers is currently
optional, it is highly recommended and some maintainers may require it. It is
optional at this time so that it can be evaluated as to its usefulness over
time. Existing patches will be updated with the tag as they are modified.

As mentioned in the above, be sure to include any URL to bug tracking, mailing
lists or other reference material pertaining to the patch. Then add a new tag
"Upstream-Status:" which contains one of the following items:

'''Pending'''
- No determination has been made yet or not yet submitted to upstream

'''Submitted''' [where]
- Submitted to upstream, waiting approval
- Optionally include where it was submitted, such as the author, mailing
list, etc.

'''Accepted'''
- Accepted in upstream, expect it to be removed at next update, include
expected version info

'''Backport'''
- Backported from new upstream version, because we are at a fixed version,
include upstream version info

'''Denied'''
- Not accepted by upstream, include reason in patch

'''Inappropriate [reason]'''
- The patch is not appropriate for upstream, include a brief reason on the
same line enclosed with []
reason can be:
not author (You are not the author and do not intend to upstream this,
source must be listed in the comments)
native
licensing
configuration
enable feature
disable feature
bugfix (add bug URL here)
embedded specific
no upstream (the upstream is no longer available -- dead project)
other (give details in comments)

The various "Inappropriate [reason]" status items are meant to indicate that the
person responsible for adding this patch to the system does not intend to
upstream the patch for a specific reason. Unless otherwise noted, another
person could submit this patch to the upstream, if they do the status should be
changed to "Submitted [where]", and an additional Signed-off-by: line added to
the patch by the person claiming responsibility for upstreaming.

For example, if the patch has been submitted upstream:

rpm: Adjusted the foo setting in bar

<nowiki>[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5</nowiki>

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Upstream-Status: Submitted [rpm5-devel@rpm5.org]

Signed-off-by: Joe Developer <joe.developer@example.com>

A future commit can change the value to "Accepted" or "Denied" as appropriate.

== Common Errors in Patch and Commit Messages ==

- Short log does not start with the file or component being modified. Such as
"foo: Update to new upstream version 5.8.9"

- Avoid starting the summary line with a capital letter, unless the component
being referred to also begins with a capital letter.

- Don't have one huge patch, split your change into logical subparts. It's
easier to track down problems afterward using tools such as git bisect. It also
makes it easy for people to cherry-pick changes into things like stable branches.

- Don't simply translate your change into English for a commit log. The log
"Change compare from zero to one" is bad because it describes only the code
change in the patch; we can see that from reading the patch itself. Let the code
tell the story of the mechanics of the change (as much as possible), and let
your comment tell the other details -- i.e. what the problem was, how it
manifested itself (symptoms), and if necessary, the justification for why the
fix was done in manner that it was. In other words, the long commit message
must describe *why* the change was needed (instead of what has changed).

- Don't start your commit log with "This patch..." -- we already know it is a patch.

- Don't repeat your short log in the long log. If you really really don't have
anything new and informational to add in as a long log, then don't put a long
log at all. This should be uncommon -- i.e. the only acceptable cases for no
long log would be something like "Documentation/README: Fix spelling mistakes".

- Don't put absolute paths to source files that are specific to your site, i.e.
if you get a compile error on "fs/exec.c" then tidy up the parts of the compile
output to just show that portion. We don't need to see that it was
/home/wally/src/git/oe-core/meta/classes/insane.bbclass or similar - that full
path has no value to others. Always reduce the path to something more
meaningful, such as oe-core/meta/classes/insane.bbclass.

- Always use the most significant ramification of the change in the words of
your subject/shortlog. For example, don't say "fix compile warning in foo" when
the compiler warning was really telling us that we were dereferencing complete
garbage off in the weeds that could in theory cause an OOPS under some
circumstances. When people are choosing commits for backports to stable or
distro kernels, the shortlog will be what they use for an initial sorting
selection. If they see "Fix possible OOPS in...." then these people will look
closer, whereas they most likely will skip over simple warning fixes.

- Don't put in the full 20 or more lines of a backtrace when really it is just
the last 5 or so function calls that are relevant to the crash/fix. If the entry
point, or start of the trace is relevant (i.e. a syscall or similar), you can
leave that, and then replace a chunk of the intermediate calls in the middle
with a single [...]

- Don't include timestamps or other unnecessary information, unless they are
relevant to the situation (i.e. indicating an unacceptable delay in a driver
initialization etc.)

- Don't use links to temporary resources like pastebin and friends. The commit
message may be read long after this resource timed out.

- Don't reference mirrors, but instead always reference original authoritative
sources.

- Avoid punctuation in the short log.

[[Category:Policy]]

Adding a secondary toolchain

2012-11-14T22:32:35Z

Mhatle: /* Implementation Example */

== Background ==

In OpenEmbedded-Core there in an included toolchain that is built up of GNU binutils, gcc, and eglibc. In addition related components such as gdb, cross-prelink, mklibs and others provider additional toolchain functionality. In this document we focus on explaining best practices for including an additional toolchain that is capable of assembling, compiling and linking code. This document does not cover replacing the existing toolchain or toolchain components with external elements.

Many companies have commercial / highly optimized toolchains for specific purposes, such as ICC from Intel, LLVM, ARM Ltd toolchains, etc.. In some cases, these highly optimized toolchains may result in better performance for some applications, however they are often not generally applicable as they can not compile all of the system software. So instead of replacing the included GNU toolchain, a mechanism for providing an alternative/secondary toolchain is desired.

== Requirements ==

'''Secondary Toolchains should be implemented in a layer'''

In order to facilitate re-use and make it easier to completely disable the secondary toolchain, a layer must be used to implement the secondary toolchain.

'''Secondary Toolchain must be generally compatible with GNU toolchain'''

The secondary toolchain must be generally compatible with the idea of sysroots, linking to the defined libc, and various GNU conventions. This compatibility may be implemented directly by the secondary toolchain or via a wrapper of some type.

'''Per recipe selection of toolchain usage'''

A mechanism is needed to activate this alternative toolchain on a per-recipe basis. Using variables it should be possible to specify on a per-recipe basis which toolchain should be used, leaving the default up to the user. (It's assumed if the user does not select a default, the system GNU toolchain will be defaulted.)

'''Blacklist/Whitelist'''

Even with the requirement to be generally compatible, we can not expect all of the possible recipes to build properly with the secondary toolchain. As a consequence of this, it will be necessary to implement a blacklist (or whitelist) of specific recipes that are known to work. This way only supported components can be build by the user, avoid numerous complaints and unresolvable defects.

'''SDK'''

The SDK generated by OpenEmbedded-Core also includes the included toolchain components. It would be useful to also provide support for the secondary toolchain within the SDK environment.

== Implementation ==

=== Bitbake variables ===

A set of common bitbake variables, defined in meta/conf/bitbake.conf, are used to define how to run the toolchain components and the proper arguments.

A number of key items may need to be changed, depending on the toolchain being provided. These items may include:

* TARGET_CPPFLAGS
* TARGET_CFLAGS
* TARGET_CXXFLAGS
* TARGET_LDFLAGS
* CC
* CXX
* F77
* CPP
* LD
* CCLD
* AR
* AS
* RANLIB
* STRIP
* OBJCOPY
* OBJDUMP
* NM
* SELECTED_OPTIMIZATION
* FULL_OPTIMIZATION
* DEBUG_OPTIMIZATION

Each of the above items has a default definition within the meta/conf/bitbake.conf file. In addition, many of the components are based on specific tune variables such as:

* TUNE_CCARGS
* TUNE_LDARGS
* TUNE_ASARGS

Deciding which items need to be overridden is up to the author of the layer. It is expected that only a subset of the above referenced variables will need secondary toolchain specific versions for most implementations.

It is suggested that the secondary toolchain values be defined in a single configuration file. Within the example this is "conf/tc-example.conf". The standard name for the file should be 'conf/tc-<toolchain>.conf'.

=== Override ===

In order to allow for the layer to selectively override the variables listed above, a standard override syntax is needed.

The standard override syntax will be 'toolchain-${TOOLCHAIN}'. This will allow us to specify multiple alternative and primary toolchain combinations as necessary.

Each of the toolchain related variables can then be overridden using the standard override syntax: <var>_toolchain-<toolchain>, such as:

CC_toolchain-icc = "icc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"

In order for an individual recipe to use the secondary toolchain, the user would define: TOOLCHAIN_pn-<recipe> = 'toolchain', such as:

TOOLCHAIN_pn-bash = "icc"

The OVERRIDE is implemented in the tc-secondary.bbclass in the example.

=== Blacklist/Whitelist ===

A standard blacklist/whitelist facility is to be implemented. This facility is independent of the existing blacklist class that is part of OpenEmbedded-Core.

Similar to the existing blacklist class, you will be able to define a blacklist, per toolchain type, such as:

TCBLACKLIST[pn] = "toolchain"

or alternatively create a whitelist facility using:

TCBLACKLIST = "toolchain"
TCWHITELIST[pn] = "toolchain"

Using the second approach will allow you to blacklist all recipes, and only enable those known to work with the alternative toolchain.

The blacklist items should be defined within conf/tc-<toolchain>-blacklist.conf. The validation of the blacklist should use the standard tc-blacklist.bbclass implementation available within the example.

=== SDK ===

In order to make it easier to use the alternative toolchain within the SDK. An additional environment-* file should be created. The purpose of this file is to simply override the settings in the main environment file.

See the example section for an implementation that will create this file automatically.

== Implementation Example ==

An example implementation has been created in the layer "meta-tc-example". The file format follows:

meta-tc-example/
meta-tc-example/bin
meta-tc-example/bin/example-toolchain
meta-tc-example/conf
meta-tc-example/conf/layer.conf
meta-tc-example/conf/tc-example.conf
meta-tc-example/conf/tc-example-blacklist.conf
meta-tc-example/classes
meta-tc-example/classes/tc-secondary.bbclass
meta-tc-example/classes/tc-blacklist.bbclass

The following example is available at: [http://gate.crashing.org/~fray/yocto/meta-tc-example.tar.bz2 meta-tc-example.tar.bz2]

=== conf/layer.conf ===

# We have a conf and classes directory, append to BBPATH
BBPATH .= ":${LAYERDIR}"

BBFILE_COLLECTIONS += "tc-example"

# If we have a recipes directory, add to BBFILES
#BBFILES += "${LAYERDIR}/recipes-*/*/*.bb ${LAYERDIR}/*/recipes-*/*/*.bbappend"
#BBFILE_COLLECTIONS += "tc-example"
#BBFILE_PATTERN_tc-example := "^${LAYERDIR}/"
#BBFILE_PRIORITY_tc-example = "5"

Above is standard layer configuration. Note, the commented out sections should be re-enabled if the secondary toolchain includes recipes and/or bbappend files.

PATH := "${PATH}:${LAYERDIR}/bin"

Add to the path a standard system path, a layer specific path. This is the way you can add a secondary toolchain to the path.

# Define the alternative toolchain
SECONDARYTC = 'example'

# Enable the secondary toolchain
INHERIT += 'tc-secondary tc-blacklist'

Define the alternative toolchain name in SECONDARYTC. The inherit causes the tc-secondary.bbclass and tc-blacklist.bbclass to be loaded later. Eventually this class used SECONDARYTC to load in the secondary toolchain configuration files.

=== conf/tc-example.conf ===

# Define default system toolchain (default to built-in)
TOOLCHAIN = 'gnu'

Define a reasonable default such as gnu. The purpose of this default is to allow someone to specify the default or provide default override items for the built-in toolchain.

# Define a secondary toolchain called 'example'
# the 'example' toolchain can be activated using:
#
# Per package (preferred):
# TOOLCHAIN_pn-bash = 'example'
#
# Globally:
# TOOLCHAIN = 'example'
# TOOLCHAIN_pn-eglibc = 'gnu'

The above is an example of how to use the alternative toolchain.

# When the secondary toolchain is used, we add an automatic depend...
DEPENDS_append_toolchain-example = " virtual/TOOLCHAIN-EXAMPLE"

# ...the depend can be provided by a recipe or as an ASSUME_PROVIDED
ASSUME_PROVIDED_append = " virtual/TOOLCHAIN-EXAMPLE"

Use the above as an example on how to add additional dependencies to specific packages. This can be used to trigger a recipe to build.

# Default values from bitbake.conf...
# Variables with a '*' are things likely to need to be overridden:
#
# FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
# DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe"
# SELECTED_OPTIMIZATION = "${@d.getVar(['FULL_OPTIMIZATION', 'DEBUG_OPTIMIZATION'][d.getVar('DEBUG_BUILD', True) == '1'], True)}"
#
# * TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
#
# * TARGET_CPPFLAGS = ""
# * TARGET_CFLAGS = "${TARGET_CPPFLAGS} ${SELECTED_OPTIMIZATION}"
# * TARGET CXXFLAGS = "${TARGET_CFLAGS} -fpermissive"
# * TARGET_LDFLAGS = "-Wl,-O1 ${TARGET_LINK_HASH_STYLE}"
#
# TARGET_CC_ARCH = "${TUNE_CCARGS}"
# TARGET_LD_ARCH = "${TUNE_LDARGS}"
# TARGET_AS_ARCH = "${TUNE_ASARGS}"
#
# HOST_CC_ARCH = "${TARGET_CC_ARCH}"
# HOST_LD_ARCH = "${TARGET_LD_ARCH}"
# HOST_AS_ARCH = "${TARGET_AS_ARCH}"
#
# * CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CXX = "${CCACHE}${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * F77 = "${CCACHE}${HOST_PREFIX}g77 ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CPP = "${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
# * LD = "${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"
# * CCLD = "${CC}"
# AR = "${HOST_PREFIX}ar"
# AS = "${HOST_PREFIX}as ${HOST_AS_ARCH}"
# RANLIB = "${HOST_PREFIX}ranlib"
# STRIP = "${HOST_PREFIX}strip"
# OBJCOPY = "${HOST_PREFIX}objcopy"
# OBJDUMP = "${HOST_PREFIX}objdump"
# NM = "${HOST_PREFIX}nm"

# Override values:
CC_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CXX_toolchain-example = "example-toolchain ${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CPP_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
LD_toolchain-example = "example-toolchain ${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"

The above implements the necessary overrides for the example toolchain.

=== conf/tc-example-blacklist.conf ===

Example configuration file that enables the blacklist and defines the defaults.

# Blacklist certain items...
TCBLACKLIST[eglibc] = 'example'
TCBLACKLIST[bash] = 'example'

#
# or
#

# Switch to a global blacklist and whitelist situation...
# TCBLACKLIST = 'example'
# TCWHITELIST[bash] = 'example'

=== classes/tc-secondary.bbclass ===

This is the standard secondary toolchain class. It should be the same in all secondary toolchain layers.

# In order to add overrides, we need to do it later then in the layers.conf
# the only standard way is using an inherit, which requires a bbclass

# Add the necessary override
TOOLCHAINOVERRIDES = ":toolchain-${TOOLCHAIN}"
TOOLCHAINOVERRIDES[vardepsexclude] = "TOOLCHAIN"

OVERRIDES .= "${TOOLCHAINOVERRIDES}"
OVERRIDES[vardepsexclude] += "TOOLCHAINOVERRIDES"

require conf/tc-${SECONDARYTC}.conf

The above code configures the basic overrides and loads the configuration file. There is one limitation to the above however. It will only load one secondary toolchain. A simple way around this is to rename the inherit and change to using something other then SECONDARYTC. This likely will not be a problem.

toolchain_create_sdk_env_script_alt () {
set -x

# Create environment setup script (secondary toolchain)
orig_script=${1:-${SDK_OUTPUT}/${SDKPATH}/environment-setup-${REAL_MULTIMACH_TARGET_SYS}}
script="$orig_script""-${SECONDARYTC}"
rm -f $script
touch $script
echo ". $orig_script" | sed -e "s:${SDK_OUTPUT}::" >> $script
if [ '${CC_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CC_toolchain-${SECONDARYTC}}' ]; then
echo 'export CC="${CC_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CXX_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CXX_toolchain-${SECONDARYTC}}' ]; then
echo 'export CXX="${CXX_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CPP_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CPP_toolchain-${SECONDARYTC}}' ]; then
echo 'export CPP="${CPP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${AS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>AS_toolchain-${SECONDARYTC}}' ]; then
echo 'export AS="${AS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${LD_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>LD_toolchain-${SECONDARYTC}}' ]; then
echo 'export LD="${LD_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${RANLIB_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>RANLIB_toolchain-${SECONDARYTC}}' ]; then
echo 'export RANLIB="${RANLIB_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${STRIP_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>STRIP_toolchain-${SECONDARYTC}}' ]; then
echo 'export STRIP="${STRIP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${OBJCOPY_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>OBJCOPY_toolchain-${SECONDARYTC}}' ]; then
echo 'export OBJCOPY="${OBJCOPY_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${OBJDUMP_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>OBJDUMP_toolchain-${SECONDARYTC}}' ]; then
echo 'export OBJDUMP="${OBJDUMP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${NM_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>NM_toolchain-${SECONDARYTC}}' ]; then
echo 'export NM="${NM_toolchain-${SECONDARYTC}}"' >> $script
fi

if [ '${CFLAGS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CFLAGS="${CFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CXXFLAGS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CXXFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CXXFLAGS="${CXXFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${LDFLAGS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>LDFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export LDFLAGS="${LDFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CPPFLAGS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CPPFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CPPFLAGS="${CPPFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi

# Fixup any sysroot references
sed -i -e "s:${STAGING_DIR_TARGET}:${SDKTARGETSYSROOT}:g" $script

# All environment scripts require OECORE_NATIVE_SYSROOT
echo 'export OECORE_NATIVE_SYSROOT="${SDKPATHNATIVE}"' >> $script
}

populate_sdk_image_append() {
toolchain_create_sdk_env_script_alt ${SDK_OUTPUT}/${SDKPATH}/environment-setup-${REAL_MULTIMACH_TARGET_SYS}
}

The above code will add an additional environment-setup-*-<toolchain> file. This can be used to enable the right environment to use the alternative toolchain. Note, it does not add anything to the path so the user will still need to configure the path to the alternative toolchain. (The layer creator can always provide the necessary 'nativesdk' or similar package that provides the alternative toolchain as part of the SDK as well.)

=== classes/tc-blacklist.bbclass ===

This is the standard secondary toolchain blacklist class. It should be the same in all secondary toolchain layers.

# Implement blacklist/whitelist functionality for alternative toolchains
# Based on the oe-core blacklist bclass
#
# To add a blacklisted package:
# TCBLACKLIST[pn] = 'toolchain'
#
# To blacklist everything, and whitelist a package:
# TCBLACKLIST = 'toolchain'
# TCWHITELIST[pn] = 'toolchain'

include conf/tc-${SECONDARYTC}-blacklist.conf

python () {
tc = d.getVar('TOOLCHAIN', True)
pn = d.getVar('PN', True)

blacklist = d.getVarFlag('TCBLACKLIST', pn, True)

if blacklist and blacklist == tc:
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))

blacklist = d.getVar('TCBLACKLIST', True)
whitelist = d.getVarFlag('TCWHITELIST', pn, True)

if blacklist and (not whitelist or whitelist != tc):
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))
}

== Further comments and information ==

[[Category:Dev]]
[[Category:Layers]]

Adding a secondary toolchain

2012-11-14T22:29:19Z

Mhatle: /* classes/tc-secondary.bbclass */

== Background ==

In OpenEmbedded-Core there in an included toolchain that is built up of GNU binutils, gcc, and eglibc. In addition related components such as gdb, cross-prelink, mklibs and others provider additional toolchain functionality. In this document we focus on explaining best practices for including an additional toolchain that is capable of assembling, compiling and linking code. This document does not cover replacing the existing toolchain or toolchain components with external elements.

Many companies have commercial / highly optimized toolchains for specific purposes, such as ICC from Intel, LLVM, ARM Ltd toolchains, etc.. In some cases, these highly optimized toolchains may result in better performance for some applications, however they are often not generally applicable as they can not compile all of the system software. So instead of replacing the included GNU toolchain, a mechanism for providing an alternative/secondary toolchain is desired.

== Requirements ==

'''Secondary Toolchains should be implemented in a layer'''

In order to facilitate re-use and make it easier to completely disable the secondary toolchain, a layer must be used to implement the secondary toolchain.

'''Secondary Toolchain must be generally compatible with GNU toolchain'''

The secondary toolchain must be generally compatible with the idea of sysroots, linking to the defined libc, and various GNU conventions. This compatibility may be implemented directly by the secondary toolchain or via a wrapper of some type.

'''Per recipe selection of toolchain usage'''

A mechanism is needed to activate this alternative toolchain on a per-recipe basis. Using variables it should be possible to specify on a per-recipe basis which toolchain should be used, leaving the default up to the user. (It's assumed if the user does not select a default, the system GNU toolchain will be defaulted.)

'''Blacklist/Whitelist'''

Even with the requirement to be generally compatible, we can not expect all of the possible recipes to build properly with the secondary toolchain. As a consequence of this, it will be necessary to implement a blacklist (or whitelist) of specific recipes that are known to work. This way only supported components can be build by the user, avoid numerous complaints and unresolvable defects.

'''SDK'''

The SDK generated by OpenEmbedded-Core also includes the included toolchain components. It would be useful to also provide support for the secondary toolchain within the SDK environment.

== Implementation ==

=== Bitbake variables ===

A set of common bitbake variables, defined in meta/conf/bitbake.conf, are used to define how to run the toolchain components and the proper arguments.

A number of key items may need to be changed, depending on the toolchain being provided. These items may include:

* TARGET_CPPFLAGS
* TARGET_CFLAGS
* TARGET_CXXFLAGS
* TARGET_LDFLAGS
* CC
* CXX
* F77
* CPP
* LD
* CCLD
* AR
* AS
* RANLIB
* STRIP
* OBJCOPY
* OBJDUMP
* NM
* SELECTED_OPTIMIZATION
* FULL_OPTIMIZATION
* DEBUG_OPTIMIZATION

Each of the above items has a default definition within the meta/conf/bitbake.conf file. In addition, many of the components are based on specific tune variables such as:

* TUNE_CCARGS
* TUNE_LDARGS
* TUNE_ASARGS

Deciding which items need to be overridden is up to the author of the layer. It is expected that only a subset of the above referenced variables will need secondary toolchain specific versions for most implementations.

It is suggested that the secondary toolchain values be defined in a single configuration file. Within the example this is "conf/tc-example.conf". The standard name for the file should be 'conf/tc-<toolchain>.conf'.

=== Override ===

In order to allow for the layer to selectively override the variables listed above, a standard override syntax is needed.

The standard override syntax will be 'toolchain-${TOOLCHAIN}'. This will allow us to specify multiple alternative and primary toolchain combinations as necessary.

Each of the toolchain related variables can then be overridden using the standard override syntax: <var>_toolchain-<toolchain>, such as:

CC_toolchain-icc = "icc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"

In order for an individual recipe to use the secondary toolchain, the user would define: TOOLCHAIN_pn-<recipe> = 'toolchain', such as:

TOOLCHAIN_pn-bash = "icc"

The OVERRIDE is implemented in the tc-secondary.bbclass in the example.

=== Blacklist/Whitelist ===

A standard blacklist/whitelist facility is to be implemented. This facility is independent of the existing blacklist class that is part of OpenEmbedded-Core.

Similar to the existing blacklist class, you will be able to define a blacklist, per toolchain type, such as:

TCBLACKLIST[pn] = "toolchain"

or alternatively create a whitelist facility using:

TCBLACKLIST = "toolchain"
TCWHITELIST[pn] = "toolchain"

Using the second approach will allow you to blacklist all recipes, and only enable those known to work with the alternative toolchain.

The blacklist items should be defined within conf/tc-<toolchain>-blacklist.conf. The validation of the blacklist should use the standard tc-blacklist.bbclass implementation available within the example.

=== SDK ===

In order to make it easier to use the alternative toolchain within the SDK. An additional environment-* file should be created. The purpose of this file is to simply override the settings in the main environment file.

See the example section for an implementation that will create this file automatically.

== Implementation Example ==

An example implementation has been created in the layer "meta-tc-example". The file format follows:

meta-tc-example/
meta-tc-example/bin
meta-tc-example/bin/example-toolchain
meta-tc-example/conf
meta-tc-example/conf/layer.conf
meta-tc-example/conf/tc-example.conf
meta-tc-example/conf/tc-example-blacklist.conf
meta-tc-example/classes
meta-tc-example/classes/tc-secondary.bbclass
meta-tc-example/classes/tc-blacklist.bbclass

=== conf/layer.conf ===

# We have a conf and classes directory, append to BBPATH
BBPATH .= ":${LAYERDIR}"

BBFILE_COLLECTIONS += "tc-example"

# If we have a recipes directory, add to BBFILES
#BBFILES += "${LAYERDIR}/recipes-*/*/*.bb ${LAYERDIR}/*/recipes-*/*/*.bbappend"
#BBFILE_COLLECTIONS += "tc-example"
#BBFILE_PATTERN_tc-example := "^${LAYERDIR}/"
#BBFILE_PRIORITY_tc-example = "5"

Above is standard layer configuration. Note, the commented out sections should be re-enabled if the secondary toolchain includes recipes and/or bbappend files.

PATH := "${PATH}:${LAYERDIR}/bin"

Add to the path a standard system path, a layer specific path. This is the way you can add a secondary toolchain to the path.

# Define the alternative toolchain
SECONDARYTC = 'example'

# Enable the secondary toolchain
INHERIT += 'tc-secondary tc-blacklist'

Define the alternative toolchain name in SECONDARYTC. The inherit causes the tc-secondary.bbclass and tc-blacklist.bbclass to be loaded later. Eventually this class used SECONDARYTC to load in the secondary toolchain configuration files.

=== conf/tc-example.conf ===

# Define default system toolchain (default to built-in)
TOOLCHAIN = 'gnu'

Define a reasonable default such as gnu. The purpose of this default is to allow someone to specify the default or provide default override items for the built-in toolchain.

# Define a secondary toolchain called 'example'
# the 'example' toolchain can be activated using:
#
# Per package (preferred):
# TOOLCHAIN_pn-bash = 'example'
#
# Globally:
# TOOLCHAIN = 'example'
# TOOLCHAIN_pn-eglibc = 'gnu'

The above is an example of how to use the alternative toolchain.

# When the secondary toolchain is used, we add an automatic depend...
DEPENDS_append_toolchain-example = " virtual/TOOLCHAIN-EXAMPLE"

# ...the depend can be provided by a recipe or as an ASSUME_PROVIDED
ASSUME_PROVIDED_append = " virtual/TOOLCHAIN-EXAMPLE"

Use the above as an example on how to add additional dependencies to specific packages. This can be used to trigger a recipe to build.

# Default values from bitbake.conf...
# Variables with a '*' are things likely to need to be overridden:
#
# FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
# DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe"
# SELECTED_OPTIMIZATION = "${@d.getVar(['FULL_OPTIMIZATION', 'DEBUG_OPTIMIZATION'][d.getVar('DEBUG_BUILD', True) == '1'], True)}"
#
# * TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
#
# * TARGET_CPPFLAGS = ""
# * TARGET_CFLAGS = "${TARGET_CPPFLAGS} ${SELECTED_OPTIMIZATION}"
# * TARGET CXXFLAGS = "${TARGET_CFLAGS} -fpermissive"
# * TARGET_LDFLAGS = "-Wl,-O1 ${TARGET_LINK_HASH_STYLE}"
#
# TARGET_CC_ARCH = "${TUNE_CCARGS}"
# TARGET_LD_ARCH = "${TUNE_LDARGS}"
# TARGET_AS_ARCH = "${TUNE_ASARGS}"
#
# HOST_CC_ARCH = "${TARGET_CC_ARCH}"
# HOST_LD_ARCH = "${TARGET_LD_ARCH}"
# HOST_AS_ARCH = "${TARGET_AS_ARCH}"
#
# * CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CXX = "${CCACHE}${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * F77 = "${CCACHE}${HOST_PREFIX}g77 ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CPP = "${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
# * LD = "${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"
# * CCLD = "${CC}"
# AR = "${HOST_PREFIX}ar"
# AS = "${HOST_PREFIX}as ${HOST_AS_ARCH}"
# RANLIB = "${HOST_PREFIX}ranlib"
# STRIP = "${HOST_PREFIX}strip"
# OBJCOPY = "${HOST_PREFIX}objcopy"
# OBJDUMP = "${HOST_PREFIX}objdump"
# NM = "${HOST_PREFIX}nm"

# Override values:
CC_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CXX_toolchain-example = "example-toolchain ${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CPP_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
LD_toolchain-example = "example-toolchain ${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"

The above implements the necessary overrides for the example toolchain.

=== conf/tc-example-blacklist.conf ===

Example configuration file that enables the blacklist and defines the defaults.

# Blacklist certain items...
TCBLACKLIST[eglibc] = 'example'
TCBLACKLIST[bash] = 'example'

#
# or
#

# Switch to a global blacklist and whitelist situation...
# TCBLACKLIST = 'example'
# TCWHITELIST[bash] = 'example'

=== classes/tc-secondary.bbclass ===

This is the standard secondary toolchain class. It should be the same in all secondary toolchain layers.

# In order to add overrides, we need to do it later then in the layers.conf
# the only standard way is using an inherit, which requires a bbclass

# Add the necessary override
TOOLCHAINOVERRIDES = ":toolchain-${TOOLCHAIN}"
TOOLCHAINOVERRIDES[vardepsexclude] = "TOOLCHAIN"

OVERRIDES .= "${TOOLCHAINOVERRIDES}"
OVERRIDES[vardepsexclude] += "TOOLCHAINOVERRIDES"

require conf/tc-${SECONDARYTC}.conf

The above code configures the basic overrides and loads the configuration file. There is one limitation to the above however. It will only load one secondary toolchain. A simple way around this is to rename the inherit and change to using something other then SECONDARYTC. This likely will not be a problem.

toolchain_create_sdk_env_script_alt () {
set -x

# Create environment setup script (secondary toolchain)
orig_script=${1:-${SDK_OUTPUT}/${SDKPATH}/environment-setup-${REAL_MULTIMACH_TARGET_SYS}}
script="$orig_script""-${SECONDARYTC}"
rm -f $script
touch $script
echo ". $orig_script" | sed -e "s:${SDK_OUTPUT}::" >> $script
if [ '${CC_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CC_toolchain-${SECONDARYTC}}' ]; then
echo 'export CC="${CC_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CXX_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CXX_toolchain-${SECONDARYTC}}' ]; then
echo 'export CXX="${CXX_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CPP_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CPP_toolchain-${SECONDARYTC}}' ]; then
echo 'export CPP="${CPP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${AS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>AS_toolchain-${SECONDARYTC}}' ]; then
echo 'export AS="${AS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${LD_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>LD_toolchain-${SECONDARYTC}}' ]; then
echo 'export LD="${LD_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${RANLIB_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>RANLIB_toolchain-${SECONDARYTC}}' ]; then
echo 'export RANLIB="${RANLIB_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${STRIP_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>STRIP_toolchain-${SECONDARYTC}}' ]; then
echo 'export STRIP="${STRIP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${OBJCOPY_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>OBJCOPY_toolchain-${SECONDARYTC}}' ]; then
echo 'export OBJCOPY="${OBJCOPY_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${OBJDUMP_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>OBJDUMP_toolchain-${SECONDARYTC}}' ]; then
echo 'export OBJDUMP="${OBJDUMP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${NM_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>NM_toolchain-${SECONDARYTC}}' ]; then
echo 'export NM="${NM_toolchain-${SECONDARYTC}}"' >> $script
fi

if [ '${CFLAGS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CFLAGS="${CFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CXXFLAGS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CXXFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CXXFLAGS="${CXXFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${LDFLAGS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>LDFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export LDFLAGS="${LDFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CPPFLAGS_toolchain-${SECONDARYTC}}' != '${<nowiki>''</nowiki>CPPFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CPPFLAGS="${CPPFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi

# Fixup any sysroot references
sed -i -e "s:${STAGING_DIR_TARGET}:${SDKTARGETSYSROOT}:g" $script

# All environment scripts require OECORE_NATIVE_SYSROOT
echo 'export OECORE_NATIVE_SYSROOT="${SDKPATHNATIVE}"' >> $script
}

populate_sdk_image_append() {
toolchain_create_sdk_env_script_alt ${SDK_OUTPUT}/${SDKPATH}/environment-setup-${REAL_MULTIMACH_TARGET_SYS}
}

The above code will add an additional environment-setup-*-<toolchain> file. This can be used to enable the right environment to use the alternative toolchain. Note, it does not add anything to the path so the user will still need to configure the path to the alternative toolchain. (The layer creator can always provide the necessary 'nativesdk' or similar package that provides the alternative toolchain as part of the SDK as well.)

=== classes/tc-blacklist.bbclass ===

This is the standard secondary toolchain blacklist class. It should be the same in all secondary toolchain layers.

# Implement blacklist/whitelist functionality for alternative toolchains
# Based on the oe-core blacklist bclass
#
# To add a blacklisted package:
# TCBLACKLIST[pn] = 'toolchain'
#
# To blacklist everything, and whitelist a package:
# TCBLACKLIST = 'toolchain'
# TCWHITELIST[pn] = 'toolchain'

include conf/tc-${SECONDARYTC}-blacklist.conf

python () {
tc = d.getVar('TOOLCHAIN', True)
pn = d.getVar('PN', True)

blacklist = d.getVarFlag('TCBLACKLIST', pn, True)

if blacklist and blacklist == tc:
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))

blacklist = d.getVar('TCBLACKLIST', True)
whitelist = d.getVarFlag('TCWHITELIST', pn, True)

if blacklist and (not whitelist or whitelist != tc):
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))
}

== Further comments and information ==

[[Category:Dev]]
[[Category:Layers]]

Adding a secondary toolchain

2012-11-14T22:25:18Z

Mhatle: /* classes/tc-secondary.bbclass */

== Background ==

In OpenEmbedded-Core there in an included toolchain that is built up of GNU binutils, gcc, and eglibc. In addition related components such as gdb, cross-prelink, mklibs and others provider additional toolchain functionality. In this document we focus on explaining best practices for including an additional toolchain that is capable of assembling, compiling and linking code. This document does not cover replacing the existing toolchain or toolchain components with external elements.

Many companies have commercial / highly optimized toolchains for specific purposes, such as ICC from Intel, LLVM, ARM Ltd toolchains, etc.. In some cases, these highly optimized toolchains may result in better performance for some applications, however they are often not generally applicable as they can not compile all of the system software. So instead of replacing the included GNU toolchain, a mechanism for providing an alternative/secondary toolchain is desired.

== Requirements ==

'''Secondary Toolchains should be implemented in a layer'''

In order to facilitate re-use and make it easier to completely disable the secondary toolchain, a layer must be used to implement the secondary toolchain.

'''Secondary Toolchain must be generally compatible with GNU toolchain'''

The secondary toolchain must be generally compatible with the idea of sysroots, linking to the defined libc, and various GNU conventions. This compatibility may be implemented directly by the secondary toolchain or via a wrapper of some type.

'''Per recipe selection of toolchain usage'''

A mechanism is needed to activate this alternative toolchain on a per-recipe basis. Using variables it should be possible to specify on a per-recipe basis which toolchain should be used, leaving the default up to the user. (It's assumed if the user does not select a default, the system GNU toolchain will be defaulted.)

'''Blacklist/Whitelist'''

Even with the requirement to be generally compatible, we can not expect all of the possible recipes to build properly with the secondary toolchain. As a consequence of this, it will be necessary to implement a blacklist (or whitelist) of specific recipes that are known to work. This way only supported components can be build by the user, avoid numerous complaints and unresolvable defects.

'''SDK'''

The SDK generated by OpenEmbedded-Core also includes the included toolchain components. It would be useful to also provide support for the secondary toolchain within the SDK environment.

== Implementation ==

=== Bitbake variables ===

A set of common bitbake variables, defined in meta/conf/bitbake.conf, are used to define how to run the toolchain components and the proper arguments.

A number of key items may need to be changed, depending on the toolchain being provided. These items may include:

* TARGET_CPPFLAGS
* TARGET_CFLAGS
* TARGET_CXXFLAGS
* TARGET_LDFLAGS
* CC
* CXX
* F77
* CPP
* LD
* CCLD
* AR
* AS
* RANLIB
* STRIP
* OBJCOPY
* OBJDUMP
* NM
* SELECTED_OPTIMIZATION
* FULL_OPTIMIZATION
* DEBUG_OPTIMIZATION

Each of the above items has a default definition within the meta/conf/bitbake.conf file. In addition, many of the components are based on specific tune variables such as:

* TUNE_CCARGS
* TUNE_LDARGS
* TUNE_ASARGS

Deciding which items need to be overridden is up to the author of the layer. It is expected that only a subset of the above referenced variables will need secondary toolchain specific versions for most implementations.

It is suggested that the secondary toolchain values be defined in a single configuration file. Within the example this is "conf/tc-example.conf". The standard name for the file should be 'conf/tc-<toolchain>.conf'.

=== Override ===

In order to allow for the layer to selectively override the variables listed above, a standard override syntax is needed.

The standard override syntax will be 'toolchain-${TOOLCHAIN}'. This will allow us to specify multiple alternative and primary toolchain combinations as necessary.

Each of the toolchain related variables can then be overridden using the standard override syntax: <var>_toolchain-<toolchain>, such as:

CC_toolchain-icc = "icc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"

In order for an individual recipe to use the secondary toolchain, the user would define: TOOLCHAIN_pn-<recipe> = 'toolchain', such as:

TOOLCHAIN_pn-bash = "icc"

The OVERRIDE is implemented in the tc-secondary.bbclass in the example.

=== Blacklist/Whitelist ===

A standard blacklist/whitelist facility is to be implemented. This facility is independent of the existing blacklist class that is part of OpenEmbedded-Core.

Similar to the existing blacklist class, you will be able to define a blacklist, per toolchain type, such as:

TCBLACKLIST[pn] = "toolchain"

or alternatively create a whitelist facility using:

TCBLACKLIST = "toolchain"
TCWHITELIST[pn] = "toolchain"

Using the second approach will allow you to blacklist all recipes, and only enable those known to work with the alternative toolchain.

The blacklist items should be defined within conf/tc-<toolchain>-blacklist.conf. The validation of the blacklist should use the standard tc-blacklist.bbclass implementation available within the example.

=== SDK ===

In order to make it easier to use the alternative toolchain within the SDK. An additional environment-* file should be created. The purpose of this file is to simply override the settings in the main environment file.

See the example section for an implementation that will create this file automatically.

== Implementation Example ==

An example implementation has been created in the layer "meta-tc-example". The file format follows:

meta-tc-example/
meta-tc-example/bin
meta-tc-example/bin/example-toolchain
meta-tc-example/conf
meta-tc-example/conf/layer.conf
meta-tc-example/conf/tc-example.conf
meta-tc-example/conf/tc-example-blacklist.conf
meta-tc-example/classes
meta-tc-example/classes/tc-secondary.bbclass
meta-tc-example/classes/tc-blacklist.bbclass

=== conf/layer.conf ===

# We have a conf and classes directory, append to BBPATH
BBPATH .= ":${LAYERDIR}"

BBFILE_COLLECTIONS += "tc-example"

# If we have a recipes directory, add to BBFILES
#BBFILES += "${LAYERDIR}/recipes-*/*/*.bb ${LAYERDIR}/*/recipes-*/*/*.bbappend"
#BBFILE_COLLECTIONS += "tc-example"
#BBFILE_PATTERN_tc-example := "^${LAYERDIR}/"
#BBFILE_PRIORITY_tc-example = "5"

Above is standard layer configuration. Note, the commented out sections should be re-enabled if the secondary toolchain includes recipes and/or bbappend files.

PATH := "${PATH}:${LAYERDIR}/bin"

Add to the path a standard system path, a layer specific path. This is the way you can add a secondary toolchain to the path.

# Define the alternative toolchain
SECONDARYTC = 'example'

# Enable the secondary toolchain
INHERIT += 'tc-secondary tc-blacklist'

Define the alternative toolchain name in SECONDARYTC. The inherit causes the tc-secondary.bbclass and tc-blacklist.bbclass to be loaded later. Eventually this class used SECONDARYTC to load in the secondary toolchain configuration files.

=== conf/tc-example.conf ===

# Define default system toolchain (default to built-in)
TOOLCHAIN = 'gnu'

Define a reasonable default such as gnu. The purpose of this default is to allow someone to specify the default or provide default override items for the built-in toolchain.

# Define a secondary toolchain called 'example'
# the 'example' toolchain can be activated using:
#
# Per package (preferred):
# TOOLCHAIN_pn-bash = 'example'
#
# Globally:
# TOOLCHAIN = 'example'
# TOOLCHAIN_pn-eglibc = 'gnu'

The above is an example of how to use the alternative toolchain.

# When the secondary toolchain is used, we add an automatic depend...
DEPENDS_append_toolchain-example = " virtual/TOOLCHAIN-EXAMPLE"

# ...the depend can be provided by a recipe or as an ASSUME_PROVIDED
ASSUME_PROVIDED_append = " virtual/TOOLCHAIN-EXAMPLE"

Use the above as an example on how to add additional dependencies to specific packages. This can be used to trigger a recipe to build.

# Default values from bitbake.conf...
# Variables with a '*' are things likely to need to be overridden:
#
# FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
# DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe"
# SELECTED_OPTIMIZATION = "${@d.getVar(['FULL_OPTIMIZATION', 'DEBUG_OPTIMIZATION'][d.getVar('DEBUG_BUILD', True) == '1'], True)}"
#
# * TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
#
# * TARGET_CPPFLAGS = ""
# * TARGET_CFLAGS = "${TARGET_CPPFLAGS} ${SELECTED_OPTIMIZATION}"
# * TARGET CXXFLAGS = "${TARGET_CFLAGS} -fpermissive"
# * TARGET_LDFLAGS = "-Wl,-O1 ${TARGET_LINK_HASH_STYLE}"
#
# TARGET_CC_ARCH = "${TUNE_CCARGS}"
# TARGET_LD_ARCH = "${TUNE_LDARGS}"
# TARGET_AS_ARCH = "${TUNE_ASARGS}"
#
# HOST_CC_ARCH = "${TARGET_CC_ARCH}"
# HOST_LD_ARCH = "${TARGET_LD_ARCH}"
# HOST_AS_ARCH = "${TARGET_AS_ARCH}"
#
# * CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CXX = "${CCACHE}${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * F77 = "${CCACHE}${HOST_PREFIX}g77 ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CPP = "${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
# * LD = "${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"
# * CCLD = "${CC}"
# AR = "${HOST_PREFIX}ar"
# AS = "${HOST_PREFIX}as ${HOST_AS_ARCH}"
# RANLIB = "${HOST_PREFIX}ranlib"
# STRIP = "${HOST_PREFIX}strip"
# OBJCOPY = "${HOST_PREFIX}objcopy"
# OBJDUMP = "${HOST_PREFIX}objdump"
# NM = "${HOST_PREFIX}nm"

# Override values:
CC_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CXX_toolchain-example = "example-toolchain ${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CPP_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
LD_toolchain-example = "example-toolchain ${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"

The above implements the necessary overrides for the example toolchain.

=== conf/tc-example-blacklist.conf ===

Example configuration file that enables the blacklist and defines the defaults.

# Blacklist certain items...
TCBLACKLIST[eglibc] = 'example'
TCBLACKLIST[bash] = 'example'

#
# or
#

# Switch to a global blacklist and whitelist situation...
# TCBLACKLIST = 'example'
# TCWHITELIST[bash] = 'example'

=== classes/tc-secondary.bbclass ===

This is the standard secondary toolchain class. It should be the same in all secondary toolchain layers.

# In order to add overrides, we need to do it later then in the layers.conf
# the only standard way is using an inherit, which requires a bbclass

# Add the necessary override
TOOLCHAINOVERRIDES = ":toolchain-${TOOLCHAIN}"
TOOLCHAINOVERRIDES[vardepsexclude] = "TOOLCHAIN"

OVERRIDES .= "${TOOLCHAINOVERRIDES}"
OVERRIDES[vardepsexclude] += "TOOLCHAINOVERRIDES"

require conf/tc-${SECONDARYTC}.conf

The above code configures the basic overrides and loads the configuration file. There is one limitation to the above however. It will only load one secondary toolchain. A simple way around this is to rename the inherit and change to using something other then SECONDARYTC. This likely will not be a problem.

toolchain_create_sdk_env_script_alt () {
set -x

# Create environment setup script (secondary toolchain)
orig_script=${1:-${SDK_OUTPUT}/${SDKPATH}/environment-setup-${REAL_MULTIMACH_TARGET_SYS}}
script="$orig_script""-${SECONDARYTC}"
rm -f $script
touch $script
echo ". $orig_script" | sed -e "s:${SDK_OUTPUT}::" >> $script
if [ '${CC_toolchain-${SECONDARYTC}}' != '${''CC_toolchain-${SECONDARYTC}}' ]; then
echo 'export CC="${CC_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CXX_toolchain-${SECONDARYTC}}' != '${''CXX_toolchain-${SECONDARYTC}}' ]; then
echo 'export CXX="${CXX_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CPP_toolchain-${SECONDARYTC}}' != '${''CPP_toolchain-${SECONDARYTC}}' ]; then
echo 'export CPP="${CPP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${AS_toolchain-${SECONDARYTC}}' != '${''AS_toolchain-${SECONDARYTC}}' ]; then
echo 'export AS="${AS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${LD_toolchain-${SECONDARYTC}}' != '${''LD_toolchain-${SECONDARYTC}}' ]; then
echo 'export LD="${LD_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${RANLIB_toolchain-${SECONDARYTC}}' != '${''RANLIB_toolchain-${SECONDARYTC}}' ]; then
echo 'export RANLIB="${RANLIB_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${STRIP_toolchain-${SECONDARYTC}}' != '${''STRIP_toolchain-${SECONDARYTC}}' ]; then
echo 'export STRIP="${STRIP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${OBJCOPY_toolchain-${SECONDARYTC}}' != '${''OBJCOPY_toolchain-${SECONDARYTC}}' ]; then
echo 'export OBJCOPY="${OBJCOPY_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${OBJDUMP_toolchain-${SECONDARYTC}}' != '${''OBJDUMP_toolchain-${SECONDARYTC}}' ]; then
echo 'export OBJDUMP="${OBJDUMP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${NM_toolchain-${SECONDARYTC}}' != '${''NM_toolchain-${SECONDARYTC}}' ]; then
echo 'export NM="${NM_toolchain-${SECONDARYTC}}"' >> $script
fi

if [ '${CFLAGS_toolchain-${SECONDARYTC}}' != '${''CFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CFLAGS="${CFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CXXFLAGS_toolchain-${SECONDARYTC}}' != '${''CXXFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CXXFLAGS="${CXXFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${LDFLAGS_toolchain-${SECONDARYTC}}' != '${''LDFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export LDFLAGS="${LDFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CPPFLAGS_toolchain-${SECONDARYTC}}' != '${''CPPFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CPPFLAGS="${CPPFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi

# Fixup any sysroot references
sed -i -e "s:${STAGING_DIR_TARGET}:${SDKTARGETSYSROOT}:g" $script

# All environment scripts require OECORE_NATIVE_SYSROOT
echo 'export OECORE_NATIVE_SYSROOT="${SDKPATHNATIVE}"' >> $script
}

populate_sdk_image_append() {
toolchain_create_sdk_env_script_alt ${SDK_OUTPUT}/${SDKPATH}/environment-setup-${REAL_MULTIMACH_TARGET_SYS}
}

The above code will add an additional environment-setup-*-<toolchain> file. This can be used to enable the right environment to use the alternative toolchain. Note, it does not add anything to the path so the user will still need to configure the path to the alternative toolchain. (The layer creator can always provide the necessary 'nativesdk' or similar package that provides the alternative toolchain as part of the SDK as well.)

=== classes/tc-blacklist.bbclass ===

This is the standard secondary toolchain blacklist class. It should be the same in all secondary toolchain layers.

# Implement blacklist/whitelist functionality for alternative toolchains
# Based on the oe-core blacklist bclass
#
# To add a blacklisted package:
# TCBLACKLIST[pn] = 'toolchain'
#
# To blacklist everything, and whitelist a package:
# TCBLACKLIST = 'toolchain'
# TCWHITELIST[pn] = 'toolchain'

include conf/tc-${SECONDARYTC}-blacklist.conf

python () {
tc = d.getVar('TOOLCHAIN', True)
pn = d.getVar('PN', True)

blacklist = d.getVarFlag('TCBLACKLIST', pn, True)

if blacklist and blacklist == tc:
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))

blacklist = d.getVar('TCBLACKLIST', True)
whitelist = d.getVarFlag('TCWHITELIST', pn, True)

if blacklist and (not whitelist or whitelist != tc):
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))
}

== Further comments and information ==

[[Category:Dev]]
[[Category:Layers]]

Adding a secondary toolchain

2012-11-14T22:24:58Z

Mhatle: /* classes/tc-secondary.bbclass */

== Background ==

In OpenEmbedded-Core there in an included toolchain that is built up of GNU binutils, gcc, and eglibc. In addition related components such as gdb, cross-prelink, mklibs and others provider additional toolchain functionality. In this document we focus on explaining best practices for including an additional toolchain that is capable of assembling, compiling and linking code. This document does not cover replacing the existing toolchain or toolchain components with external elements.

Many companies have commercial / highly optimized toolchains for specific purposes, such as ICC from Intel, LLVM, ARM Ltd toolchains, etc.. In some cases, these highly optimized toolchains may result in better performance for some applications, however they are often not generally applicable as they can not compile all of the system software. So instead of replacing the included GNU toolchain, a mechanism for providing an alternative/secondary toolchain is desired.

== Requirements ==

'''Secondary Toolchains should be implemented in a layer'''

In order to facilitate re-use and make it easier to completely disable the secondary toolchain, a layer must be used to implement the secondary toolchain.

'''Secondary Toolchain must be generally compatible with GNU toolchain'''

The secondary toolchain must be generally compatible with the idea of sysroots, linking to the defined libc, and various GNU conventions. This compatibility may be implemented directly by the secondary toolchain or via a wrapper of some type.

'''Per recipe selection of toolchain usage'''

A mechanism is needed to activate this alternative toolchain on a per-recipe basis. Using variables it should be possible to specify on a per-recipe basis which toolchain should be used, leaving the default up to the user. (It's assumed if the user does not select a default, the system GNU toolchain will be defaulted.)

'''Blacklist/Whitelist'''

Even with the requirement to be generally compatible, we can not expect all of the possible recipes to build properly with the secondary toolchain. As a consequence of this, it will be necessary to implement a blacklist (or whitelist) of specific recipes that are known to work. This way only supported components can be build by the user, avoid numerous complaints and unresolvable defects.

'''SDK'''

The SDK generated by OpenEmbedded-Core also includes the included toolchain components. It would be useful to also provide support for the secondary toolchain within the SDK environment.

== Implementation ==

=== Bitbake variables ===

A set of common bitbake variables, defined in meta/conf/bitbake.conf, are used to define how to run the toolchain components and the proper arguments.

A number of key items may need to be changed, depending on the toolchain being provided. These items may include:

* TARGET_CPPFLAGS
* TARGET_CFLAGS
* TARGET_CXXFLAGS
* TARGET_LDFLAGS
* CC
* CXX
* F77
* CPP
* LD
* CCLD
* AR
* AS
* RANLIB
* STRIP
* OBJCOPY
* OBJDUMP
* NM
* SELECTED_OPTIMIZATION
* FULL_OPTIMIZATION
* DEBUG_OPTIMIZATION

Each of the above items has a default definition within the meta/conf/bitbake.conf file. In addition, many of the components are based on specific tune variables such as:

* TUNE_CCARGS
* TUNE_LDARGS
* TUNE_ASARGS

Deciding which items need to be overridden is up to the author of the layer. It is expected that only a subset of the above referenced variables will need secondary toolchain specific versions for most implementations.

It is suggested that the secondary toolchain values be defined in a single configuration file. Within the example this is "conf/tc-example.conf". The standard name for the file should be 'conf/tc-<toolchain>.conf'.

=== Override ===

In order to allow for the layer to selectively override the variables listed above, a standard override syntax is needed.

The standard override syntax will be 'toolchain-${TOOLCHAIN}'. This will allow us to specify multiple alternative and primary toolchain combinations as necessary.

Each of the toolchain related variables can then be overridden using the standard override syntax: <var>_toolchain-<toolchain>, such as:

CC_toolchain-icc = "icc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"

In order for an individual recipe to use the secondary toolchain, the user would define: TOOLCHAIN_pn-<recipe> = 'toolchain', such as:

TOOLCHAIN_pn-bash = "icc"

The OVERRIDE is implemented in the tc-secondary.bbclass in the example.

=== Blacklist/Whitelist ===

A standard blacklist/whitelist facility is to be implemented. This facility is independent of the existing blacklist class that is part of OpenEmbedded-Core.

Similar to the existing blacklist class, you will be able to define a blacklist, per toolchain type, such as:

TCBLACKLIST[pn] = "toolchain"

or alternatively create a whitelist facility using:

TCBLACKLIST = "toolchain"
TCWHITELIST[pn] = "toolchain"

Using the second approach will allow you to blacklist all recipes, and only enable those known to work with the alternative toolchain.

The blacklist items should be defined within conf/tc-<toolchain>-blacklist.conf. The validation of the blacklist should use the standard tc-blacklist.bbclass implementation available within the example.

=== SDK ===

In order to make it easier to use the alternative toolchain within the SDK. An additional environment-* file should be created. The purpose of this file is to simply override the settings in the main environment file.

See the example section for an implementation that will create this file automatically.

== Implementation Example ==

An example implementation has been created in the layer "meta-tc-example". The file format follows:

meta-tc-example/
meta-tc-example/bin
meta-tc-example/bin/example-toolchain
meta-tc-example/conf
meta-tc-example/conf/layer.conf
meta-tc-example/conf/tc-example.conf
meta-tc-example/conf/tc-example-blacklist.conf
meta-tc-example/classes
meta-tc-example/classes/tc-secondary.bbclass
meta-tc-example/classes/tc-blacklist.bbclass

=== conf/layer.conf ===

# We have a conf and classes directory, append to BBPATH
BBPATH .= ":${LAYERDIR}"

BBFILE_COLLECTIONS += "tc-example"

# If we have a recipes directory, add to BBFILES
#BBFILES += "${LAYERDIR}/recipes-*/*/*.bb ${LAYERDIR}/*/recipes-*/*/*.bbappend"
#BBFILE_COLLECTIONS += "tc-example"
#BBFILE_PATTERN_tc-example := "^${LAYERDIR}/"
#BBFILE_PRIORITY_tc-example = "5"

Above is standard layer configuration. Note, the commented out sections should be re-enabled if the secondary toolchain includes recipes and/or bbappend files.

PATH := "${PATH}:${LAYERDIR}/bin"

Add to the path a standard system path, a layer specific path. This is the way you can add a secondary toolchain to the path.

# Define the alternative toolchain
SECONDARYTC = 'example'

# Enable the secondary toolchain
INHERIT += 'tc-secondary tc-blacklist'

Define the alternative toolchain name in SECONDARYTC. The inherit causes the tc-secondary.bbclass and tc-blacklist.bbclass to be loaded later. Eventually this class used SECONDARYTC to load in the secondary toolchain configuration files.

=== conf/tc-example.conf ===

# Define default system toolchain (default to built-in)
TOOLCHAIN = 'gnu'

Define a reasonable default such as gnu. The purpose of this default is to allow someone to specify the default or provide default override items for the built-in toolchain.

# Define a secondary toolchain called 'example'
# the 'example' toolchain can be activated using:
#
# Per package (preferred):
# TOOLCHAIN_pn-bash = 'example'
#
# Globally:
# TOOLCHAIN = 'example'
# TOOLCHAIN_pn-eglibc = 'gnu'

The above is an example of how to use the alternative toolchain.

# When the secondary toolchain is used, we add an automatic depend...
DEPENDS_append_toolchain-example = " virtual/TOOLCHAIN-EXAMPLE"

# ...the depend can be provided by a recipe or as an ASSUME_PROVIDED
ASSUME_PROVIDED_append = " virtual/TOOLCHAIN-EXAMPLE"

Use the above as an example on how to add additional dependencies to specific packages. This can be used to trigger a recipe to build.

# Default values from bitbake.conf...
# Variables with a '*' are things likely to need to be overridden:
#
# FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
# DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe"
# SELECTED_OPTIMIZATION = "${@d.getVar(['FULL_OPTIMIZATION', 'DEBUG_OPTIMIZATION'][d.getVar('DEBUG_BUILD', True) == '1'], True)}"
#
# * TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
#
# * TARGET_CPPFLAGS = ""
# * TARGET_CFLAGS = "${TARGET_CPPFLAGS} ${SELECTED_OPTIMIZATION}"
# * TARGET CXXFLAGS = "${TARGET_CFLAGS} -fpermissive"
# * TARGET_LDFLAGS = "-Wl,-O1 ${TARGET_LINK_HASH_STYLE}"
#
# TARGET_CC_ARCH = "${TUNE_CCARGS}"
# TARGET_LD_ARCH = "${TUNE_LDARGS}"
# TARGET_AS_ARCH = "${TUNE_ASARGS}"
#
# HOST_CC_ARCH = "${TARGET_CC_ARCH}"
# HOST_LD_ARCH = "${TARGET_LD_ARCH}"
# HOST_AS_ARCH = "${TARGET_AS_ARCH}"
#
# * CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CXX = "${CCACHE}${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * F77 = "${CCACHE}${HOST_PREFIX}g77 ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CPP = "${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
# * LD = "${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"
# * CCLD = "${CC}"
# AR = "${HOST_PREFIX}ar"
# AS = "${HOST_PREFIX}as ${HOST_AS_ARCH}"
# RANLIB = "${HOST_PREFIX}ranlib"
# STRIP = "${HOST_PREFIX}strip"
# OBJCOPY = "${HOST_PREFIX}objcopy"
# OBJDUMP = "${HOST_PREFIX}objdump"
# NM = "${HOST_PREFIX}nm"

# Override values:
CC_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CXX_toolchain-example = "example-toolchain ${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CPP_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
LD_toolchain-example = "example-toolchain ${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"

The above implements the necessary overrides for the example toolchain.

=== conf/tc-example-blacklist.conf ===

Example configuration file that enables the blacklist and defines the defaults.

# Blacklist certain items...
TCBLACKLIST[eglibc] = 'example'
TCBLACKLIST[bash] = 'example'

#
# or
#

# Switch to a global blacklist and whitelist situation...
# TCBLACKLIST = 'example'
# TCWHITELIST[bash] = 'example'

=== classes/tc-secondary.bbclass ===

This is the standard secondary toolchain class. It should be the same in all secondary toolchain layers.

# In order to add overrides, we need to do it later then in the layers.conf
# the only standard way is using an inherit, which requires a bbclass

# Add the necessary override
TOOLCHAINOVERRIDES = ":toolchain-${TOOLCHAIN}"
TOOLCHAINOVERRIDES[vardepsexclude] = "TOOLCHAIN"

OVERRIDES .= "${TOOLCHAINOVERRIDES}"
OVERRIDES[vardepsexclude] += "TOOLCHAINOVERRIDES"

require conf/tc-${SECONDARYTC}.conf

The above code configures the basic overrides and loads the configuration file. There is one limitation to the above however. It will only load one secondary toolchain. A simple way around this is to rename the inherit and change to using something other then SECONDARYTC. This likely will not be a problem.

toolchain_create_sdk_env_script_alt () {
set -x

# Create environment setup script (secondary toolchain)
orig_script=${1:-${SDK_OUTPUT}/${SDKPATH}/environment-setup-${REAL_MULTIMACH_TARGET_SYS}}
script="$orig_script""-${SECONDARYTC}"
rm -f $script
touch $script
echo ". $orig_script" | sed -e "s:${SDK_OUTPUT}::" >> $script
if [ '${CC_toolchain-${SECONDARYTC}}' != '${''CC_toolchain-${SECONDARYTC}}' ]; then
echo 'export CC="${CC_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CXX_toolchain-${SECONDARYTC}}' != '${''CXX_toolchain-${SECONDARYTC}}' ]; then
echo 'export CXX="${CXX_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CPP_toolchain-${SECONDARYTC}}' != '${''CPP_toolchain-${SECONDARYTC}}' ]; then
echo 'export CPP="${CPP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${AS_toolchain-${SECONDARYTC}}' != '${''AS_toolchain-${SECONDARYTC}}' ]; then
echo 'export AS="${AS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${LD_toolchain-${SECONDARYTC}}' != '${''LD_toolchain-${SECONDARYTC}}' ]; then
echo 'export LD="${LD_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${RANLIB_toolchain-${SECONDARYTC}}' != '${''RANLIB_toolchain-${SECONDARYTC}}' ]; then
echo 'export RANLIB="${RANLIB_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${STRIP_toolchain-${SECONDARYTC}}' != '${''STRIP_toolchain-${SECONDARYTC}}' ]; then
echo 'export STRIP="${STRIP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${OBJCOPY_toolchain-${SECONDARYTC}}' != '${''OBJCOPY_toolchain-${SECONDARYTC}}' ]; then
echo 'export OBJCOPY="${OBJCOPY_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${OBJDUMP_toolchain-${SECONDARYTC}}' != '${''OBJDUMP_toolchain-${SECONDARYTC}}' ]; then
echo 'export OBJDUMP="${OBJDUMP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${NM_toolchain-${SECONDARYTC}}' != '${''NM_toolchain-${SECONDARYTC}}' ]; then
echo 'export NM="${NM_toolchain-${SECONDARYTC}}"' >> $script
fi

if [ '${CFLAGS_toolchain-${SECONDARYTC}}' != '${''CFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CFLAGS="${CFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CXXFLAGS_toolchain-${SECONDARYTC}}' != '${''CXXFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CXXFLAGS="${CXXFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${LDFLAGS_toolchain-${SECONDARYTC}}' != '${''LDFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export LDFLAGS="${LDFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CPPFLAGS_toolchain-${SECONDARYTC}}' != '${''CPPFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CPPFLAGS="${CPPFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi

# Fixup any sysroot references
sed -i -e "s:${STAGING_DIR_TARGET}:${SDKTARGETSYSROOT}:g" $script

# All environment scripts require OECORE_NATIVE_SYSROOT
echo 'export OECORE_NATIVE_SYSROOT="${SDKPATHNATIVE}"' >> $script
}

populate_sdk_image_append() {
toolchain_create_sdk_env_script_alt ${SDK_OUTPUT}/${SDKPATH}/environment-setup-${REAL_MULTIMACH_TARGET_SYS}
}

The above code will add an additional environment-setup-*-<toolchain> file. This can be used to enable the right environment to use the alternative toolchain. Note, it does not add anything to the path so the user will still need to configure the path to the alternative toolchain. (The layer creator can always provide the necessary 'nativesdk' or similar package that provides the alternative toolchain as part of the SDK as well.)

=== classes/tc-blacklist.bbclass ===

This is the standard secondary toolchain blacklist class. It should be the same in all secondary toolchain layers.

# Implement blacklist/whitelist functionality for alternative toolchains
# Based on the oe-core blacklist bclass
#
# To add a blacklisted package:
# TCBLACKLIST[pn] = 'toolchain'
#
# To blacklist everything, and whitelist a package:
# TCBLACKLIST = 'toolchain'
# TCWHITELIST[pn] = 'toolchain'

include conf/tc-${SECONDARYTC}-blacklist.conf

python () {
tc = d.getVar('TOOLCHAIN', True)
pn = d.getVar('PN', True)

blacklist = d.getVarFlag('TCBLACKLIST', pn, True)

if blacklist and blacklist == tc:
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))

blacklist = d.getVar('TCBLACKLIST', True)
whitelist = d.getVarFlag('TCWHITELIST', pn, True)

if blacklist and (not whitelist or whitelist != tc):
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))
}

== Further comments and information ==

[[Category:Dev]]
[[Category:Layers]]

Adding a secondary toolchain

2012-11-14T22:24:15Z

Mhatle: /* classes/tc-secondary.bbclass */

== Background ==

In OpenEmbedded-Core there in an included toolchain that is built up of GNU binutils, gcc, and eglibc. In addition related components such as gdb, cross-prelink, mklibs and others provider additional toolchain functionality. In this document we focus on explaining best practices for including an additional toolchain that is capable of assembling, compiling and linking code. This document does not cover replacing the existing toolchain or toolchain components with external elements.

Many companies have commercial / highly optimized toolchains for specific purposes, such as ICC from Intel, LLVM, ARM Ltd toolchains, etc.. In some cases, these highly optimized toolchains may result in better performance for some applications, however they are often not generally applicable as they can not compile all of the system software. So instead of replacing the included GNU toolchain, a mechanism for providing an alternative/secondary toolchain is desired.

== Requirements ==

'''Secondary Toolchains should be implemented in a layer'''

In order to facilitate re-use and make it easier to completely disable the secondary toolchain, a layer must be used to implement the secondary toolchain.

'''Secondary Toolchain must be generally compatible with GNU toolchain'''

The secondary toolchain must be generally compatible with the idea of sysroots, linking to the defined libc, and various GNU conventions. This compatibility may be implemented directly by the secondary toolchain or via a wrapper of some type.

'''Per recipe selection of toolchain usage'''

A mechanism is needed to activate this alternative toolchain on a per-recipe basis. Using variables it should be possible to specify on a per-recipe basis which toolchain should be used, leaving the default up to the user. (It's assumed if the user does not select a default, the system GNU toolchain will be defaulted.)

'''Blacklist/Whitelist'''

Even with the requirement to be generally compatible, we can not expect all of the possible recipes to build properly with the secondary toolchain. As a consequence of this, it will be necessary to implement a blacklist (or whitelist) of specific recipes that are known to work. This way only supported components can be build by the user, avoid numerous complaints and unresolvable defects.

'''SDK'''

The SDK generated by OpenEmbedded-Core also includes the included toolchain components. It would be useful to also provide support for the secondary toolchain within the SDK environment.

== Implementation ==

=== Bitbake variables ===

A set of common bitbake variables, defined in meta/conf/bitbake.conf, are used to define how to run the toolchain components and the proper arguments.

A number of key items may need to be changed, depending on the toolchain being provided. These items may include:

* TARGET_CPPFLAGS
* TARGET_CFLAGS
* TARGET_CXXFLAGS
* TARGET_LDFLAGS
* CC
* CXX
* F77
* CPP
* LD
* CCLD
* AR
* AS
* RANLIB
* STRIP
* OBJCOPY
* OBJDUMP
* NM
* SELECTED_OPTIMIZATION
* FULL_OPTIMIZATION
* DEBUG_OPTIMIZATION

Each of the above items has a default definition within the meta/conf/bitbake.conf file. In addition, many of the components are based on specific tune variables such as:

* TUNE_CCARGS
* TUNE_LDARGS
* TUNE_ASARGS

Deciding which items need to be overridden is up to the author of the layer. It is expected that only a subset of the above referenced variables will need secondary toolchain specific versions for most implementations.

It is suggested that the secondary toolchain values be defined in a single configuration file. Within the example this is "conf/tc-example.conf". The standard name for the file should be 'conf/tc-<toolchain>.conf'.

=== Override ===

In order to allow for the layer to selectively override the variables listed above, a standard override syntax is needed.

The standard override syntax will be 'toolchain-${TOOLCHAIN}'. This will allow us to specify multiple alternative and primary toolchain combinations as necessary.

Each of the toolchain related variables can then be overridden using the standard override syntax: <var>_toolchain-<toolchain>, such as:

CC_toolchain-icc = "icc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"

In order for an individual recipe to use the secondary toolchain, the user would define: TOOLCHAIN_pn-<recipe> = 'toolchain', such as:

TOOLCHAIN_pn-bash = "icc"

The OVERRIDE is implemented in the tc-secondary.bbclass in the example.

=== Blacklist/Whitelist ===

A standard blacklist/whitelist facility is to be implemented. This facility is independent of the existing blacklist class that is part of OpenEmbedded-Core.

Similar to the existing blacklist class, you will be able to define a blacklist, per toolchain type, such as:

TCBLACKLIST[pn] = "toolchain"

or alternatively create a whitelist facility using:

TCBLACKLIST = "toolchain"
TCWHITELIST[pn] = "toolchain"

Using the second approach will allow you to blacklist all recipes, and only enable those known to work with the alternative toolchain.

The blacklist items should be defined within conf/tc-<toolchain>-blacklist.conf. The validation of the blacklist should use the standard tc-blacklist.bbclass implementation available within the example.

=== SDK ===

In order to make it easier to use the alternative toolchain within the SDK. An additional environment-* file should be created. The purpose of this file is to simply override the settings in the main environment file.

See the example section for an implementation that will create this file automatically.

== Implementation Example ==

An example implementation has been created in the layer "meta-tc-example". The file format follows:

meta-tc-example/
meta-tc-example/bin
meta-tc-example/bin/example-toolchain
meta-tc-example/conf
meta-tc-example/conf/layer.conf
meta-tc-example/conf/tc-example.conf
meta-tc-example/conf/tc-example-blacklist.conf
meta-tc-example/classes
meta-tc-example/classes/tc-secondary.bbclass
meta-tc-example/classes/tc-blacklist.bbclass

=== conf/layer.conf ===

# We have a conf and classes directory, append to BBPATH
BBPATH .= ":${LAYERDIR}"

BBFILE_COLLECTIONS += "tc-example"

# If we have a recipes directory, add to BBFILES
#BBFILES += "${LAYERDIR}/recipes-*/*/*.bb ${LAYERDIR}/*/recipes-*/*/*.bbappend"
#BBFILE_COLLECTIONS += "tc-example"
#BBFILE_PATTERN_tc-example := "^${LAYERDIR}/"
#BBFILE_PRIORITY_tc-example = "5"

Above is standard layer configuration. Note, the commented out sections should be re-enabled if the secondary toolchain includes recipes and/or bbappend files.

PATH := "${PATH}:${LAYERDIR}/bin"

Add to the path a standard system path, a layer specific path. This is the way you can add a secondary toolchain to the path.

# Define the alternative toolchain
SECONDARYTC = 'example'

# Enable the secondary toolchain
INHERIT += 'tc-secondary tc-blacklist'

Define the alternative toolchain name in SECONDARYTC. The inherit causes the tc-secondary.bbclass and tc-blacklist.bbclass to be loaded later. Eventually this class used SECONDARYTC to load in the secondary toolchain configuration files.

=== conf/tc-example.conf ===

# Define default system toolchain (default to built-in)
TOOLCHAIN = 'gnu'

Define a reasonable default such as gnu. The purpose of this default is to allow someone to specify the default or provide default override items for the built-in toolchain.

# Define a secondary toolchain called 'example'
# the 'example' toolchain can be activated using:
#
# Per package (preferred):
# TOOLCHAIN_pn-bash = 'example'
#
# Globally:
# TOOLCHAIN = 'example'
# TOOLCHAIN_pn-eglibc = 'gnu'

The above is an example of how to use the alternative toolchain.

# When the secondary toolchain is used, we add an automatic depend...
DEPENDS_append_toolchain-example = " virtual/TOOLCHAIN-EXAMPLE"

# ...the depend can be provided by a recipe or as an ASSUME_PROVIDED
ASSUME_PROVIDED_append = " virtual/TOOLCHAIN-EXAMPLE"

Use the above as an example on how to add additional dependencies to specific packages. This can be used to trigger a recipe to build.

# Default values from bitbake.conf...
# Variables with a '*' are things likely to need to be overridden:
#
# FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
# DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe"
# SELECTED_OPTIMIZATION = "${@d.getVar(['FULL_OPTIMIZATION', 'DEBUG_OPTIMIZATION'][d.getVar('DEBUG_BUILD', True) == '1'], True)}"
#
# * TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
#
# * TARGET_CPPFLAGS = ""
# * TARGET_CFLAGS = "${TARGET_CPPFLAGS} ${SELECTED_OPTIMIZATION}"
# * TARGET CXXFLAGS = "${TARGET_CFLAGS} -fpermissive"
# * TARGET_LDFLAGS = "-Wl,-O1 ${TARGET_LINK_HASH_STYLE}"
#
# TARGET_CC_ARCH = "${TUNE_CCARGS}"
# TARGET_LD_ARCH = "${TUNE_LDARGS}"
# TARGET_AS_ARCH = "${TUNE_ASARGS}"
#
# HOST_CC_ARCH = "${TARGET_CC_ARCH}"
# HOST_LD_ARCH = "${TARGET_LD_ARCH}"
# HOST_AS_ARCH = "${TARGET_AS_ARCH}"
#
# * CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CXX = "${CCACHE}${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * F77 = "${CCACHE}${HOST_PREFIX}g77 ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CPP = "${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
# * LD = "${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"
# * CCLD = "${CC}"
# AR = "${HOST_PREFIX}ar"
# AS = "${HOST_PREFIX}as ${HOST_AS_ARCH}"
# RANLIB = "${HOST_PREFIX}ranlib"
# STRIP = "${HOST_PREFIX}strip"
# OBJCOPY = "${HOST_PREFIX}objcopy"
# OBJDUMP = "${HOST_PREFIX}objdump"
# NM = "${HOST_PREFIX}nm"

# Override values:
CC_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CXX_toolchain-example = "example-toolchain ${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CPP_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
LD_toolchain-example = "example-toolchain ${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"

The above implements the necessary overrides for the example toolchain.

=== conf/tc-example-blacklist.conf ===

Example configuration file that enables the blacklist and defines the defaults.

# Blacklist certain items...
TCBLACKLIST[eglibc] = 'example'
TCBLACKLIST[bash] = 'example'

#
# or
#

# Switch to a global blacklist and whitelist situation...
# TCBLACKLIST = 'example'
# TCWHITELIST[bash] = 'example'

=== classes/tc-secondary.bbclass ===

This is the standard secondary toolchain class. It should be the same in all secondary toolchain layers.

# In order to add overrides, we need to do it later then in the layers.conf
# the only standard way is using an inherit, which requires a bbclass

# Add the necessary override
TOOLCHAINOVERRIDES = ":toolchain-${TOOLCHAIN}"
TOOLCHAINOVERRIDES[vardepsexclude] = "TOOLCHAIN"

OVERRIDES .= "${TOOLCHAINOVERRIDES}"
OVERRIDES[vardepsexclude] += "TOOLCHAINOVERRIDES"

require conf/tc-${SECONDARYTC}.conf

The above code configures the basic overrides and loads the configuration file. There is one limitation to the above however. It will only load one secondary toolchain. A simple way around this is to rename the inherit and change to using something other then SECONDARYTC. This likely will not be a problem.

toolchain_create_sdk_env_script_alt () {
set -x

# Create environment setup script (secondary toolchain)
orig_script=${1:-${SDK_OUTPUT}/${SDKPATH}/environment-setup-${REAL_MULTIMACH_TARGET_SYS}}
script="$orig_script""-${SECONDARYTC}"
rm -f $script
touch $script
echo ". $orig_script" | sed -e "s:${SDK_OUTPUT}::" >> $script
if [ '${CC_toolchain-${SECONDARYTC}}' != '${''CC_toolchain-${SECONDARYTC}}' ]; then
echo 'export CC="${CC_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CXX_toolchain-${SECONDARYTC}}' != '${''CXX_toolchain-${SECONDARYTC}}' ]; then
echo 'export CXX="${CXX_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CPP_toolchain-${SECONDARYTC}}' != '${''CPP_toolchain-${SECONDARYTC}}' ]; then
echo 'export CPP="${CPP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${AS_toolchain-${SECONDARYTC}}' != '${''AS_toolchain-${SECONDARYTC}}' ]; then
echo 'export AS="${AS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${LD_toolchain-${SECONDARYTC}}' != '${''LD_toolchain-${SECONDARYTC}}' ]; then
echo 'export LD="${LD_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${RANLIB_toolchain-${SECONDARYTC}}' != '${''RANLIB_toolchain-${SECONDARYTC}}' ]; then
echo 'export RANLIB="${RANLIB_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${STRIP_toolchain-${SECONDARYTC}}' != '${''STRIP_toolchain-${SECONDARYTC}}' ]; then
echo 'export STRIP="${STRIP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${OBJCOPY_toolchain-${SECONDARYTC}}' != '${''OBJCOPY_toolchain-${SECONDARYTC}}' ]; then
echo 'export OBJCOPY="${OBJCOPY_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${OBJDUMP_toolchain-${SECONDARYTC}}' != '${''OBJDUMP_toolchain-${SECONDARYTC}}' ]; then
echo 'export OBJDUMP="${OBJDUMP_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${NM_toolchain-${SECONDARYTC}}' != '${''NM_toolchain-${SECONDARYTC}}' ]; then
echo 'export NM="${NM_toolchain-${SECONDARYTC}}"' >> $script
fi

if [ '${CFLAGS_toolchain-${SECONDARYTC}}' != '${''CFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CFLAGS="${CFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CXXFLAGS_toolchain-${SECONDARYTC}}' != '${''CXXFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CXXFLAGS="${CXXFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${LDFLAGS_toolchain-${SECONDARYTC}}' != '${''LDFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export LDFLAGS="${LDFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi
if [ '${CPPFLAGS_toolchain-${SECONDARYTC}}' != '${''CPPFLAGS_toolchain-${SECONDARYTC}}' ]; then
echo 'export CPPFLAGS="${CPPFLAGS_toolchain-${SECONDARYTC}}"' >> $script
fi

# Fixup any sysroot references
sed -i -e "s:${STAGING_DIR_TARGET}:${SDKTARGETSYSROOT}:g" $script

# All environment scripts require OECORE_NATIVE_SYSROOT
echo 'export OECORE_NATIVE_SYSROOT="${SDKPATHNATIVE}"' >> $script
}

populate_sdk_image_append() {
toolchain_create_sdk_env_script_alt ${SDK_OUTPUT}/${SDKPATH}/environment-setup-${REAL_MULTIMACH_TARGET_SYS}
}

The above code will add an additional environment-setup-*-<toolchain> file. This can be used to enable the right environment to use the alternative toolchain. Note, it does not add anything to the path so the user will still need to configure the path to the alternative toolchain. (The layer creator can always provide the necessary 'nativesdk' or similar package that provides the alternative toolchain as part of the SDK as well.)

=== classes/tc-blacklist.bbclass ===

This is the standard secondary toolchain blacklist class. It should be the same in all secondary toolchain layers.

# Implement blacklist/whitelist functionality for alternative toolchains
# Based on the oe-core blacklist bclass
#
# To add a blacklisted package:
# TCBLACKLIST[pn] = 'toolchain'
#
# To blacklist everything, and whitelist a package:
# TCBLACKLIST = 'toolchain'
# TCWHITELIST[pn] = 'toolchain'

include conf/tc-${SECONDARYTC}-blacklist.conf

python () {
tc = d.getVar('TOOLCHAIN', True)
pn = d.getVar('PN', True)

blacklist = d.getVarFlag('TCBLACKLIST', pn, True)

if blacklist and blacklist == tc:
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))

blacklist = d.getVar('TCBLACKLIST', True)
whitelist = d.getVarFlag('TCWHITELIST', pn, True)

if blacklist and (not whitelist or whitelist != tc):
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))
}

== Further comments and information ==

[[Category:Dev]]
[[Category:Layers]]

Adding a secondary toolchain

2012-11-14T22:19:17Z

Mhatle: /* SDK */

== Background ==

In OpenEmbedded-Core there in an included toolchain that is built up of GNU binutils, gcc, and eglibc. In addition related components such as gdb, cross-prelink, mklibs and others provider additional toolchain functionality. In this document we focus on explaining best practices for including an additional toolchain that is capable of assembling, compiling and linking code. This document does not cover replacing the existing toolchain or toolchain components with external elements.

Many companies have commercial / highly optimized toolchains for specific purposes, such as ICC from Intel, LLVM, ARM Ltd toolchains, etc.. In some cases, these highly optimized toolchains may result in better performance for some applications, however they are often not generally applicable as they can not compile all of the system software. So instead of replacing the included GNU toolchain, a mechanism for providing an alternative/secondary toolchain is desired.

== Requirements ==

'''Secondary Toolchains should be implemented in a layer'''

In order to facilitate re-use and make it easier to completely disable the secondary toolchain, a layer must be used to implement the secondary toolchain.

'''Secondary Toolchain must be generally compatible with GNU toolchain'''

The secondary toolchain must be generally compatible with the idea of sysroots, linking to the defined libc, and various GNU conventions. This compatibility may be implemented directly by the secondary toolchain or via a wrapper of some type.

'''Per recipe selection of toolchain usage'''

A mechanism is needed to activate this alternative toolchain on a per-recipe basis. Using variables it should be possible to specify on a per-recipe basis which toolchain should be used, leaving the default up to the user. (It's assumed if the user does not select a default, the system GNU toolchain will be defaulted.)

'''Blacklist/Whitelist'''

Even with the requirement to be generally compatible, we can not expect all of the possible recipes to build properly with the secondary toolchain. As a consequence of this, it will be necessary to implement a blacklist (or whitelist) of specific recipes that are known to work. This way only supported components can be build by the user, avoid numerous complaints and unresolvable defects.

'''SDK'''

The SDK generated by OpenEmbedded-Core also includes the included toolchain components. It would be useful to also provide support for the secondary toolchain within the SDK environment.

== Implementation ==

=== Bitbake variables ===

A set of common bitbake variables, defined in meta/conf/bitbake.conf, are used to define how to run the toolchain components and the proper arguments.

A number of key items may need to be changed, depending on the toolchain being provided. These items may include:

* TARGET_CPPFLAGS
* TARGET_CFLAGS
* TARGET_CXXFLAGS
* TARGET_LDFLAGS
* CC
* CXX
* F77
* CPP
* LD
* CCLD
* AR
* AS
* RANLIB
* STRIP
* OBJCOPY
* OBJDUMP
* NM
* SELECTED_OPTIMIZATION
* FULL_OPTIMIZATION
* DEBUG_OPTIMIZATION

Each of the above items has a default definition within the meta/conf/bitbake.conf file. In addition, many of the components are based on specific tune variables such as:

* TUNE_CCARGS
* TUNE_LDARGS
* TUNE_ASARGS

Deciding which items need to be overridden is up to the author of the layer. It is expected that only a subset of the above referenced variables will need secondary toolchain specific versions for most implementations.

It is suggested that the secondary toolchain values be defined in a single configuration file. Within the example this is "conf/tc-example.conf". The standard name for the file should be 'conf/tc-<toolchain>.conf'.

=== Override ===

In order to allow for the layer to selectively override the variables listed above, a standard override syntax is needed.

The standard override syntax will be 'toolchain-${TOOLCHAIN}'. This will allow us to specify multiple alternative and primary toolchain combinations as necessary.

Each of the toolchain related variables can then be overridden using the standard override syntax: <var>_toolchain-<toolchain>, such as:

CC_toolchain-icc = "icc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"

In order for an individual recipe to use the secondary toolchain, the user would define: TOOLCHAIN_pn-<recipe> = 'toolchain', such as:

TOOLCHAIN_pn-bash = "icc"

The OVERRIDE is implemented in the tc-secondary.bbclass in the example.

=== Blacklist/Whitelist ===

A standard blacklist/whitelist facility is to be implemented. This facility is independent of the existing blacklist class that is part of OpenEmbedded-Core.

Similar to the existing blacklist class, you will be able to define a blacklist, per toolchain type, such as:

TCBLACKLIST[pn] = "toolchain"

or alternatively create a whitelist facility using:

TCBLACKLIST = "toolchain"
TCWHITELIST[pn] = "toolchain"

Using the second approach will allow you to blacklist all recipes, and only enable those known to work with the alternative toolchain.

The blacklist items should be defined within conf/tc-<toolchain>-blacklist.conf. The validation of the blacklist should use the standard tc-blacklist.bbclass implementation available within the example.

=== SDK ===

In order to make it easier to use the alternative toolchain within the SDK. An additional environment-* file should be created. The purpose of this file is to simply override the settings in the main environment file.

See the example section for an implementation that will create this file automatically.

== Implementation Example ==

An example implementation has been created in the layer "meta-tc-example". The file format follows:

meta-tc-example/
meta-tc-example/bin
meta-tc-example/bin/example-toolchain
meta-tc-example/conf
meta-tc-example/conf/layer.conf
meta-tc-example/conf/tc-example.conf
meta-tc-example/conf/tc-example-blacklist.conf
meta-tc-example/classes
meta-tc-example/classes/tc-secondary.bbclass
meta-tc-example/classes/tc-blacklist.bbclass

=== conf/layer.conf ===

# We have a conf and classes directory, append to BBPATH
BBPATH .= ":${LAYERDIR}"

BBFILE_COLLECTIONS += "tc-example"

# If we have a recipes directory, add to BBFILES
#BBFILES += "${LAYERDIR}/recipes-*/*/*.bb ${LAYERDIR}/*/recipes-*/*/*.bbappend"
#BBFILE_COLLECTIONS += "tc-example"
#BBFILE_PATTERN_tc-example := "^${LAYERDIR}/"
#BBFILE_PRIORITY_tc-example = "5"

Above is standard layer configuration. Note, the commented out sections should be re-enabled if the secondary toolchain includes recipes and/or bbappend files.

PATH := "${PATH}:${LAYERDIR}/bin"

Add to the path a standard system path, a layer specific path. This is the way you can add a secondary toolchain to the path.

# Define the alternative toolchain
SECONDARYTC = 'example'

# Enable the secondary toolchain
INHERIT += 'tc-secondary tc-blacklist'

Define the alternative toolchain name in SECONDARYTC. The inherit causes the tc-secondary.bbclass and tc-blacklist.bbclass to be loaded later. Eventually this class used SECONDARYTC to load in the secondary toolchain configuration files.

=== conf/tc-example.conf ===

# Define default system toolchain (default to built-in)
TOOLCHAIN = 'gnu'

Define a reasonable default such as gnu. The purpose of this default is to allow someone to specify the default or provide default override items for the built-in toolchain.

# Define a secondary toolchain called 'example'
# the 'example' toolchain can be activated using:
#
# Per package (preferred):
# TOOLCHAIN_pn-bash = 'example'
#
# Globally:
# TOOLCHAIN = 'example'
# TOOLCHAIN_pn-eglibc = 'gnu'

The above is an example of how to use the alternative toolchain.

# When the secondary toolchain is used, we add an automatic depend...
DEPENDS_append_toolchain-example = " virtual/TOOLCHAIN-EXAMPLE"

# ...the depend can be provided by a recipe or as an ASSUME_PROVIDED
ASSUME_PROVIDED_append = " virtual/TOOLCHAIN-EXAMPLE"

Use the above as an example on how to add additional dependencies to specific packages. This can be used to trigger a recipe to build.

# Default values from bitbake.conf...
# Variables with a '*' are things likely to need to be overridden:
#
# FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
# DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe"
# SELECTED_OPTIMIZATION = "${@d.getVar(['FULL_OPTIMIZATION', 'DEBUG_OPTIMIZATION'][d.getVar('DEBUG_BUILD', True) == '1'], True)}"
#
# * TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
#
# * TARGET_CPPFLAGS = ""
# * TARGET_CFLAGS = "${TARGET_CPPFLAGS} ${SELECTED_OPTIMIZATION}"
# * TARGET CXXFLAGS = "${TARGET_CFLAGS} -fpermissive"
# * TARGET_LDFLAGS = "-Wl,-O1 ${TARGET_LINK_HASH_STYLE}"
#
# TARGET_CC_ARCH = "${TUNE_CCARGS}"
# TARGET_LD_ARCH = "${TUNE_LDARGS}"
# TARGET_AS_ARCH = "${TUNE_ASARGS}"
#
# HOST_CC_ARCH = "${TARGET_CC_ARCH}"
# HOST_LD_ARCH = "${TARGET_LD_ARCH}"
# HOST_AS_ARCH = "${TARGET_AS_ARCH}"
#
# * CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CXX = "${CCACHE}${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * F77 = "${CCACHE}${HOST_PREFIX}g77 ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CPP = "${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
# * LD = "${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"
# * CCLD = "${CC}"
# AR = "${HOST_PREFIX}ar"
# AS = "${HOST_PREFIX}as ${HOST_AS_ARCH}"
# RANLIB = "${HOST_PREFIX}ranlib"
# STRIP = "${HOST_PREFIX}strip"
# OBJCOPY = "${HOST_PREFIX}objcopy"
# OBJDUMP = "${HOST_PREFIX}objdump"
# NM = "${HOST_PREFIX}nm"

# Override values:
CC_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CXX_toolchain-example = "example-toolchain ${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CPP_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
LD_toolchain-example = "example-toolchain ${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"

The above implements the necessary overrides for the example toolchain.

=== conf/tc-example-blacklist.conf ===

Example configuration file that enables the blacklist and defines the defaults.

# Blacklist certain items...
TCBLACKLIST[eglibc] = 'example'
TCBLACKLIST[bash] = 'example'

#
# or
#

# Switch to a global blacklist and whitelist situation...
# TCBLACKLIST = 'example'
# TCWHITELIST[bash] = 'example'

=== classes/tc-secondary.bbclass ===

This is the standard secondary toolchain class. It should be the same in all secondary toolchain layers.

# In order to add overrides, we need to do it later then in the layers.conf
# the only standard way is using an inherit, which requires a bbclass

# Add the necessary override
TOOLCHAINOVERRIDES = ":toolchain-${TOOLCHAIN}"
TOOLCHAINOVERRIDES[vardepsexclude] = "TOOLCHAIN"

OVERRIDES .= "${TOOLCHAINOVERRIDES}"
OVERRIDES[vardepsexclude] += "TOOLCHAINOVERRIDES"

require conf/tc-${SECONDARYTC}.conf

=== classes/tc-blacklist.bbclass ===

This is the standard secondary toolchain blacklist class. It should be the same in all secondary toolchain layers.

# Implement blacklist/whitelist functionality for alternative toolchains
# Based on the oe-core blacklist bclass
#
# To add a blacklisted package:
# TCBLACKLIST[pn] = 'toolchain'
#
# To blacklist everything, and whitelist a package:
# TCBLACKLIST = 'toolchain'
# TCWHITELIST[pn] = 'toolchain'

include conf/tc-${SECONDARYTC}-blacklist.conf

python () {
tc = d.getVar('TOOLCHAIN', True)
pn = d.getVar('PN', True)

blacklist = d.getVarFlag('TCBLACKLIST', pn, True)

if blacklist and blacklist == tc:
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))

blacklist = d.getVar('TCBLACKLIST', True)
whitelist = d.getVarFlag('TCWHITELIST', pn, True)

if blacklist and (not whitelist or whitelist != tc):
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))
}

== Further comments and information ==

[[Category:Dev]]
[[Category:Layers]]

Adding a secondary toolchain

2012-11-14T17:45:45Z

Mhatle: /* Implementation Example */

== Background ==

In OpenEmbedded-Core there in an included toolchain that is built up of GNU binutils, gcc, and eglibc. In addition related components such as gdb, cross-prelink, mklibs and others provider additional toolchain functionality. In this document we focus on explaining best practices for including an additional toolchain that is capable of assembling, compiling and linking code. This document does not cover replacing the existing toolchain or toolchain components with external elements.

Many companies have commercial / highly optimized toolchains for specific purposes, such as ICC from Intel, LLVM, ARM Ltd toolchains, etc.. In some cases, these highly optimized toolchains may result in better performance for some applications, however they are often not generally applicable as they can not compile all of the system software. So instead of replacing the included GNU toolchain, a mechanism for providing an alternative/secondary toolchain is desired.

== Requirements ==

'''Secondary Toolchains should be implemented in a layer'''

In order to facilitate re-use and make it easier to completely disable the secondary toolchain, a layer must be used to implement the secondary toolchain.

'''Secondary Toolchain must be generally compatible with GNU toolchain'''

The secondary toolchain must be generally compatible with the idea of sysroots, linking to the defined libc, and various GNU conventions. This compatibility may be implemented directly by the secondary toolchain or via a wrapper of some type.

'''Per recipe selection of toolchain usage'''

A mechanism is needed to activate this alternative toolchain on a per-recipe basis. Using variables it should be possible to specify on a per-recipe basis which toolchain should be used, leaving the default up to the user. (It's assumed if the user does not select a default, the system GNU toolchain will be defaulted.)

'''Blacklist/Whitelist'''

Even with the requirement to be generally compatible, we can not expect all of the possible recipes to build properly with the secondary toolchain. As a consequence of this, it will be necessary to implement a blacklist (or whitelist) of specific recipes that are known to work. This way only supported components can be build by the user, avoid numerous complaints and unresolvable defects.

'''SDK'''

The SDK generated by OpenEmbedded-Core also includes the included toolchain components. It would be useful to also provide support for the secondary toolchain within the SDK environment.

== Implementation ==

=== Bitbake variables ===

A set of common bitbake variables, defined in meta/conf/bitbake.conf, are used to define how to run the toolchain components and the proper arguments.

A number of key items may need to be changed, depending on the toolchain being provided. These items may include:

* TARGET_CPPFLAGS
* TARGET_CFLAGS
* TARGET_CXXFLAGS
* TARGET_LDFLAGS
* CC
* CXX
* F77
* CPP
* LD
* CCLD
* AR
* AS
* RANLIB
* STRIP
* OBJCOPY
* OBJDUMP
* NM
* SELECTED_OPTIMIZATION
* FULL_OPTIMIZATION
* DEBUG_OPTIMIZATION

Each of the above items has a default definition within the meta/conf/bitbake.conf file. In addition, many of the components are based on specific tune variables such as:

* TUNE_CCARGS
* TUNE_LDARGS
* TUNE_ASARGS

Deciding which items need to be overridden is up to the author of the layer. It is expected that only a subset of the above referenced variables will need secondary toolchain specific versions for most implementations.

It is suggested that the secondary toolchain values be defined in a single configuration file. Within the example this is "conf/tc-example.conf". The standard name for the file should be 'conf/tc-<toolchain>.conf'.

=== Override ===

In order to allow for the layer to selectively override the variables listed above, a standard override syntax is needed.

The standard override syntax will be 'toolchain-${TOOLCHAIN}'. This will allow us to specify multiple alternative and primary toolchain combinations as necessary.

Each of the toolchain related variables can then be overridden using the standard override syntax: <var>_toolchain-<toolchain>, such as:

CC_toolchain-icc = "icc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"

In order for an individual recipe to use the secondary toolchain, the user would define: TOOLCHAIN_pn-<recipe> = 'toolchain', such as:

TOOLCHAIN_pn-bash = "icc"

The OVERRIDE is implemented in the tc-secondary.bbclass in the example.

=== Blacklist/Whitelist ===

A standard blacklist/whitelist facility is to be implemented. This facility is independent of the existing blacklist class that is part of OpenEmbedded-Core.

Similar to the existing blacklist class, you will be able to define a blacklist, per toolchain type, such as:

TCBLACKLIST[pn] = "toolchain"

or alternatively create a whitelist facility using:

TCBLACKLIST = "toolchain"
TCWHITELIST[pn] = "toolchain"

Using the second approach will allow you to blacklist all recipes, and only enable those known to work with the alternative toolchain.

The blacklist items should be defined within conf/tc-<toolchain>-blacklist.conf. The validation of the blacklist should use the standard tc-blacklist.bbclass implementation available within the example.

=== SDK ===

TBD: Add SDK extension details

Notes:

* need to add a custom toolchain_create_sdk_env_script for the secondary toolchain(s) (supporting override).
* need to support more then one secondary toolchain
* populate_sdk_image_append -- define a python class to dump the secondary toolchain bits...

== Implementation Example ==

An example implementation has been created in the layer "meta-tc-example". The file format follows:

meta-tc-example/
meta-tc-example/bin
meta-tc-example/bin/example-toolchain
meta-tc-example/conf
meta-tc-example/conf/layer.conf
meta-tc-example/conf/tc-example.conf
meta-tc-example/conf/tc-example-blacklist.conf
meta-tc-example/classes
meta-tc-example/classes/tc-secondary.bbclass
meta-tc-example/classes/tc-blacklist.bbclass

=== conf/layer.conf ===

# We have a conf and classes directory, append to BBPATH
BBPATH .= ":${LAYERDIR}"

BBFILE_COLLECTIONS += "tc-example"

# If we have a recipes directory, add to BBFILES
#BBFILES += "${LAYERDIR}/recipes-*/*/*.bb ${LAYERDIR}/*/recipes-*/*/*.bbappend"
#BBFILE_COLLECTIONS += "tc-example"
#BBFILE_PATTERN_tc-example := "^${LAYERDIR}/"
#BBFILE_PRIORITY_tc-example = "5"

Above is standard layer configuration. Note, the commented out sections should be re-enabled if the secondary toolchain includes recipes and/or bbappend files.

PATH := "${PATH}:${LAYERDIR}/bin"

Add to the path a standard system path, a layer specific path. This is the way you can add a secondary toolchain to the path.

# Define the alternative toolchain
SECONDARYTC = 'example'

# Enable the secondary toolchain
INHERIT += 'tc-secondary tc-blacklist'

Define the alternative toolchain name in SECONDARYTC. The inherit causes the tc-secondary.bbclass and tc-blacklist.bbclass to be loaded later. Eventually this class used SECONDARYTC to load in the secondary toolchain configuration files.

=== conf/tc-example.conf ===

# Define default system toolchain (default to built-in)
TOOLCHAIN = 'gnu'

Define a reasonable default such as gnu. The purpose of this default is to allow someone to specify the default or provide default override items for the built-in toolchain.

# Define a secondary toolchain called 'example'
# the 'example' toolchain can be activated using:
#
# Per package (preferred):
# TOOLCHAIN_pn-bash = 'example'
#
# Globally:
# TOOLCHAIN = 'example'
# TOOLCHAIN_pn-eglibc = 'gnu'

The above is an example of how to use the alternative toolchain.

# When the secondary toolchain is used, we add an automatic depend...
DEPENDS_append_toolchain-example = " virtual/TOOLCHAIN-EXAMPLE"

# ...the depend can be provided by a recipe or as an ASSUME_PROVIDED
ASSUME_PROVIDED_append = " virtual/TOOLCHAIN-EXAMPLE"

Use the above as an example on how to add additional dependencies to specific packages. This can be used to trigger a recipe to build.

# Default values from bitbake.conf...
# Variables with a '*' are things likely to need to be overridden:
#
# FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
# DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe"
# SELECTED_OPTIMIZATION = "${@d.getVar(['FULL_OPTIMIZATION', 'DEBUG_OPTIMIZATION'][d.getVar('DEBUG_BUILD', True) == '1'], True)}"
#
# * TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
#
# * TARGET_CPPFLAGS = ""
# * TARGET_CFLAGS = "${TARGET_CPPFLAGS} ${SELECTED_OPTIMIZATION}"
# * TARGET CXXFLAGS = "${TARGET_CFLAGS} -fpermissive"
# * TARGET_LDFLAGS = "-Wl,-O1 ${TARGET_LINK_HASH_STYLE}"
#
# TARGET_CC_ARCH = "${TUNE_CCARGS}"
# TARGET_LD_ARCH = "${TUNE_LDARGS}"
# TARGET_AS_ARCH = "${TUNE_ASARGS}"
#
# HOST_CC_ARCH = "${TARGET_CC_ARCH}"
# HOST_LD_ARCH = "${TARGET_LD_ARCH}"
# HOST_AS_ARCH = "${TARGET_AS_ARCH}"
#
# * CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CXX = "${CCACHE}${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * F77 = "${CCACHE}${HOST_PREFIX}g77 ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CPP = "${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
# * LD = "${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"
# * CCLD = "${CC}"
# AR = "${HOST_PREFIX}ar"
# AS = "${HOST_PREFIX}as ${HOST_AS_ARCH}"
# RANLIB = "${HOST_PREFIX}ranlib"
# STRIP = "${HOST_PREFIX}strip"
# OBJCOPY = "${HOST_PREFIX}objcopy"
# OBJDUMP = "${HOST_PREFIX}objdump"
# NM = "${HOST_PREFIX}nm"

# Override values:
CC_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CXX_toolchain-example = "example-toolchain ${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
CPP_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
LD_toolchain-example = "example-toolchain ${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"

The above implements the necessary overrides for the example toolchain.

=== conf/tc-example-blacklist.conf ===

Example configuration file that enables the blacklist and defines the defaults.

# Blacklist certain items...
TCBLACKLIST[eglibc] = 'example'
TCBLACKLIST[bash] = 'example'

#
# or
#

# Switch to a global blacklist and whitelist situation...
# TCBLACKLIST = 'example'
# TCWHITELIST[bash] = 'example'

=== classes/tc-secondary.bbclass ===

This is the standard secondary toolchain class. It should be the same in all secondary toolchain layers.

# In order to add overrides, we need to do it later then in the layers.conf
# the only standard way is using an inherit, which requires a bbclass

# Add the necessary override
TOOLCHAINOVERRIDES = ":toolchain-${TOOLCHAIN}"
TOOLCHAINOVERRIDES[vardepsexclude] = "TOOLCHAIN"

OVERRIDES .= "${TOOLCHAINOVERRIDES}"
OVERRIDES[vardepsexclude] += "TOOLCHAINOVERRIDES"

require conf/tc-${SECONDARYTC}.conf

=== classes/tc-blacklist.bbclass ===

This is the standard secondary toolchain blacklist class. It should be the same in all secondary toolchain layers.

# Implement blacklist/whitelist functionality for alternative toolchains
# Based on the oe-core blacklist bclass
#
# To add a blacklisted package:
# TCBLACKLIST[pn] = 'toolchain'
#
# To blacklist everything, and whitelist a package:
# TCBLACKLIST = 'toolchain'
# TCWHITELIST[pn] = 'toolchain'

include conf/tc-${SECONDARYTC}-blacklist.conf

python () {
tc = d.getVar('TOOLCHAIN', True)
pn = d.getVar('PN', True)

blacklist = d.getVarFlag('TCBLACKLIST', pn, True)

if blacklist and blacklist == tc:
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))

blacklist = d.getVar('TCBLACKLIST', True)
whitelist = d.getVarFlag('TCWHITELIST', pn, True)

if blacklist and (not whitelist or whitelist != tc):
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))
}

== Further comments and information ==

[[Category:Dev]]
[[Category:Layers]]

Adding a secondary toolchain

2012-11-13T23:31:38Z

Mhatle: /* SDK */

== Background ==

In OpenEmbedded-Core there in an included toolchain that is built up of GNU binutils, gcc, and eglibc. In addition related components such as gdb, cross-prelink, mklibs and others provider additional toolchain functionality. In this document we focus on explaining best practices for including an additional toolchain that is capable of assembling, compiling and linking code. This document does not cover replacing the existing toolchain or toolchain components with external elements.

Many companies have commercial / highly optimized toolchains for specific purposes, such as ICC from Intel, LLVM, ARM Ltd toolchains, etc.. In some cases, these highly optimized toolchains may result in better performance for some applications, however they are often not generally applicable as they can not compile all of the system software. So instead of replacing the included GNU toolchain, a mechanism for providing an alternative/secondary toolchain is desired.

== Requirements ==

'''Secondary Toolchains should be implemented in a layer'''

In order to facilitate re-use and make it easier to completely disable the secondary toolchain, a layer must be used to implement the secondary toolchain.

'''Secondary Toolchain must be generally compatible with GNU toolchain'''

The secondary toolchain must be generally compatible with the idea of sysroots, linking to the defined libc, and various GNU conventions. This compatibility may be implemented directly by the secondary toolchain or via a wrapper of some type.

'''Per recipe selection of toolchain usage'''

A mechanism is needed to activate this alternative toolchain on a per-recipe basis. Using variables it should be possible to specify on a per-recipe basis which toolchain should be used, leaving the default up to the user. (It's assumed if the user does not select a default, the system GNU toolchain will be defaulted.)

'''Blacklist/Whitelist'''

Even with the requirement to be generally compatible, we can not expect all of the possible recipes to build properly with the secondary toolchain. As a consequence of this, it will be necessary to implement a blacklist (or whitelist) of specific recipes that are known to work. This way only supported components can be build by the user, avoid numerous complaints and unresolvable defects.

'''SDK'''

The SDK generated by OpenEmbedded-Core also includes the included toolchain components. It would be useful to also provide support for the secondary toolchain within the SDK environment.

== Implementation ==

=== Bitbake variables ===

A set of common bitbake variables, defined in meta/conf/bitbake.conf, are used to define how to run the toolchain components and the proper arguments.

A number of key items may need to be changed, depending on the toolchain being provided. These items may include:

* TARGET_CPPFLAGS
* TARGET_CFLAGS
* TARGET_CXXFLAGS
* TARGET_LDFLAGS
* CC
* CXX
* F77
* CPP
* LD
* CCLD
* AR
* AS
* RANLIB
* STRIP
* OBJCOPY
* OBJDUMP
* NM
* SELECTED_OPTIMIZATION
* FULL_OPTIMIZATION
* DEBUG_OPTIMIZATION

Each of the above items has a default definition within the meta/conf/bitbake.conf file. In addition, many of the components are based on specific tune variables such as:

* TUNE_CCARGS
* TUNE_LDARGS
* TUNE_ASARGS

Deciding which items need to be overridden is up to the author of the layer. It is expected that only a subset of the above referenced variables will need secondary toolchain specific versions for most implementations.

It is suggested that the secondary toolchain values be defined in a single configuration file. Within the example this is "conf/tc-example.conf". The standard name for the file should be 'conf/tc-<toolchain>.conf'.

=== Override ===

In order to allow for the layer to selectively override the variables listed above, a standard override syntax is needed.

The standard override syntax will be 'toolchain-${TOOLCHAIN}'. This will allow us to specify multiple alternative and primary toolchain combinations as necessary.

Each of the toolchain related variables can then be overridden using the standard override syntax: <var>_toolchain-<toolchain>, such as:

CC_toolchain-icc = "icc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"

In order for an individual recipe to use the secondary toolchain, the user would define: TOOLCHAIN_pn-<recipe> = 'toolchain', such as:

TOOLCHAIN_pn-bash = "icc"

The OVERRIDE is implemented in the tc-secondary.bbclass in the example.

=== Blacklist/Whitelist ===

A standard blacklist/whitelist facility is to be implemented. This facility is independent of the existing blacklist class that is part of OpenEmbedded-Core.

Similar to the existing blacklist class, you will be able to define a blacklist, per toolchain type, such as:

TCBLACKLIST[pn] = "toolchain"

or alternatively create a whitelist facility using:

TCBLACKLIST = "toolchain"
TCWHITELIST[pn] = "toolchain"

Using the second approach will allow you to blacklist all recipes, and only enable those known to work with the alternative toolchain.

The blacklist items should be defined within conf/tc-<toolchain>-blacklist.conf. The validation of the blacklist should use the standard tc-blacklist.bbclass implementation available within the example.

=== SDK ===

TBD: Add SDK extension details

Notes:

* need to add a custom toolchain_create_sdk_env_script for the secondary toolchain(s) (supporting override).
* need to support more then one secondary toolchain
* populate_sdk_image_append -- define a python class to dump the secondary toolchain bits...

== Implementation Example ==

An example implementation has been created in the layer "meta-tc-example". The file format follows:

meta-tc-example/
meta-tc-example/bin
meta-tc-example/bin/example-toolchain
meta-tc-example/conf
meta-tc-example/conf/layer.conf
meta-tc-example/conf/tc-example.conf
meta-tc-example/conf/tc-example-blacklist.conf
meta-tc-example/classes
meta-tc-example/classes/tc-secondary.bbclass
meta-tc-example/classes/tc-blacklist.bbclass

=== conf/layer.conf ===

# We have a conf and classes directory, append to BBPATH
BBPATH .= ":${LAYERDIR}"

BBFILE_COLLECTIONS += "tc-example"

# If we have a recipes directory, add to BBFILES
#BBFILES += "${LAYERDIR}/recipes-*/*/*.bb ${LAYERDIR}/*/recipes-*/*/*.bbappend"
#BBFILE_COLLECTIONS += "tc-example"
#BBFILE_PATTERN_tc-example := "^${LAYERDIR}/"
#BBFILE_PRIORITY_tc-example = "5"

Above is standard layer configuration. Note, the commented out sections should be re-enabled if the secondary toolchain includes recipes and/or bbappend files.

PATH := "${PATH}:${LAYERDIR}/bin"

Add to the path a standard system path, a layer specific path. This is the way you can add a secondary toolchain to the path.

# Define the alternative toolchain
SECONDARYTC = 'example'
require ${LAYERDIR}/conf/tc-${SECONDARYTC}.conf
include ${LAYERDIR}/conf/tc-${SECONDARYTC}-blacklist.conf

Define the alternative toolchain name in SECONDARYTC. This is then used to bring in the two configuration files. The first tc-example.conf is required, while the tc-example-blacklist.conf is optional.

=== conf/tc-example.conf ===

# Enable the override:
INHERIT += 'tc-secondary'

This inherit is mandatory and adds the secondary toolchain control class. All secondary toolchains should contain the same class.

# Define default system toolchain (default to built-in)
TOOLCHAIN = 'gnu'

Define a reasonable default such as gnu. The purpose of this default is to allow someone to specify the default or provide default override items for the built-in toolchain.

# Define a secondary toolchain called 'example'
# the 'example' toolchain can be activated using:
#
# Per package (preferred):
# TOOLCHAIN_pn-bash = 'example'
#
# Globally:
# TOOLCHAIN = 'example'
# TOOLCHAIN_pn-eglibc = 'gnu'

The above is an example of how to use the alternative toolchain.

# When the secondary toolchain is used, we add an automatic depend...
DEPENDS_append_toolchain-example = " virtual/TOOLCHAIN-EXAMPLE"

# ...the depend can be provided by a recipe or as an ASSUME_PROVIDED
ASSUME_PROVIDED_append = " virtual/TOOLCHAIN-EXAMPLE"

Use the above as an example on how to add additional dependencies to specific packages. This can be used to trigger a recipe to build.

# Default values from bitbake.conf...
# Variables with a '*' are things likely to need to be overridden:
#
# FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
# DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe"
# SELECTED_OPTIMIZATION = "${@d.getVar(['FULL_OPTIMIZATION', 'DEBUG_OPTIMIZATION'][d.getVar('DEBUG_BUILD', True) == '1'], True)}"
#
# * TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
#
# * TARGET_CPPFLAGS = ""
# * TARGET_CFLAGS = "${TARGET_CPPFLAGS} ${SELECTED_OPTIMIZATION}"
# * TARGET CXXFLAGS = "${TARGET_CFLAGS} -fpermissive"
# * TARGET_LDFLAGS = "-Wl,-O1 ${TARGET_LINK_HASH_STYLE}"
#
# TARGET_CC_ARCH = "${TUNE_CCARGS}"
# TARGET_LD_ARCH = "${TUNE_LDARGS}"
# TARGET_AS_ARCH = "${TUNE_ASARGS}"
#
# HOST_CC_ARCH = "${TARGET_CC_ARCH}"
# HOST_LD_ARCH = "${TARGET_LD_ARCH}"
# HOST_AS_ARCH = "${TARGET_AS_ARCH}"
#
# * CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CXX = "${CCACHE}${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * F77 = "${CCACHE}${HOST_PREFIX}g77 ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CPP = "${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
# * LD = "${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"
# * CCLD = "${CC}"
# AR = "${HOST_PREFIX}ar"
# AS = "${HOST_PREFIX}as ${HOST_AS_ARCH}"
# RANLIB = "${HOST_PREFIX}ranlib"
# STRIP = "${HOST_PREFIX}strip"
# OBJCOPY = "${HOST_PREFIX}objcopy"
# OBJDUMP = "${HOST_PREFIX}objdump"
# NM = "${HOST_PREFIX}nm"

# Override values:
export CC_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
export CXX_toolchain-example = "example-toolchain ${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
export CPP_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
export LD_toolchain-example = "example-toolchain ${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"

The above implements the necessary overrides for the example toolchain.

=== conf/tc-example-blacklist.conf ===

Example configuration file that enables the blacklist and defines the defaults.

INHERIT += 'tc-blacklist'

# Blacklist certain items...
TCBLACKLIST[eglibc] = 'example'
TCBLACKLIST[bash] = 'example'

#
# or
#

# Switch to a global blacklist and whitelist situation...
# TCBLACKLIST = 'example'
# TCWHITELIST[bash] = 'example'

=== classes/tc-secondary.bbclass ===

This is the standard secondary toolchain class. It should be the same in all secondary toolchain layers.

# In order to add overrides, we need to do it later then in the layers.conf
# the only standard way is using an inherit, which requires a bbclass

# Add the necessary override
TOOLCHAINOVERRIDES = ":toolchain-${TOOLCHAIN}"
TOOLCHAINOVERRIDES[vardepsexclude] = "TOOLCHAIN"

OVERRIDES .= "${TOOLCHAINOVERRIDES}"
OVERRIDES[vardepsexclude] += "TOOLCHAINOVERRIDES"

=== classes/tc-blacklist.bbclass ===

This is the standard secondary toolchain blacklist class. It should be the same in all secondary toolchain layers.

# Implement blacklist/whitelist functionality for alternative toolchains
# Based on the oe-core blacklist bclass
#
# To add a blacklisted package:
# TCBLACKLIST[pn] = 'toolchain'
#
# To blacklist everything, and whitelist a package:
# TCBLACKLIST = 'toolchain'
# TCWHITELIST[pn] = 'toolchain'

python () {
tc = d.getVar('TOOLCHAIN', True)
pn = d.getVar('PN', True)

blacklist = d.getVarFlag('TCBLACKLIST', pn, True)

if blacklist and blacklist == tc:
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))

blacklist = d.getVar('TCBLACKLIST', True)
whitelist = d.getVarFlag('TCWHITELIST', pn, True)

if blacklist and (not whitelist or whitelist != tc):
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))
}

== Further comments and information ==

[[Category:Dev]]
[[Category:Layers]]

Adding a secondary toolchain

2012-11-13T23:16:04Z

Mhatle: /* Implementation Example */

== Background ==

In OpenEmbedded-Core there in an included toolchain that is built up of GNU binutils, gcc, and eglibc. In addition related components such as gdb, cross-prelink, mklibs and others provider additional toolchain functionality. In this document we focus on explaining best practices for including an additional toolchain that is capable of assembling, compiling and linking code. This document does not cover replacing the existing toolchain or toolchain components with external elements.

Many companies have commercial / highly optimized toolchains for specific purposes, such as ICC from Intel, LLVM, ARM Ltd toolchains, etc.. In some cases, these highly optimized toolchains may result in better performance for some applications, however they are often not generally applicable as they can not compile all of the system software. So instead of replacing the included GNU toolchain, a mechanism for providing an alternative/secondary toolchain is desired.

== Requirements ==

'''Secondary Toolchains should be implemented in a layer'''

In order to facilitate re-use and make it easier to completely disable the secondary toolchain, a layer must be used to implement the secondary toolchain.

'''Secondary Toolchain must be generally compatible with GNU toolchain'''

The secondary toolchain must be generally compatible with the idea of sysroots, linking to the defined libc, and various GNU conventions. This compatibility may be implemented directly by the secondary toolchain or via a wrapper of some type.

'''Per recipe selection of toolchain usage'''

A mechanism is needed to activate this alternative toolchain on a per-recipe basis. Using variables it should be possible to specify on a per-recipe basis which toolchain should be used, leaving the default up to the user. (It's assumed if the user does not select a default, the system GNU toolchain will be defaulted.)

'''Blacklist/Whitelist'''

Even with the requirement to be generally compatible, we can not expect all of the possible recipes to build properly with the secondary toolchain. As a consequence of this, it will be necessary to implement a blacklist (or whitelist) of specific recipes that are known to work. This way only supported components can be build by the user, avoid numerous complaints and unresolvable defects.

'''SDK'''

The SDK generated by OpenEmbedded-Core also includes the included toolchain components. It would be useful to also provide support for the secondary toolchain within the SDK environment.

== Implementation ==

=== Bitbake variables ===

A set of common bitbake variables, defined in meta/conf/bitbake.conf, are used to define how to run the toolchain components and the proper arguments.

A number of key items may need to be changed, depending on the toolchain being provided. These items may include:

* TARGET_CPPFLAGS
* TARGET_CFLAGS
* TARGET_CXXFLAGS
* TARGET_LDFLAGS
* CC
* CXX
* F77
* CPP
* LD
* CCLD
* AR
* AS
* RANLIB
* STRIP
* OBJCOPY
* OBJDUMP
* NM
* SELECTED_OPTIMIZATION
* FULL_OPTIMIZATION
* DEBUG_OPTIMIZATION

Each of the above items has a default definition within the meta/conf/bitbake.conf file. In addition, many of the components are based on specific tune variables such as:

* TUNE_CCARGS
* TUNE_LDARGS
* TUNE_ASARGS

Deciding which items need to be overridden is up to the author of the layer. It is expected that only a subset of the above referenced variables will need secondary toolchain specific versions for most implementations.

It is suggested that the secondary toolchain values be defined in a single configuration file. Within the example this is "conf/tc-example.conf". The standard name for the file should be 'conf/tc-<toolchain>.conf'.

=== Override ===

In order to allow for the layer to selectively override the variables listed above, a standard override syntax is needed.

The standard override syntax will be 'toolchain-${TOOLCHAIN}'. This will allow us to specify multiple alternative and primary toolchain combinations as necessary.

Each of the toolchain related variables can then be overridden using the standard override syntax: <var>_toolchain-<toolchain>, such as:

CC_toolchain-icc = "icc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"

In order for an individual recipe to use the secondary toolchain, the user would define: TOOLCHAIN_pn-<recipe> = 'toolchain', such as:

TOOLCHAIN_pn-bash = "icc"

The OVERRIDE is implemented in the tc-secondary.bbclass in the example.

=== Blacklist/Whitelist ===

A standard blacklist/whitelist facility is to be implemented. This facility is independent of the existing blacklist class that is part of OpenEmbedded-Core.

Similar to the existing blacklist class, you will be able to define a blacklist, per toolchain type, such as:

TCBLACKLIST[pn] = "toolchain"

or alternatively create a whitelist facility using:

TCBLACKLIST = "toolchain"
TCWHITELIST[pn] = "toolchain"

Using the second approach will allow you to blacklist all recipes, and only enable those known to work with the alternative toolchain.

The blacklist items should be defined within conf/tc-<toolchain>-blacklist.conf. The validation of the blacklist should use the standard tc-blacklist.bbclass implementation available within the example.

=== SDK ===

MGH: Add SDK extension details

== Implementation Example ==

An example implementation has been created in the layer "meta-tc-example". The file format follows:

meta-tc-example/
meta-tc-example/bin
meta-tc-example/bin/example-toolchain
meta-tc-example/conf
meta-tc-example/conf/layer.conf
meta-tc-example/conf/tc-example.conf
meta-tc-example/conf/tc-example-blacklist.conf
meta-tc-example/classes
meta-tc-example/classes/tc-secondary.bbclass
meta-tc-example/classes/tc-blacklist.bbclass

=== conf/layer.conf ===

# We have a conf and classes directory, append to BBPATH
BBPATH .= ":${LAYERDIR}"

BBFILE_COLLECTIONS += "tc-example"

# If we have a recipes directory, add to BBFILES
#BBFILES += "${LAYERDIR}/recipes-*/*/*.bb ${LAYERDIR}/*/recipes-*/*/*.bbappend"
#BBFILE_COLLECTIONS += "tc-example"
#BBFILE_PATTERN_tc-example := "^${LAYERDIR}/"
#BBFILE_PRIORITY_tc-example = "5"

Above is standard layer configuration. Note, the commented out sections should be re-enabled if the secondary toolchain includes recipes and/or bbappend files.

PATH := "${PATH}:${LAYERDIR}/bin"

Add to the path a standard system path, a layer specific path. This is the way you can add a secondary toolchain to the path.

# Define the alternative toolchain
SECONDARYTC = 'example'
require ${LAYERDIR}/conf/tc-${SECONDARYTC}.conf
include ${LAYERDIR}/conf/tc-${SECONDARYTC}-blacklist.conf

Define the alternative toolchain name in SECONDARYTC. This is then used to bring in the two configuration files. The first tc-example.conf is required, while the tc-example-blacklist.conf is optional.

=== conf/tc-example.conf ===

# Enable the override:
INHERIT += 'tc-secondary'

This inherit is mandatory and adds the secondary toolchain control class. All secondary toolchains should contain the same class.

# Define default system toolchain (default to built-in)
TOOLCHAIN = 'gnu'

Define a reasonable default such as gnu. The purpose of this default is to allow someone to specify the default or provide default override items for the built-in toolchain.

# Define a secondary toolchain called 'example'
# the 'example' toolchain can be activated using:
#
# Per package (preferred):
# TOOLCHAIN_pn-bash = 'example'
#
# Globally:
# TOOLCHAIN = 'example'
# TOOLCHAIN_pn-eglibc = 'gnu'

The above is an example of how to use the alternative toolchain.

# When the secondary toolchain is used, we add an automatic depend...
DEPENDS_append_toolchain-example = " virtual/TOOLCHAIN-EXAMPLE"

# ...the depend can be provided by a recipe or as an ASSUME_PROVIDED
ASSUME_PROVIDED_append = " virtual/TOOLCHAIN-EXAMPLE"

Use the above as an example on how to add additional dependencies to specific packages. This can be used to trigger a recipe to build.

# Default values from bitbake.conf...
# Variables with a '*' are things likely to need to be overridden:
#
# FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
# DEBUG_OPTIMIZATION = "-O -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe"
# SELECTED_OPTIMIZATION = "${@d.getVar(['FULL_OPTIMIZATION', 'DEBUG_OPTIMIZATION'][d.getVar('DEBUG_BUILD', True) == '1'], True)}"
#
# * TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
#
# * TARGET_CPPFLAGS = ""
# * TARGET_CFLAGS = "${TARGET_CPPFLAGS} ${SELECTED_OPTIMIZATION}"
# * TARGET CXXFLAGS = "${TARGET_CFLAGS} -fpermissive"
# * TARGET_LDFLAGS = "-Wl,-O1 ${TARGET_LINK_HASH_STYLE}"
#
# TARGET_CC_ARCH = "${TUNE_CCARGS}"
# TARGET_LD_ARCH = "${TUNE_LDARGS}"
# TARGET_AS_ARCH = "${TUNE_ASARGS}"
#
# HOST_CC_ARCH = "${TARGET_CC_ARCH}"
# HOST_LD_ARCH = "${TARGET_LD_ARCH}"
# HOST_AS_ARCH = "${TARGET_AS_ARCH}"
#
# * CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CXX = "${CCACHE}${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * F77 = "${CCACHE}${HOST_PREFIX}g77 ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
# * CPP = "${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
# * LD = "${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"
# * CCLD = "${CC}"
# AR = "${HOST_PREFIX}ar"
# AS = "${HOST_PREFIX}as ${HOST_AS_ARCH}"
# RANLIB = "${HOST_PREFIX}ranlib"
# STRIP = "${HOST_PREFIX}strip"
# OBJCOPY = "${HOST_PREFIX}objcopy"
# OBJDUMP = "${HOST_PREFIX}objdump"
# NM = "${HOST_PREFIX}nm"

# Override values:
export CC_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
export CXX_toolchain-example = "example-toolchain ${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
export CPP_toolchain-example = "example-toolchain ${HOST_PREFIX}gcc -E${TOOLCHAIN_OPTIONS} ${HOST_CC_ARCH}"
export LD_toolchain-example = "example-toolchain ${HOST_PREFIX}ld${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"

The above implements the necessary overrides for the example toolchain.

=== conf/tc-example-blacklist.conf ===

Example configuration file that enables the blacklist and defines the defaults.

INHERIT += 'tc-blacklist'

# Blacklist certain items...
TCBLACKLIST[eglibc] = 'example'
TCBLACKLIST[bash] = 'example'

#
# or
#

# Switch to a global blacklist and whitelist situation...
# TCBLACKLIST = 'example'
# TCWHITELIST[bash] = 'example'

=== classes/tc-secondary.bbclass ===

This is the standard secondary toolchain class. It should be the same in all secondary toolchain layers.

# In order to add overrides, we need to do it later then in the layers.conf
# the only standard way is using an inherit, which requires a bbclass

# Add the necessary override
TOOLCHAINOVERRIDES = ":toolchain-${TOOLCHAIN}"
TOOLCHAINOVERRIDES[vardepsexclude] = "TOOLCHAIN"

OVERRIDES .= "${TOOLCHAINOVERRIDES}"
OVERRIDES[vardepsexclude] += "TOOLCHAINOVERRIDES"

=== classes/tc-blacklist.bbclass ===

This is the standard secondary toolchain blacklist class. It should be the same in all secondary toolchain layers.

# Implement blacklist/whitelist functionality for alternative toolchains
# Based on the oe-core blacklist bclass
#
# To add a blacklisted package:
# TCBLACKLIST[pn] = 'toolchain'
#
# To blacklist everything, and whitelist a package:
# TCBLACKLIST = 'toolchain'
# TCWHITELIST[pn] = 'toolchain'

python () {
tc = d.getVar('TOOLCHAIN', True)
pn = d.getVar('PN', True)

blacklist = d.getVarFlag('TCBLACKLIST', pn, True)

if blacklist and blacklist == tc:
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))

blacklist = d.getVar('TCBLACKLIST', True)
whitelist = d.getVarFlag('TCWHITELIST', pn, True)

if blacklist and (not whitelist or whitelist != tc):
raise bb.parse.SkipPackage("Recipe '%s' is blacklisted for the '%s' toolchain" % (pn, tc))
}

== Further comments and information ==

[[Category:Dev]]
[[Category:Layers]]

Adding a secondary toolchain

2012-11-13T22:48:01Z

Mhatle: /* Blacklist/Whitelist */

Adding a secondary toolchain

2012-11-13T22:47:31Z

Mhatle: /* Blacklist/Whitelist */

Adding a secondary toolchain

2012-11-13T22:45:16Z

Mhatle: /* Bitbake variables */

Adding a secondary toolchain

2012-11-13T22:39:58Z

Mhatle: /* Override */

Adding a secondary toolchain

2012-11-13T22:39:03Z

Mhatle: /* Blacklist/Whitelist */

Adding a secondary toolchain

2012-11-13T19:10:25Z

Mhatle: /* Implementation */

Adding a secondary toolchain

2012-11-13T18:55:53Z

Mhatle: /* Implementation */

Adding a secondary toolchain

2012-11-13T18:31:52Z

Mhatle:

Adding a secondary toolchain

2012-11-13T18:07:24Z

Mhatle: How to add an alternative/secondary toolchain to or-core

Test

Barcelona, 2012

2012-10-30T14:52:11Z

Mhatle: /* Participants */

==Location ==

Hotel Fira Palace · Barcelona, Spain
Nov 7, 2012 at 17:00.

== Agenda ==

* Financial report
* Board Elections, Dr. Michael Lauer, Florian Boor, and Philip Balister's terms are ending.

== Participants ==

The sign [P] means he/she is available as proxy.

* Marco Cavallini [P]
* Eric Bénard [P]
* Mark Hatle (fray) [P]

== Forms ==

* Proxy form [[File:Proxy_instructions-oe.pdf]]

Commit Patch Message Guidelines

2011-06-14T18:39:21Z

Mhatle:

This set of guidelines is intended to help both the developer and reviewers of
changes determine reasonable patch headers and change commit messages. Often
when working with the code, you forget that not everyone is as familiar with the
problem and/or fix as you are. Often the next person in the code doesn't
understand what or why something is done so they quickly look at patch header
and commit messages. Unless these messages are clear it will be difficult to
understand the relevance of a given change and how future changes may impact
previous decisions.

This policy refers both to patches that are being applied by recipes as well as
commit messages that are part of the source control system, usually git. A patch
file needs a header in order to describe the specific changes that are made
within that patch, while a commit message describes one or more changes to the
overall project or system. Both the patch headers and commit messages require
the same attention and basic details, however the purposes of the messages are
slightly different. A commit message documents all of the changes made as part
of a commit, while a patch header documents items specific to a single patch. In
many cases the patch header can also be used as the commit message.

This policy does not cover the testing of the changes, or the technical criteria
for accepting a patch.

By following these guidelines we will have a better record of the problems and
solutions made over the course of development. It will also help establish a
clear provenance of all of the code and changes.

''Approved by the Technical Steering Committee 2011/06/09.''

== General Information ==

While specific to the Linux kernel development, the following could also be
considered a general guide for any Open Source development:

http://ldn.linuxfoundation.org/book/how-participate-linux-community

Many of the guidelines in this document are related to the items in that
information.

Pay particular attention to section 5.3 that talks about patch preparation.
The key thing to remember is to break up your changes into logical sections.
Otherwise you run the risk of not being able to even explain the purpose of a
change in the patch headers!

In addition section 5.4 explains the purpose of the Signed-off-by: tag line as
discussed in later parts of this document. Due to the importance of the
Signed-off-by: tag line a copy of the description follows:

Signed-off-by: this is a developer's certification that he or she has
the right to submit the patch for inclusion into the [project]. It is
an agreement to the Developer's Certificate of Origin, the full text of
which can be found in [Linux Kernel] Documentation/SubmittingPatches.
Code without a proper signoff cannot be merged into the mainline.

For more information on the Developer's Certificate of Origin and the Linux
Kernel Documentation/SubmittingPatches, see:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/SubmittingPatches

== Patch Headers and Commit Messages ==

There seems to always be a question or two surrounding what a person
should put in a patch header, or commit message.

The general rules always apply, those being that there is a single line
short log or summary of the change (think of this as the subject when the patch
is e-mailed), and then the more detailed long log, and closure with tag lines
such as "Signed-off-by:".

== New Development ==

A minimal patch or commit message would be of the format:

Short log / Statement of what needed to be changed.

(Optional pointers to external resources, such as defect tracking)

The intent of your change.

(Optional, if it's not clear from above) how your change resolves the
issues in the first part.

Tag line(s) at the end.

For example:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The minimal commit message is good for new code development and simple changes.

The single short log message indicates what needed to be changed. It should
begin with an indicator as to the primary item changed by this work,
followed by a short summary of the change. In the above case we're indicating
that we've changed the "foobar" item, by "adjusting the foo setting in bar".

The single short log message is analogous to the git "commit summary". While no
maximum line length is specified by this policy, it is suggested that it remains
under 78 characters wherever possible.

Optionally, you may include pointers to defects this change corrects. Unless
the defect format is specified by the component you are modifying, it is
suggested that you use a full URL to specify the reference to the defect
information. Generally these pointers will precede any long description, but as
an optional item it may be after the long description.

For example, you might refer to an open source defect in the RPM project:

rpm: Adjusted the foo setting in bar

<nowiki>[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5</nowiki>

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

You must then have a full description of the change. Specifying the intent of
your change and if necessary how the change resolves the issue.

As mentioned above this is intended to answer the "what were you thinking"
questions down the road and to know what other impacts are likely to
result of the change needs to be reverted. It also allows for better
solutions if someone comes along later and agrees with paragraph 1
(the problem statement) and either disagrees with paragraph 2
(the design) or paragraph 3 (the implementation).

Finally one or more tag lines should exist. Each developer responsible
for working on the patch is responsible for adding a Signed-off-by: tag line.

It is not acceptable to have an empty or non-existent header, or just a single
line message. The summary and description is required for all changes.

== Importing from Elsewhere ==

If you are importing work from somewhere else, such as a cherry-pick from
another repository or a code patch pulled from a mailing list, the minimum patch
header or commit message is not enough. It does not clearly establish the
provenance of the code.

The following specifies the additional guidelines required for importing changes
from elsewhere.

By default you should keep the original author's summary and description,
along with any tag lines such as, "Signed-off-by:". If the original change that
was imported does not have a summary and/or commit message from the original
author, it is still your responsibility to add the summary and commit message to
the patch header. Just as if you wrote the code, you should be able to clearly
explain what the change does. It is also necessary to document the original
author of the change. You should indicate the original author by simply stating
"written by" or "posted to the ... mailing list by". You must not add a
Signed-off-by: tag line with their name, only the original author may add their
own Signed-off-by: tag line.

It is also required that the origin of the work be fully documented. The origin
should be specified as part of the commit message in a way that an average
developer would be able to track down the original code. URLs should reference
original authoritative sources and not mirrors.

If changes were required to resolve conflicts, they should be documented as
well. When incorporating a commit or patch from an external source, changes to
the functionality not related to resolving conflicts should be made in a
second commit or patch. This preserves the original external commit, and makes
the modifications clearly visible, and prevents confusion over the author of the
code.

Finally you must add a "Signed-off-by:" tag line to indicate that you worked
with the patch.

== Example: Importing from Elsewhere Unmodified ==

For example, if you were to pull in the patch from the above example unmodified:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The patch was imported from the OpenEmbedded git server
(<nowiki>git://git.openembedded.org/openembedded</nowiki>) as of commit id
b65a0e0c84cf489bfa00d6aa6c48abc5a237100f.

Signed-off-by: Your Name <your.name@openembedded.org>

The above example shows a clear way for a developer to find the original source
of the change. It indicates that the OpenEmbedded developer simply integrated
the change into the system without making any further modifications.

== Example: Importing from Elsewhere Modified ==

When importing a patch, occasionally changes have to be made in order to resolve
conflicts. The following is an example of the commit message when the item needed
to be modified:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The patch was imported from the OpenEmbedded git server
(<nowiki>git://git.openembedded.org/openembedded</nowiki>) as of commit id
b65a0e0c84cf489bfa00d6aa6c48abc5a237100f.

A previous change had modified the memory threshold calculation,
causing a conflict. The conflict was resolved by preserving the
memory threshold calculation from the existing implementation.

Signed-off-by: Your Name <your.name@openembedded.org>

== Patch Header Recommendations ==

In order to keep track of patches and ultimately reduce the number of patches
that are required to be maintained, we need to track the status of the patches
that are created. The following items are specific to patches applied by recipes.

In addition to the items mentioned above, we also want to track if it's
appropriate to get this particular patch into the upstream project. Since we
want to track this information, we need a consistent tag and status indicator
that can be searched.

While adding this tracking information to the patch headers is currently
optional, it is highly recommended and some maintainers may require it. It is
optional at this time so that it can be evaluated as to its usefulness over
time. Existing patches will be updated with the tag as they are modified.

As mentioned in the above, be sure to include any URL to bug tracking, mailing
lists or other reference material pertaining to the patch. Then add a new tag
"Upstream-Status:" which contains one of the following items:

'''Pending'''
- No determination has been made yet or not yet submitted to upstream

'''Submitted''' [where]
- Submitted to upstream, waiting approval
- Optionally include where it was submitted, such as the author, mailing
list, etc.

'''Accepted'''
- Accepted in upstream, expect it to be removed at next update, include
expected version info

'''Backport'''
- Backported from new upstream version, because we are at a fixed version,
include upstream version info

'''Denied'''
- Not accepted by upstream, include reason in patch

'''Inappropriate [reason]'''
- The patch is not appropriate for upstream, include a brief reason on the
same line enclosed with []
reason can be:
not author (You are not the author and do not intend to upstream this,
source must be listed in the comments)
native
licensing
configuration
enable feature
disable feature
bugfix (add bug URL here)
embedded specific
no upstream (the upstream is no longer available -- dead project)
other (give details in comments)

The various "Inappropriate [reason]" status items are meant to indicate that the
person responsible for adding this patch to the system does not intend to
upstream the patch for a specific reason. Unless otherwise noted, another
person could submit this patch to the upstream, if they do the status should be
changed to "Submitted [where]", and an additional Signed-off-by: line added to
the patch by the person claiming responsibility for upstreaming.

For example, if the patch has been submitted upstream:

rpm: Adjusted the foo setting in bar

<nowiki>[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5</nowiki>

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Upstream-Status: Submitted [rpm5-devel@rpm5.org]

Signed-off-by: Joe Developer <joe.developer@example.com>

A future commit can change the value to "Accepted" or "Denied" as appropriate.

== Common Errors in Patch and Commit Messages ==

- Short log does not start with the file or component being modified. Such as
"foo: Update to new upstream version 5.8.9"

- Avoid starting the summary line with a capital letter, unless the component
being referred to also begins with a capital letter.

- Don't have one huge patch, split your change into logical subparts. It's
easier to track down problems afterward using tools such as git bisect. It also
makes it easy for people to cherry-pick changes into things like stable branches.

- Don't simply translate your change into English for a commit log. The log
"Change compare from zero to one" is bad because it describes only the code
change in the patch; we can see that from reading the patch itself. Let the code
tell the story of the mechanics of the change (as much as possible), and let
your comment tell the other details -- i.e. what the problem was, how it
manifested itself (symptoms), and if necessary, the justification for why the
fix was done in manner that it was. In other words, the long commit message
must describe *why* the change was needed (instead of what has changed).

- Don't start your commit log with "This patch..." -- we already know it is a patch.

- Don't repeat your short log in the long log. If you really really don't have
anything new and informational to add in as a long log, then don't put a long
log at all. This should be uncommon -- i.e. the only acceptable cases for no
long log would be something like "Documentation/README: Fix spelling mistakes".

- Don't put absolute paths to source files that are specific to your site, i.e.
if you get a compile error on "fs/exec.c" then tidy up the parts of the compile
output to just show that portion. We don't need to see that it was
/home/wally/src/git/oe-core/meta/classes/insane.bbclass or similar - that full
path has no value to others. Always reduce the path to something more
meaningful, such as oe-core/meta/classes/insane.bbclass.

- Always use the most significant ramification of the change in the words of
your subject/shortlog. For example, don't say "fix compile warning in foo" when
the compiler warning was really telling us that we were dereferencing complete
garbage off in the weeds that could in theory cause an OOPS under some
circumstances. When people are choosing commits for backports to stable or
distro kernels, the shortlog will be what they use for an initial sorting
selection. If they see "Fix possible OOPS in...." then these people will look
closer, whereas they most likely will skip over simple warning fixes.

- Don't put in the full 20 or more lines of a backtrace when really it is just
the last 5 or so function calls that are relevant to the crash/fix. If the entry
point, or start of the trace is relevant (i.e. a syscall or similar), you can
leave that, and then replace a chunk of the intermediate calls in the middle
with a single [...]

- Don't include timestamps or other unnecessary information, unless they are
relevant to the situation (i.e. indicating an unacceptable delay in a driver
initialization etc.)

- Don't use links to temporary resources like pastebin and friends. The commit
message may be read long after this resource timed out.

- Don't reference mirrors, but instead always reference original authoritative
sources.

- Avoid punctuation in the short log.

[[Category:Policy]]

User:Mhatle

2011-06-14T18:36:24Z

Mhatle: Created page with "Name: Mark Hatle IRC Alias: fray"

Name: Mark Hatle

IRC Alias: fray

Commit Patch Message Guidelines

2011-06-14T18:32:55Z

Mhatle:

This set of guidelines is intended to help both the developer and reviewers of
changes determine reasonable patch headers and change commit messages. Often
when working with the code, you forget that not everyone is as familiar with the
problem and/or fix as you are. Often the next person in the code doesn't
understand what or why something is done so they quickly look at patch header
and commit messages. Unless these messages are clear it will be difficult to
understand the relevance of a given change and how future changes may impact
previous decisions.

This policy refers both to patches that are being applied by recipes as well as
commit messages that are part of the source control system, usually git. A patch
file needs a header in order to describe the specific changes that are made
within that patch, while a commit message describes one or more changes to the
overall project or system. Both the patch headers and commit messages require
the same attention and basic details, however the purposes of the messages are
slightly different. A commit message documents all of the changes made as part
of a commit, while a patch header documents items specific to a single patch. In
many cases the patch header can also be used as the commit message.

This policy does not cover the testing of the changes, or the technical criteria
for accepting a patch.

By following these guidelines we will have a better record of the problems and
solutions made over the course of development. It will also help establish a
clear provenance of all of the code and changes.

''Approved by the Technical Steering Committee 2011/06/09.''

== General Information ==

While specific to the Linux kernel development, the following could also be
considered a general guide for any Open Source development:

http://ldn.linuxfoundation.org/book/how-participate-linux-community

Many of the guidelines in this document are related to the items in that
information.

Pay particular attention to section 5.3 that talks about patch preparation.
The key thing to remember is to break up your changes into logical sections.
Otherwise you run the risk of not being able to even explain the purpose of a
change in the patch headers!

In addition section 5.4 explains the purpose of the Signed-off-by: tag line as
discussed in later parts of this document. Due to the importance of the
Signed-off-by: tag line a copy of the description follows:

Signed-off-by: this is a developer's certification that he or she has
the right to submit the patch for inclusion into the [project]. It is
an agreement to the Developer's Certificate of Origin, the full text of
which can be found in [Linux Kernel] Documentation/SubmittingPatches.
Code without a proper signoff cannot be merged into the mainline.

For more information on the Developer's Certificate of Origin and the Linux
Kernel Documentation/SubmittingPatches, see:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/SubmittingPatches

== Patch Headers and Commit Messages ==

There seems to always be a question or two surrounding what a person
should put in a patch header, or commit message.

The general rules always apply, those being that there is a single line
short log or summary of the change (think of this as the subject when the patch
is e-mailed), and then the more detailed long log, and closure with tag lines
such as "Signed-off-by:".

== New Development ==

A minimal patch or commit message would be of the format:

Short log / Statement of what needed to be changed.

(Optional pointers to external resources, such as defect tracking)

The intent of your change.

(Optional, if it's not clear from above) how your change resolves the
issues in the first part.

Tag line(s) at the end.

For example:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The minimal commit message is good for new code development and simple changes.

The single short log message indicates what needed to be changed. It should
begin with an indicator as to the primary item changed by this work,
followed by a short summary of the change. In the above case we're indicating
that we've changed the "foobar" item, by "adjusting the foo setting in bar".

The single short log message is analogous to the git "commit summary". While no
maximum line length is specified by this policy, it is suggested that it remains
under 78 characters wherever possible.

Optionally, you may include pointers to defects this change corrects. Unless
the defect format is specified by the component you are modifying, it is
suggested that you use a full URL to specify the reference to the defect
information. Generally these pointers will precede any long description, but as
an optional item it may be after the long description.

For example, you might refer to an open source defect in the RPM project:

rpm: Adjusted the foo setting in bar

<nowiki>[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5</nowiki>

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

You must then have a full description of the change. Specifying the intent of
your change and if necessary how the change resolves the issue.

As mentioned above this is intended to answer the "what were you thinking"
questions down the road and to know what other impacts are likely to
result of the change needs to be reverted. It also allows for better
solutions if someone comes along later and agrees with paragraph 1
(the problem statement) and either disagrees with paragraph 2
(the design) or paragraph 3 (the implementation).

Finally one or more tag lines should exist. Each developer responsible
for working on the patch is responsible for adding a Signed-off-by: tag line.

It is not acceptable to have an empty or non-existent header, or just a single
line message. The summary and description is required for all changes.

== Importing from Elsewhere ==

If you are importing work from somewhere else, such as a cherry-pick from
another repository or a code patch pulled from a mailing list, the minimum patch
header or commit message is not enough. It does not clearly establish the
provenance of the code.

The following specifies the additional guidelines required for importing changes
from elsewhere.

By default you should keep the original author's summary and description,
along with any tag lines such as, "Signed-off-by:". If the original change that
was imported does not have a summary and/or commit message from the original
author, it is still your responsibility to add the summary and commit message to
the patch header. Just as if you wrote the code, you should be able to clearly
explain what the change does. It is also necessary to document the original
author of the change. You should indicate the original author by simply stating
"written by" or "posted to the ... mailing list by". You must not add a
Signed-off-by: tag line with their name, only the original author may add their
own Signed-off-by: tag line.

It is also required that the origin of the work be fully documented. The origin
should be specified as part of the commit message in a way that an average
developer would be able to track down the original code. URLs should reference
original authoritative sources and not mirrors.

If changes were required to resolve conflicts, they should be documented as
well. When incorporating a commit or patch from an external source, changes to
the functionality not related to resolving conflicts should be made in a
second commit or patch. This preserves the original external commit, and makes
the modifications clearly visible, and prevents confusion over the author of the
code.

Finally you must add a "Signed-off-by:" tag line to indicate that you worked
with the patch.

== Example: Importing from Elsewhere Unmodified ==

For example, if you were to pull in the patch from the above example unmodified:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The patch was imported from the OpenEmbedded git server
(<nowiki>git://git.openembedded.org/openembedded</nowiki>) as of commit id
b65a0e0c84cf489bfa00d6aa6c48abc5a237100f.

Signed-off-by: Your Name <your.name@openembedded.org>

The above example shows a clear way for a developer to find the original source
of the change. It indicates that the OpenEmbedded developer simply integrated
the change into the system without making any further modifications.

== Example: Importing from Elsewhere Modified ==

When importing a patch, occasionally changes have to be made in order to resolve
conflicts. The following is an example of the commit message when the item needed
to be modified:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The patch was imported from the OpenEmbedded git server
(<nowiki>git://git.openembedded.org/openembedded</nowiki>) as of commit id
b65a0e0c84cf489bfa00d6aa6c48abc5a237100f.

A previous change had modified the memory threshold calculation,
causing a conflict. The conflict was resolved by preserving the
memory threshold calculation from the existing implementation.

Signed-off-by: Your Name <your.name@openembedded.org>

== Patch Header Recommendations ==

In order to keep track of patches and ultimately reduce the number of patches
that are required to be maintained, we need to track the status of the patches
that are created. The following items are specific to patches applied by recipes.

In addition to the items mentioned above, we also want to track if it's
appropriate to get this particular patch into the upstream project. Since we
want to track this information, we need a consistent tag and status indicator
that can be searched.

While adding this tracking information to the patch headers is currently
optional, it is highly recommended and some maintainers may require it. It is
optional at this time so that it can be evaluated as to its usefulness over
time. Existing patches will be updated with the tag as they are modified.

As mentioned in the above, be sure to include any URL to bug tracking, mailing
lists or other reference material pertaining to the patch. Then add a new tag
"Upstream-Status:" which contains one of the following items:

'''Pending'''
- No determination has been made yet or not yet submitted to upstream

'''Submitted''' [where]
- Submitted to upstream, waiting approval
- Optionally include where it was submitted, such as the author, mailing
list, etc.

'''Accepted'''
- Accepted in upstream, expect it to be removed at next update, include
expected version info

'''Backport'''
- Backported from new upstream version, because we are at a fixed version,
include upstream version info

'''Denied'''
- Not accepted by upstream, include reason in patch

'''Inappropriate [reason]'''
- The patch is not appropriate for upstream, include a brief reason on the
same line enclosed with []
reason can be:
not author (You are not the author and do not intend to upstream this,
source must be listed in the comments)
native
licensing
configuration
enable feature
disable feature
bugfix (add bug URL here)
embedded specific
no upstream (the upstream is no longer available -- dead project)
other (give details in comments)

The various "Inappropriate [reason]" status items are meant to indicate that the
person responsible for adding this patch to the system does not intend to
upstream the patch for a specific reason. Unless otherwise noted, another
person could submit this patch to the upstream, if they do the status should be
changed to "Submitted [where]", and an additional Signed-off-by: line added to
the patch by the person claiming responsibility for upstreaming.

For example, if the patch has been submitted upstream:

rpm: Adjusted the foo setting in bar

<nowiki>[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5</nowiki>

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Upstream-Status: Submitted [rpm5-devel@rpm5.org]

Signed-off-by: Joe Developer <joe.developer@example.com>

A future commit can change the value to "Accepted" or "Denied" as appropriate.

== Common Errors in Patch and Commit Messages ==

- Short log does not start with the file or component being modified. Such as
"foo: Update to new upstream version 5.8.9"

- Avoid starting the summary line with a capital letter, unless the component
being referred to also begins with a capital letter.

- Don't have one huge patch, split your change into logical subparts. It's
easier to track down problems afterward using tools such as git bisect. It also
makes it easy for people to cherry-pick changes into things like stable branches.

- Don't simply translate your change into English for a commit log. The log
"Change compare from zero to one" is bad because it describes only the code
change in the patch; we can see that from reading the patch itself. Let the code
tell the story of the mechanics of the change (as much as possible), and let
your comment tell the other details -- i.e. what the problem was, how it
manifested itself (symptoms), and if necessary, the justification for why the
fix was done in manner that it was. In other words, the long commit message
must describe *why* the change was needed (instead of what has changed).

- Don't start your commit log with "This patch..." -- we already know it is a patch.

- Don't repeat your short log in the long log. If you really really don't have
anything new and informational to add in as a long log, then don't put a long
log at all. This should be uncommon -- i.e. the only acceptable cases for no
long log would be something like "Documentation/README: Fix spelling mistakes".

- Don't put absolute paths to source files that are specific to your site, i.e.
if you get a compile error on "fs/exec.c" then tidy up the parts of the compile
output to just show that portion. We don't need to see that it was
/home/wally/src/git/oe-core/meta/classes/insane.bbclass or similar - that full
path has no value to others. Always reduce the path to something more
meaningful, such as oe-core/meta/classes/insane.bbclass.

- Always use the most significant ramification of the change in the words of
your subject/shortlog. For example, don't say "fix compile warning in foo" when
the compiler warning was really telling us that we were dereferencing complete
garbage off in the weeds that could in theory cause an OOPS under some
circumstances. When people are choosing commits for backports to stable or
distro kernels, the shortlog will be what they use for an initial sorting
selection. If they see "Fix possible OOPS in...." then these people will look
closer, whereas they most likely will skip over simple warning fixes.

- Don't put in the full 20 or more lines of a backtrace when really it is just
the last 5 or so function calls that are relevant to the crash/fix. If the entry
point, or start of the trace is relevant (i.e. a syscall or similar), you can
leave that, and then replace a chunk of the intermediate calls in the middle
with a single [...]

- Don't include timestamps or other unnecessary information, unless they are
relevant to the situation (i.e. indicating an unacceptable delay in a driver
initialization etc.)

- Don't use links to temporary resources like pastebin and friends. The commit
message may be read long after this resource timed out.

- Don't reference mirrors, but instead always reference original authoritative
sources.

- Avoid punctuation in the short log.

Commit Patch Message

2011-06-14T16:17:46Z

Mhatle: moved Commit Patch Message to Commit Patch Message Guidelines: Clear Up Confusion in the name

#REDIRECT [[Commit Patch Message Guidelines]]

Commit Patch Message Guidelines

2011-06-14T16:17:46Z

Mhatle: moved Commit Patch Message to Commit Patch Message Guidelines: Clear Up Confusion in the name

This set of guidelines is intended to help both the developer and reviewers of
changes determine reasonable patch headers and change commit messages. Often
when working with the code, you forget that not everyone is as familiar with the
problem and/or fix as you are. Often the next person in the code doesn't
understand what or why something is done so they quickly look at patch header
and commit messages. Unless these messages are clear it will be difficult to
understand the relevance of a given change and how future changes may impact
previous decisions.

This policy refers both to patches that are being applied by recipes as well as
commit messages that are part of the source control system, usually git. A patch
file needs a header in order to describe the specific changes that are made
within that patch, while a commit message describes one or more changes to the
overall project or system. Both the patch headers and commit messages require
the same attention and basic details, however the purposes of the messages are
slightly different. A commit message documents all of the changes made as part
of a commit, while a patch header documents items specific to a single patch. In
many cases the patch header can also be used as the commit message.

This policy does not cover the testing of the changes, or the technical criteria
for accepting a patch.

By following these guidelines we will have a better record of the problems and
solutions made over the course of development. It will also help establish a
clear provenance of all of the code and changes.

''Approved by the Technical Steering Committee 2011/06/09.''

== General Information ==

While specific to the Linux kernel development, the following could also be
considered a general guide for any Open Source development:

http://ldn.linuxfoundation.org/book/how-participate-linux-community

Many of the guidelines in this document are related to the items in that
information.

Pay particular attention to section 5.3 that talks about patch preparation.
The key thing to remember is to break up your changes into logical sections.
Otherwise you run the risk of not being able to even explain the purpose of a
change in the patch headers!

In addition section 5.4 explains the purpose of the Signed-off-by: tag line as
discussed in later parts of this document. Due to the importance of the
Signed-off-by: tag line a copy of the description follows:

Signed-off-by: this is a developer's certification that he or she has
the right to submit the patch for inclusion into the [project]. It is
an agreement to the Developer's Certificate of Origin, the full text of
which can be found in [Linux Kernel] Documentation/SubmittingPatches.
Code without a proper signoff cannot be merged into the mainline.

For more information on the Developer's Certificate of Origin and the Linux
Kernel Documentation/SubmittingPatches, see:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/SubmittingPatches

== Patch Headers and Commit Messages ==

There seems to always be a question or two surrounding what a person
should put in a patch header, or commit message.

The general rules always apply, those being that there is a single line
short log or summary of the change (think of this as the subject when the patch
is e-mailed), and then the more detailed long log, and closure with tag lines
such as "Signed-off-by:".

== New development ==

A minimal patch or commit message would be of the format:

Short log / Statement of what needed to be changed.

(Optional pointers to external resources, such as defect tracking)

The intent of your change.

(Optional, if it's not clear from above) how your change resolves the
issues in the first part.

Tag line(s) at the end.

For example:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The minimal commit message is good for new code development and simple changes.

The single short log message indicates what needed to be changed. It should
begin with an indicator as to the primary item changed by this work,
followed by a short summary of the change. In the above case we're indicating
that we've changed the "foobar" item, by "adjusting the foo setting in bar".

The single short log message is analogous to the git "commit summary". While no
maximum line length is specified by this policy, it is suggested that it remains
under 78 characters wherever possible.

Optionally, you may include pointers to defects this change corrects. Unless
the defect format is specified by the component you are modifying, it is
suggested that you use a full URL to specify the reference to the defect
information. Generally these pointers will precede any long description, but as
an optional item it may be after the long description.

For example, you might refer to an open source defect in the RPM project:

rpm: Adjusted the foo setting in bar

<nowiki>[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5</nowiki>

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

You must then have a full description of the change. Specifying the intent of
your change and if necessary how the change resolves the issue.

As mentioned above this is intended to answer the "what were you thinking"
questions down the road and to know what other impacts are likely to
result of the change needs to be reverted. It also allows for better
solutions if someone comes along later and agrees with paragraph 1
(the problem statement) and either disagrees with paragraph 2
(the design) or paragraph 3 (the implementation).

Finally one or more tag lines should exist. Each developer responsible
for working on the patch is responsible for adding a Signed-off-by: tag line.

It is not acceptable to have an empty or non-existent header, or just a single
line message. The summary and description is required for all changes.

== Importing from elsewhere ==

If you are importing work from somewhere else, such as a cherry-pick from
another repository or a code patch pulled from a mailing list, the minimum patch
header or commit message is not enough. It does not clearly establish the
provenance of the code.

The following specifies the additional guidelines required for importing changes
from elsewhere.

By default you should keep the original author's summary and description,
along with any tag lines such as, "Signed-off-by:". If the original change that
was imported does not have a summary and/or commit message from the original
author, it is still your responsibility to add the summary and commit message to
the patch header. Just as if you wrote the code, you should be able to clearly
explain what the change does. It is also necessary to document the original
author of the change. You should indicate the original author by simply stating
"written by" or "posted to the ... mailing list by". You must not add a
Signed-off-by: tag line with their name, only the original author may add their
own Signed-off-by: tag line.

It is also required that the origin of the work be fully documented. The origin
should be specified as part of the commit message in a way that an average
developer would be able to track down the original code. URLs should reference
original authoritative sources and not mirrors.

If changes were required to resolve conflicts, they should be documented as
well. When incorporating a commit or patch from an external source, changes to
the functionality not related to resolving conflicts should be made in a
second commit or patch. This preserves the original external commit, and makes
the modifications clearly visible, and prevents confusion over the author of the
code.

Finally you must add a "Signed-off-by:" tag line to indicate that you worked
with the patch.

== Example: Importing form elsewhere unmodified ==

For example, if you were to pull in the patch from the above example unmodified:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The patch was imported from the OpenEmbedded git server
(<nowiki>git://git.openembedded.org/openembedded</nowiki>) as of commit id
b65a0e0c84cf489bfa00d6aa6c48abc5a237100f.

Signed-off-by: Your Name <your.name@openembedded.org>

The above example shows a clear way for a developer to find the original source
of the change. It indicates that the OpenEmbedded developer simply integrated
the change into the system without making any further modifications.

== Example: Importing from Elsewhere modified ==

When importing a patch, occasionally changes have to be made in order to resolve
conflicts. The following is an example of the commit message when the item needed
to be modified:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The patch was imported from the OpenEmbedded git server
(<nowiki>git://git.openembedded.org/openembedded</nowiki>) as of commit id
b65a0e0c84cf489bfa00d6aa6c48abc5a237100f.

A previous change had modified the memory threshold calculation,
causing a conflict. The conflict was resolved by preserving the
memory threshold calculation from the existing implementation.

Signed-off-by: Your Name <your.name@openembedded.org>

== Patch Header Recommendations ==

In order to keep track of patches and ultimately reduce the number of patches
that are required to be maintained, we need to track the status of the patches
that are created. The following items are specific to patches applied by recipes.

In addition to the items mentioned above, we also want to track if it's
appropriate to get this particular patch into the upstream project. Since we
want to track this information, we need a consistent tag and status indicator
that can be searched.

While adding this tracking information to the patch headers is currently
optional, it is highly recommended and some maintainers may require it. It is
optional at this time so that it can be evaluated as to its usefulness over
time. Existing patches will be modified with the tag as they are modified.

As mentioned in the above, be sure to include any URL to bug tracking, mailing
lists or other reference material pertaining to the patch. Then add a new tag
"Upstream-Status:" which contains one of the following items:

'''Pending'''
- No determination has been made yet or not yet submitted to upstream

'''Submitted''' [where]
- Submitted to upstream, waiting approval
- Optionally include where it was submitted, such as the author, mailing
list, etc.

'''Accepted'''
- Accepted in upstream, expect it to be removed at next update, include
expected version info

'''Backport'''
- Backported from new upstream version, because we are at a fixed version,
include upstream version info

'''Denied'''
- Not accepted by upstream, include reason in patch

'''Inappropriate [reason]'''
- The patch is not appropriate for upstream, include a brief reason on the
same line enclosed with []
reason can be:
not author (You are not the author and do not intend to upstream this,
source must be listed in the comments)
native
licensing
configuration
enable feature
disable feature
bugfix (add bug URL here)
embedded specific
no upstream (the upstream is no longer available -- dead project)
other (give details in comments)

The various "Inappropriate [reason]" status items are meant to indicate that the
person responsible for adding this patch to the system does not intend to
upstream the patch for a specific reason. Unless otherwise noted, another
person could submit this patch to the upstream, if they do the status should be
changed to "Submitted [where]", and an additional Signed-off-by: line added to
the patch by the person claiming responsibility for upstreaming.

For example, if the patch has been submitted upstream:

rpm: Adjusted the foo setting in bar

<nowiki>[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5</nowiki>

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Upstream-Status: Submitted [rpm5-devel@rpm5.org]

Signed-off-by: Joe Developer <joe.developer@example.com>

A future commit can change the value to "Accepted" or "Denied" as appropriate.

== Common Errors in Patch and Commit Messages ==

- Short log does not start with the file or component being modified. Such as
"foo: Update to new upstream version 5.8.9"

- Avoid starting the summary line with a capital letter, unless the component
being referred to also begins with a capital letter.

- Don't have one huge patch, split your change into logical subparts. It's
easier to track down problems afterward using tools such as git bisect. It also
makes it easy for people to cherry-pick changes into things like stable branches.

- Don't simply translate your change into English for a commit log. The log
"Change compare from zero to one" is bad because it describes only the code
change in the patch; we can see that from reading the patch itself. Let the code
tell the story of the mechanics of the change (as much as possible), and let
your comment tell the other details -- i.e. what the problem was, how it
manifested itself (symptoms), and if necessary, the justification for why the
fix was done in manner that it was. In other words, the long commit message
must describe *why* the change was needed (instead of what has changed).

- Don't start your commit log with "This patch..." -- we already know it is a patch.

- Don't repeat your short log in the long log. If you really really don't have
anything new and informational to add in as a long log, then don't put a long
log at all. This should be uncommon -- i.e. the only acceptable cases for no
long log would be something like "Documentation/README: Fix spelling mistakes".

- Don't put absolute paths to source files that are specific to your site, i.e.
if you get a compile error on "fs/exec.c" then tidy up the parts of the compile
output to just show that portion. We don't need to see that it was
/home/wally/src/git/oe-core/meta/classes/insane.bbclass or similar - that full
path has no value to others. Always reduce the path to something more
meaningful, such as oe-core/meta/classes/insane.bbclass.

- Always use the most significant ramification of the change in the words of
your subject/shortlog. For example, don't say "fix compile warning in foo" when
the compiler warning was really telling us that we were dereferencing complete
garbage off in the weeds that could in theory cause an OOPS under some
circumstances. When people are choosing commits for backports to stable or
distro kernels, the shortlog will be what they use for an initial sorting
selection. If they see "Fix possible OOPS in...." then these people will look
closer, whereas they most likely will skip over simple warning fixes.

- Don't put in the full 20 or more lines of a backtrace when really it is just
the last 5 or so function calls that are relevant to the crash/fix. If the entry
point, or start of the trace is relevant (i.e. a syscall or similar), you can
leave that, and then replace a chunk of the intermediate calls in the middle
with a single [...]

- Don't include timestamps or other unnecessary information, unless they are
relevant to the situation (i.e. indicating an unacceptable delay in a driver
initialization etc.)

- Don't use links to temporary resources like pastebin and friends. The commit
message may be read long after this resource timed out.

- Don't reference mirrors, but instead always reference original authoritative
sources.

- Avoid punctuation in the short log.

Commit Patch Message Guidelines

2011-06-14T15:54:07Z

Mhatle: Created page with "This set of guidelines is intended to help both the developer and reviewers of changes determine reasonable patch headers and change commit messages. Often when working with the ..."

This set of guidelines is intended to help both the developer and reviewers of
changes determine reasonable patch headers and change commit messages. Often
when working with the code, you forget that not everyone is as familiar with the
problem and/or fix as you are. Often the next person in the code doesn't
understand what or why something is done so they quickly look at patch header
and commit messages. Unless these messages are clear it will be difficult to
understand the relevance of a given change and how future changes may impact
previous decisions.

This policy refers both to patches that are being applied by recipes as well as
commit messages that are part of the source control system, usually git. A patch
file needs a header in order to describe the specific changes that are made
within that patch, while a commit message describes one or more changes to the
overall project or system. Both the patch headers and commit messages require
the same attention and basic details, however the purposes of the messages are
slightly different. A commit message documents all of the changes made as part
of a commit, while a patch header documents items specific to a single patch. In
many cases the patch header can also be used as the commit message.

This policy does not cover the testing of the changes, or the technical criteria
for accepting a patch.

By following these guidelines we will have a better record of the problems and
solutions made over the course of development. It will also help establish a
clear provenance of all of the code and changes.

''Approved by the Technical Steering Committee 2011/06/09.''

== General Information ==

While specific to the Linux kernel development, the following could also be
considered a general guide for any Open Source development:

http://ldn.linuxfoundation.org/book/how-participate-linux-community

Many of the guidelines in this document are related to the items in that
information.

Pay particular attention to section 5.3 that talks about patch preparation.
The key thing to remember is to break up your changes into logical sections.
Otherwise you run the risk of not being able to even explain the purpose of a
change in the patch headers!

In addition section 5.4 explains the purpose of the Signed-off-by: tag line as
discussed in later parts of this document. Due to the importance of the
Signed-off-by: tag line a copy of the description follows:

Signed-off-by: this is a developer's certification that he or she has
the right to submit the patch for inclusion into the [project]. It is
an agreement to the Developer's Certificate of Origin, the full text of
which can be found in [Linux Kernel] Documentation/SubmittingPatches.
Code without a proper signoff cannot be merged into the mainline.

For more information on the Developer's Certificate of Origin and the Linux
Kernel Documentation/SubmittingPatches, see:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/SubmittingPatches

== Patch Headers and Commit Messages ==

There seems to always be a question or two surrounding what a person
should put in a patch header, or commit message.

The general rules always apply, those being that there is a single line
short log or summary of the change (think of this as the subject when the patch
is e-mailed), and then the more detailed long log, and closure with tag lines
such as "Signed-off-by:".

== New development ==

A minimal patch or commit message would be of the format:

Short log / Statement of what needed to be changed.

(Optional pointers to external resources, such as defect tracking)

The intent of your change.

(Optional, if it's not clear from above) how your change resolves the
issues in the first part.

Tag line(s) at the end.

For example:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The minimal commit message is good for new code development and simple changes.

The single short log message indicates what needed to be changed. It should
begin with an indicator as to the primary item changed by this work,
followed by a short summary of the change. In the above case we're indicating
that we've changed the "foobar" item, by "adjusting the foo setting in bar".

The single short log message is analogous to the git "commit summary". While no
maximum line length is specified by this policy, it is suggested that it remains
under 78 characters wherever possible.

Optionally, you may include pointers to defects this change corrects. Unless
the defect format is specified by the component you are modifying, it is
suggested that you use a full URL to specify the reference to the defect
information. Generally these pointers will precede any long description, but as
an optional item it may be after the long description.

For example, you might refer to an open source defect in the RPM project:

rpm: Adjusted the foo setting in bar

<nowiki>[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5</nowiki>

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

You must then have a full description of the change. Specifying the intent of
your change and if necessary how the change resolves the issue.

As mentioned above this is intended to answer the "what were you thinking"
questions down the road and to know what other impacts are likely to
result of the change needs to be reverted. It also allows for better
solutions if someone comes along later and agrees with paragraph 1
(the problem statement) and either disagrees with paragraph 2
(the design) or paragraph 3 (the implementation).

Finally one or more tag lines should exist. Each developer responsible
for working on the patch is responsible for adding a Signed-off-by: tag line.

It is not acceptable to have an empty or non-existent header, or just a single
line message. The summary and description is required for all changes.

== Importing from elsewhere ==

If you are importing work from somewhere else, such as a cherry-pick from
another repository or a code patch pulled from a mailing list, the minimum patch
header or commit message is not enough. It does not clearly establish the
provenance of the code.

The following specifies the additional guidelines required for importing changes
from elsewhere.

By default you should keep the original author's summary and description,
along with any tag lines such as, "Signed-off-by:". If the original change that
was imported does not have a summary and/or commit message from the original
author, it is still your responsibility to add the summary and commit message to
the patch header. Just as if you wrote the code, you should be able to clearly
explain what the change does. It is also necessary to document the original
author of the change. You should indicate the original author by simply stating
"written by" or "posted to the ... mailing list by". You must not add a
Signed-off-by: tag line with their name, only the original author may add their
own Signed-off-by: tag line.

It is also required that the origin of the work be fully documented. The origin
should be specified as part of the commit message in a way that an average
developer would be able to track down the original code. URLs should reference
original authoritative sources and not mirrors.

If changes were required to resolve conflicts, they should be documented as
well. When incorporating a commit or patch from an external source, changes to
the functionality not related to resolving conflicts should be made in a
second commit or patch. This preserves the original external commit, and makes
the modifications clearly visible, and prevents confusion over the author of the
code.

Finally you must add a "Signed-off-by:" tag line to indicate that you worked
with the patch.

== Example: Importing form elsewhere unmodified ==

For example, if you were to pull in the patch from the above example unmodified:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The patch was imported from the OpenEmbedded git server
(<nowiki>git://git.openembedded.org/openembedded</nowiki>) as of commit id
b65a0e0c84cf489bfa00d6aa6c48abc5a237100f.

Signed-off-by: Your Name <your.name@openembedded.org>

The above example shows a clear way for a developer to find the original source
of the change. It indicates that the OpenEmbedded developer simply integrated
the change into the system without making any further modifications.

== Example: Importing from Elsewhere modified ==

When importing a patch, occasionally changes have to be made in order to resolve
conflicts. The following is an example of the commit message when the item needed
to be modified:

foobar: Adjusted the foo setting in bar

When using foobar on systems with less than a gigabyte of RAM common
usage patterns often result in an Out-of-memory condition causing
slowdowns and unexpected application termination.

Low-memory systems should continue to function without running into
memory-starvation conditions with minimal cost to systems with more
available memory. High-memory systems will be less able to use the
full extent of the system, a dynamically tunable option may be best,
long-term.

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Signed-off-by: Joe Developer <joe.developer@example.com>

The patch was imported from the OpenEmbedded git server
(<nowiki>git://git.openembedded.org/openembedded</nowiki>) as of commit id
b65a0e0c84cf489bfa00d6aa6c48abc5a237100f.

A previous change had modified the memory threshold calculation,
causing a conflict. The conflict was resolved by preserving the
memory threshold calculation from the existing implementation.

Signed-off-by: Your Name <your.name@openembedded.org>

== Patch Header Recommendations ==

In order to keep track of patches and ultimately reduce the number of patches
that are required to be maintained, we need to track the status of the patches
that are created. The following items are specific to patches applied by recipes.

In addition to the items mentioned above, we also want to track if it's
appropriate to get this particular patch into the upstream project. Since we
want to track this information, we need a consistent tag and status indicator
that can be searched.

While adding this tracking information to the patch headers is currently
optional, it is highly recommended and some maintainers may require it. It is
optional at this time so that it can be evaluated as to its usefulness over
time. Existing patches will be modified with the tag as they are modified.

As mentioned in the above, be sure to include any URL to bug tracking, mailing
lists or other reference material pertaining to the patch. Then add a new tag
"Upstream-Status:" which contains one of the following items:

'''Pending'''
- No determination has been made yet or not yet submitted to upstream

'''Submitted''' [where]
- Submitted to upstream, waiting approval
- Optionally include where it was submitted, such as the author, mailing
list, etc.

'''Accepted'''
- Accepted in upstream, expect it to be removed at next update, include
expected version info

'''Backport'''
- Backported from new upstream version, because we are at a fixed version,
include upstream version info

'''Denied'''
- Not accepted by upstream, include reason in patch

'''Inappropriate [reason]'''
- The patch is not appropriate for upstream, include a brief reason on the
same line enclosed with []
reason can be:
not author (You are not the author and do not intend to upstream this,
source must be listed in the comments)
native
licensing
configuration
enable feature
disable feature
bugfix (add bug URL here)
embedded specific
no upstream (the upstream is no longer available -- dead project)
other (give details in comments)

The various "Inappropriate [reason]" status items are meant to indicate that the
person responsible for adding this patch to the system does not intend to
upstream the patch for a specific reason. Unless otherwise noted, another
person could submit this patch to the upstream, if they do the status should be
changed to "Submitted [where]", and an additional Signed-off-by: line added to
the patch by the person claiming responsibility for upstreaming.

For example, if the patch has been submitted upstream:

rpm: Adjusted the foo setting in bar

<nowiki>[RPM Ticket #65] -- http://rpm5.org/cvs/tktview?tn=65,5</nowiki>

The foo setting in bar was decreased from X to X-50% in order to
ensure we don't exhaust all system memory with foobar threads.

Upstream-Status: Submitted [rpm5-devel@rpm5.org]

Signed-off-by: Joe Developer <joe.developer@example.com>

A future commit can change the value to "Accepted" or "Denied" as appropriate.

== Common Errors in Patch and Commit Messages ==

- Short log does not start with the file or component being modified. Such as
"foo: Update to new upstream version 5.8.9"

- Avoid starting the summary line with a capital letter, unless the component
being referred to also begins with a capital letter.

- Don't have one huge patch, split your change into logical subparts. It's
easier to track down problems afterward using tools such as git bisect. It also
makes it easy for people to cherry-pick changes into things like stable branches.

- Don't simply translate your change into English for a commit log. The log
"Change compare from zero to one" is bad because it describes only the code
change in the patch; we can see that from reading the patch itself. Let the code
tell the story of the mechanics of the change (as much as possible), and let
your comment tell the other details -- i.e. what the problem was, how it
manifested itself (symptoms), and if necessary, the justification for why the
fix was done in manner that it was. In other words, the long commit message
must describe *why* the change was needed (instead of what has changed).

- Don't start your commit log with "This patch..." -- we already know it is a patch.

- Don't repeat your short log in the long log. If you really really don't have
anything new and informational to add in as a long log, then don't put a long
log at all. This should be uncommon -- i.e. the only acceptable cases for no
long log would be something like "Documentation/README: Fix spelling mistakes".

- Don't put absolute paths to source files that are specific to your site, i.e.
if you get a compile error on "fs/exec.c" then tidy up the parts of the compile
output to just show that portion. We don't need to see that it was
/home/wally/src/git/oe-core/meta/classes/insane.bbclass or similar - that full
path has no value to others. Always reduce the path to something more
meaningful, such as oe-core/meta/classes/insane.bbclass.

- Always use the most significant ramification of the change in the words of
your subject/shortlog. For example, don't say "fix compile warning in foo" when
the compiler warning was really telling us that we were dereferencing complete
garbage off in the weeds that could in theory cause an OOPS under some
circumstances. When people are choosing commits for backports to stable or
distro kernels, the shortlog will be what they use for an initial sorting
selection. If they see "Fix possible OOPS in...." then these people will look
closer, whereas they most likely will skip over simple warning fixes.

- Don't put in the full 20 or more lines of a backtrace when really it is just
the last 5 or so function calls that are relevant to the crash/fix. If the entry
point, or start of the trace is relevant (i.e. a syscall or similar), you can
leave that, and then replace a chunk of the intermediate calls in the middle
with a single [...]

- Don't include timestamps or other unnecessary information, unless they are
relevant to the situation (i.e. indicating an unacceptable delay in a driver
initialization etc.)

- Don't use links to temporary resources like pastebin and friends. The commit
message may be read long after this resource timed out.

- Don't reference mirrors, but instead always reference original authoritative
sources.

- Avoid punctuation in the short log.