[OE-core] shell script guidelines in oe-core? (was Re: [PATCH v4] create-pull-request: cleanup bashisms)

[Christopher Larson]

On Wed, Aug 12, 2015 at 10:46 AM, Christopher Larson <clarson at kergoth.com>
wrote:

> On Wed, Aug 12, 2015 at 9:56 AM, Khem Raj <raj.khem at gmail.com> wrote:
>
>> On Wed, Aug 12, 2015 at 2:49 AM,  <Mikko.Rapeli at bmw.de> wrote:
>> > This reminded me of the problems I've seen with various oe-core shell
>> scripts:
>> > they are missing systematic error handling.
>> >
>> > IMO using bash and 'set -euxo pipefail' are a good approach to catch
>> errors
>> > early in shell scripts. Manually checking for $? is error prone and
>> things
>> > like pipes can hide them. Using undefined variables is another thing
>> > which should fail early before things go horribly wrong. And in build
>> automation
>> > I'd rather see output of -x since things can fail in mysterious ways.
>> > And then there's quoting....
>> >
>> > One useful guide is to use shellcheck as static analysis tool for shell
>> scripts.
>> >
>> > I guess internally bitbake uses at least set -e.
>> >
>> > Comments?
>>
>> all these are good suggestions, I would recommend to open bugs for all
>> of these some of them might need
>> to cite the scripts which need fixing.
>
>
> Agreed, it's important to check the behavior of shell scripts is kosher
> and standards-compliant. shellcheck is extremely useful, I recommend it as
> well, but combined with the vim syntastic plugin, so every time you save
> the file all the errors and warnings are identified :) Makes it harder to
> ignore or forget about.


That reminds me, there's a shell portability issue / standards-complaince
issue that's not identified by shellcheck. Typically, to negate a  bracket
expression in a regular expression, one uses ^, e.g. [^a-z] is everything
that isn't in the range a to z, but that's not the case in shell, e.g. at a
prompt or in a ${foo#<pattern>}:

    "[...]a bracket expression as in XBD *RE Bracket Expression* , except
that the <exclamation-mark> character ( '!' ) shall replace the
<circumflex> character ( '^' ) in its role in a non-matching list in the
regular expression notation"

So in shell, you'd want [!a-z] rather than [^a-z]. Nearly all shells handle
both, but the behavior of the latter is actually unspecified according to
the standard:

    "A bracket expression starting with an unquoted <circumflex> character
produces unspecified results."

I recently got bitten by this with one of my shell scripts on a system
running dash.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20150812/bb9ee632/attachment-0002.html>