[OE-core] [pseudo][PATCH] Fix to fcntl guts to ignore flags that can be ORed into cmd

[Burton, Ross]

Hi Will,

This should go to the yocto at yoctoproject.org list really.  I've CC'd the
maintainer to be sure he spots it.

Ross

On 15 September 2017 at 23:27, Will Page <Will.Page at ni.com> wrote:

> The fcntl guts switch on "cmd" parameter to identify the fcntl
> command being issued, but isn't aware of the file creation flags that
> can be ORed in.
>
> This change masks out the flags from the command only for the switch
> statement so that it can correctly determine whether it needs to pass
> "lock" or "arg".  When real_fcntl() is called, the original value is
> still passed on. This corrects an issue observed using pseudo on modern
> linux desktops resulting in "pseudo: unknown fcntl argument 1030,
> assuming long argument." diagnostics in the output.  Decimal 1030 is
> 0x0406, which translates to O_NOCTTY | F_SETLK, and should call
> "rc = real_fcntl(fd, cmd, lock);", but instead falls through to the
> default case instead calling "rc = real_fcntl(fd, cmd, arg);".
>
> Tested the change using perftest - without this patch, the issue is
> trivially reproducible on my Ubuntu Xenial system, and after patching it
> emitted no "unknown fcntl argument" diagnostics.
>
> Signed-off-by: Will Page <Will.Page at ni.com>
> ---
>  ports/linux/guts/fcntl.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/ports/linux/guts/fcntl.c b/ports/linux/guts/fcntl.c
> index 639fd24..d278a8c 100644
> --- a/ports/linux/guts/fcntl.c
> +++ b/ports/linux/guts/fcntl.c
> @@ -8,6 +8,10 @@
>   */
>         long arg;
>         int save_errno;
> +       /* some bits can be ORed into the cmd should be explicitly ignored
> +        * see fcntl documentation on F_SETFL */
> +       int o_mode_mask = (O_RDONLY | O_WRONLY | O_RDWR) |
> +               (O_CREAT | O_EXCL | O_NOCTTY | O_TRUNC);
>
>         /* we don't know whether we need lock or arg; grab both, which
>          * should be safe enough on Linuxy systems. */
> @@ -15,7 +19,7 @@
>         arg = va_arg(ap, long);
>         va_end(ap);
>
> -       switch (cmd) {
> +       switch (cmd & ~(o_mode_mask)) {
>         case F_DUPFD:
>  #ifdef F_DUPFD_CLOEXEC
>         case F_DUPFD_CLOEXEC:
> --
> 2.7.4
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20170915/a8755686/attachment-0002.html>