[OE-core] [rocko][PATCH] weston: add patch to set pitch correctly for subsampled textures

[Andrey Konovalov]

This exact patch also applies to pyro.
Should I send the separate "[OE-core][pyro][PATCH] weston: add patch to set pitch correctly for subsampled textures" email,
or just asking to merge it into pyro as well is enough?

Thanks,
Andrey

On 24.11.2017 22:47, Andrey Konovalov wrote:
> This fixes display issue with YUV420/I420 and NV12 formats, that
> can result in crash of weston.
> 
> The master branch has this fix as part of commit 148920f3971d "weston:
> Bump version to 3.0.0". The patch has been rebased to apply cleanly
> to weston 2.0.0.
> 
> Signed-off-by: Andrey Konovalov <andrey.konovalov at linaro.org>
> ---
>   ...t-pitch-correctly-for-subsampled-textures.patch | 55 ++++++++++++++++++++++
>   meta/recipes-graphics/wayland/weston_2.0.0.bb      |  1 +
>   2 files changed, 56 insertions(+)
>   create mode 100644 meta/recipes-graphics/wayland/weston/weston-gl-renderer-Set-pitch-correctly-for-subsampled-textures.patch
> 
> diff --git a/meta/recipes-graphics/wayland/weston/weston-gl-renderer-Set-pitch-correctly-for-subsampled-textures.patch b/meta/recipes-graphics/wayland/weston/weston-gl-renderer-Set-pitch-correctly-for-subsampled-textures.patch
> new file mode 100644
> index 0000000..c188018
> --- /dev/null
> +++ b/meta/recipes-graphics/wayland/weston/weston-gl-renderer-Set-pitch-correctly-for-subsampled-textures.patch
> @@ -0,0 +1,55 @@
> +Multi-plane sub-sampled textures have partial width/height, e.g.
> +YUV420/I420 has a full-size Y plane, followed by a half-width/height U
> +plane, and a half-width/height V plane.
> +
> +zwp_linux_dmabuf_v1 allows clients to pass an explicit pitch for each
> +plane, but for wl_shm this must be inferred. gl-renderer was correctly
> +accounting for the width and height when subsampling, but the pitch was
> +being taken as the pitch for the first plane.
> +
> +This does not match the requirements for GStreamer's waylandsink, in
> +particular, as well as other clients. Fix the SHM upload path to
> +correctly set the pitch for each plane, according to subsampling.
> +
> +Tested with:
> +  $ gst-launch-1.0 videotestsrc ! waylandsink
> +
> +Upstream-Status: Backport [https://patchwork.freedesktop.org/patch/180767/]
> +
> +Signed-off-by: Daniel Stone <daniels at collabora.com>
> +Fixes: fdeefe42418 ("gl-renderer: add support of WL_SHM_FORMAT_YUV420")
> +Reported-by: Fabien Lahoudere <fabien.lahoudere at collabora.co.uk>
> +Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=103063
> +
> +---
> + libweston/gl-renderer.c | 4 ++--
> + 1 file changed, 2 insertions(+), 2 deletions(-)
> +
> +diff --git a/libweston/gl-renderer.c b/libweston/gl-renderer.c
> +index 244ce309..40bf0bb6 100644
> +--- a/libweston/gl-renderer.c
> ++++ b/libweston/gl-renderer.c
> +@@ -1285,14 +1285,13 @@ gl_renderer_flush_damage(struct weston_surface *surface)
> + 		goto done;
> + 	}
> +
> +-	glPixelStorei(GL_UNPACK_ROW_LENGTH_EXT, gs->pitch);
> +-
> + 	if (gs->needs_full_upload) {
> + 		glPixelStorei(GL_UNPACK_SKIP_PIXELS_EXT, 0);
> + 		glPixelStorei(GL_UNPACK_SKIP_ROWS_EXT, 0);
> + 		wl_shm_buffer_begin_access(buffer->shm_buffer);
> + 		for (j = 0; j < gs->num_textures; j++) {
> + 			glBindTexture(GL_TEXTURE_2D, gs->textures[j]);
> ++			glPixelStorei(GL_UNPACK_ROW_LENGTH_EXT, gs->pitch / gs->hsub[j]);
> + 			glTexImage2D(GL_TEXTURE_2D, 0,
> + 				     gs->gl_format[j],
> + 				     gs->pitch / gs->hsub[j],
> +@@ -1317,6 +1316,7 @@ gl_renderer_flush_damage(struct weston_surface *surface)
> + 		glPixelStorei(GL_UNPACK_SKIP_ROWS_EXT, r.y1);
> + 		for (j = 0; j < gs->num_textures; j++) {
> + 			glBindTexture(GL_TEXTURE_2D, gs->textures[j]);
> ++			glPixelStorei(GL_UNPACK_ROW_LENGTH_EXT, gs->pitch / gs->hsub[j]);
> + 			glTexSubImage2D(GL_TEXTURE_2D, 0,
> + 					r.x1 / gs->hsub[j],
> + 					r.y1 / gs->vsub[j],
> diff --git a/meta/recipes-graphics/wayland/weston_2.0.0.bb b/meta/recipes-graphics/wayland/weston_2.0.0.bb
> index 54b07bd..063494c 100644
> --- a/meta/recipes-graphics/wayland/weston_2.0.0.bb
> +++ b/meta/recipes-graphics/wayland/weston_2.0.0.bb
> @@ -12,6 +12,7 @@ SRC_URI = "https://wayland.freedesktop.org/releases/${BPN}-${PV}.tar.xz \
>              file://0001-configure.ac-Fix-wayland-protocols-path.patch \
>              file://xwayland.weston-start \
>              file://0001-weston-launch-Provide-a-default-version-that-doesn-t.patch \
> +           file://weston-gl-renderer-Set-pitch-correctly-for-subsampled-textures.patch \
>   "
>   SRC_URI[md5sum] = "15f38945942bf2a91fe2687145fb4c7d"
>   SRC_URI[sha256sum] = "b4e446ac27f118196f1609dab89bb3cb3e81652d981414ad860e733b355365d8"
>