Mutter/gnome-shell on wayland implement output rotation through DRM properties, this method is preferred over the fallback code that performs output rotation through an intermediate offscreen framebuffer. This however seems to fail on Skylake hardware for rotate-90 and rotate-270, the drmModeObjectSetProperty() call at [1] trying to set the "rotation" property fails with "Invalid argument" despite the two modes being previously obtained by parsing the properties at [2]. I filed https://bugzilla.gnome.org/show_bug.cgi?id=772512 to mutter in order to handle this more gracefully, there still seems to be something fishy about the "Invalid argument" errors. [1] https://git.gnome.org/browse/mutter/tree/src/backends/native/meta-monitor-manager-kms.c#n1360 [2] https://git.gnome.org/browse/mutter/tree/src/backends/native/meta-monitor-manager-kms.c#n564
rotate-90/270 requires the framebuffer output be Y-tiled. That requires teaching GBM or whatever mutter is using that it can use Y-tiling on gen9.
Based on closed 772512 and quick chat on IRC:intel-gfx@freenode.net marking resolved+notabug.
I forgot about this bug... you're most probably right, mutter is not doing the Y-tiling request for the affected framebuffers. However mutter is just smart enough now to fallback to the same paths that handle portrait rotation on gen<9, I filed https://bugzilla.gnome.org/show_bug.cgi?id=780141 to handle the remaining bits after the resolution of this bug. You're welcome to chime in there, since I'm unclear how to tell gbm to enable y-tiling.
Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.