when the xwayland module is loaded, any app can call x11 to see the whole screen, not just its own windows.
to see this, run "xwd -root > root.xwd" and open with gimp.
can this be fixed so x11 can only see other x11 apps?
Which compositor did you test this on?
According to my understanding of how Xwayland works, Xwayland never has access to the whole screen, unless that whole screen is made of X11 windows. We do not have a mechanism to forward native Wayland window contents to Xwayland and to X11 apps AFAIK, unless the compositor goes out of its way to push the composited image back to Xwayland as well as to screen.
Someone might argue, that X11 screen capturing should continue to work with Xwayland, catching also Wayland apps, so maybe they did it on purpose?
On Weston, X11 apps should not see any Wayland windows ever. It'll be a while before I can test myself, though.
Tried this in both mutter and weston and all I get is
X Error of failed request: BadMatch (invalid parameter attributes)
Major opcode of failed request: 73 (X_GetImage)
Serial number of failed request: 810
Current serial number in output stream: 810
and an empty 'root.xwd'.
whatevers default in fedora25. it looked like gnome-shell in pstree.
the last update seems to have fixed it. what i didnt mention in the last report was a black border around all windows. if your curious, i saved that window dump, but now im also getting the failed request. so its good now :)
the fix still allows xwd without -root, and only works on x11 windows. would have preferred no such functionality, but i guess its a compromise.