Bug 101436

Could you also clarify what "screen size" and "monitor size" mean and what they are used for via Xwayland? I kind of know what they are in X11, but I have no idea how X11 applications use them.

E.g. for fullscreening, I would hope apps used WM protocols to ask what their fullscreen size should be, but I understand that's wishful thinking. So how exactly are these details used?

This report sound like a continuation of the long irc discussion I had with Olivier Fourdan, where talked about how one could support HiDPI and mixed-dpi systems with Xwayland. Should we include that topic here as well?

You now want to support fractional scaling behind applications' back. Was this not foreseen when the wl_output and wl_buffer scale protocol was designed? What has changed? Why is it now feasible?

(In reply to Pekka Paalanen from comment #1)
> Could you also clarify what "screen size" and "monitor size" mean and what
> they are used for via Xwayland? I kind of know what they are in X11, but I
> have no idea how X11 applications use them.

They may for example position a popup menu in relation to the edges. For example if you open context menu far to the right in Firefox, it'll expand to the left instead of right if it wouldn't fit to the right. They also tend to use it to be "smart" about an initial size, e.g. to cover N % of the screen space or something.

> 
> E.g. for fullscreening, I would hope apps used WM protocols to ask what
> their fullscreen size should be, but I understand that's wishful thinking.
> So how exactly are these details used?
> 

How maximize and fullscreen works I'm not very familiar with.

> This report sound like a continuation of the long irc discussion I had with
> Olivier Fourdan, where talked about how one could support HiDPI and
> mixed-dpi systems with Xwayland. Should we include that topic here as well?

Mixed DPI systems is also covered in https://bugs.freedesktop.org/show_bug.cgi?id=93315 . But yea, mixed DPI systems are relevant here. Are the conclusions / raised issues etc from that discussion summarized anywhere?

> 
> You now want to support fractional scaling behind applications' back. Was
> this not foreseen when the wl_output and wl_buffer scale protocol was
> designed? What has changed? Why is it now feasible?

Currently it is done by telling clients to scale with ceilf(scale) then downscale. For example if you scale the area of a monitor with scale 1.5, you'll tell clients to draw at 2, then downscale to 1.5.

Right now this works by pretending that the mode of wl_output is the logical monitor size. Xwayland windows will of course still always to be considered scale 1, and upscaled to 1.5.

I somewhat touched on it in this E-mail: https://mail.gnome.org/archives/gnome-shell-list/2017-June/msg00000.html

(In reply to Jonas Ådahl from comment #2)
> (In reply to Pekka Paalanen from comment #1)
> [...]
> > 
> > E.g. for fullscreening, I would hope apps used WM protocols to ask what
> > their fullscreen size should be, but I understand that's wishful thinking.
> > So how exactly are these details used?
> > 
> 
> How maximize and fullscreen works I'm not very familiar with.

For maximized state and fullscreen, x11 apps use the relevant EWMH protocol:

 * _NET_WM_STATE for maximized and "normal" fullscreen [1]
 * _NET_WM_FULLSCREEN_MONITORS for fulscreen across different monitors [2]

The later uses Xinerama monitor indices to list the monitors (!)

[1] https://specifications.freedesktop.org/wm-spec/wm-spec-latest.html#idm140200472615568
[2] https://specifications.freedesktop.org/wm-spec/wm-spec-latest.html#idm140200472531472

(In reply to Jonas Ådahl from comment #2)
> (In reply to Pekka Paalanen from comment #1)
> > This report sound like a continuation of the long irc discussion I had with
> > Olivier Fourdan, where talked about how one could support HiDPI and
> > mixed-dpi systems with Xwayland. Should we include that topic here as well?
> 
> Mixed DPI systems is also covered in
> https://bugs.freedesktop.org/show_bug.cgi?id=93315 . But yea, mixed DPI
> systems are relevant here. Are the conclusions / raised issues etc from that
> discussion summarized anywhere?

Not written down, no - we didn't really reach a conclusion. The end result was that we were both confused about Mutter's way of doing things, which seemed to be fundamentally different from Weston's. So different that we would actually need two different operating modes in Xwayland to handle them, and that was before the idea of fractional scaling. It seemed like Mutter's current design could never support mixed-dpi setups with Xwayland.

https://people.freedesktop.org/~cbrill/dri-log/index.php?channel=wayland&date=2017-06-06
From 13:00 to 14:25.

I wonder if it is at all possible to even have X11 windows with drawn scales (app-side scaling). That seems fundamentally incompatible with compositor scaling, given the requirement of a consistent global coordinate system used for both input and output in X11.

> > 
> > You now want to support fractional scaling behind applications' back. Was
> > this not foreseen when the wl_output and wl_buffer scale protocol was
> > designed? What has changed? Why is it now feasible?
> 
> Currently it is done by telling clients to scale with ceilf(scale) then
> downscale. For example if you scale the area of a monitor with scale 1.5,
> you'll tell clients to draw at 2, then downscale to 1.5.
> 
> Right now this works by pretending that the mode of wl_output is the logical
> monitor size. Xwayland windows will of course still always to be considered
> scale 1, and upscaled to 1.5.
> 
> I somewhat touched on it in this E-mail:
> https://mail.gnome.org/archives/gnome-shell-list/2017-June/msg00000.html

But why was that implemented? Is fractional scaling not too ugly to be used after all? Why do you want it? What is different from the time the scaling feature was designed? If it's really needed, then I feel we made a mistake with the integer-only scales design in the protocol. Should we have also a divisor for the scale factor?

The email does not explain why, but it seems to contradict some claims that arose during my and Olivier's irc discussion.

From the beginning, wl_output's modes are the video modes on the hardware which means that if a client aims for perfect fit with a fullscreen, directly scanned out buffer, it needs to make the buffer size exactly the advertized mode size. I think this is the fundamental feature we need to keep intact. It is a bypass of the shell protocols unfortunately, I'm not sure we can make that go away.

If RandR information in Xwayland needs to be different from the hardware video modes, I believe that should be computed in Xwayland, otherwise we screw up all Wayland clients; or, sending different parameters to one client (Xwayland) vs. others in a non-Xwayland-specific interface is far too subtle to rely on IMO.

I think it would be ok for shell protocols to offer alternative fullscreen window sizing methods that worked in the logical pixels instead of output hardware pixels - essentially xdg_shell already does that AFAIU. Fitting this together with the wl_output size is awkward, but I think it is doable with wl_viewport: you make the buffer with the wl_output size and use wl_viewport to make the window size match what the shell protocol asks for. That might allow for fractional scaling combined with a direct scanout path for e.g. native Wayland games.

But all that seems fundamentally incompatible with Xwayland. I think games there need to just rely on display hardware doing the scaling since drawing a buffer matching the hardware mode does not seem possible for output scales other than one.

Is there something in the above that just cannot be made to work with Mutter or denies features you want like the fractional scaling?

Should we look forward to deprecating wl_buffer scale from the protocol (as in, recommend it to be always left to one) if we use wl_viewport as a recommended solution?

(In reply to Pekka Paalanen from comment #4)
> (In reply to Jonas Ådahl from comment #2)
> > (In reply to Pekka Paalanen from comment #1)

... snip ..
 
> > > 
> > > You now want to support fractional scaling behind applications' back. Was
> > > this not foreseen when the wl_output and wl_buffer scale protocol was
> > > designed? What has changed? Why is it now feasible?
> > 
> > Currently it is done by telling clients to scale with ceilf(scale) then
> > downscale. For example if you scale the area of a monitor with scale 1.5,
> > you'll tell clients to draw at 2, then downscale to 1.5.
> > 
> > Right now this works by pretending that the mode of wl_output is the logical
> > monitor size. Xwayland windows will of course still always to be considered
> > scale 1, and upscaled to 1.5.
> > 
> > I somewhat touched on it in this E-mail:
> > https://mail.gnome.org/archives/gnome-shell-list/2017-June/msg00000.html
> 
> But why was that implemented? Is fractional scaling not too ugly to be used
> after all? Why do you want it? What is different from the time the scaling
> feature was designed? If it's really needed, then I feel we made a mistake
> with the integer-only scales design in the protocol. Should we have also a
> divisor for the scale factor?
> 
> The email does not explain why, but it seems to contradict some claims that
> arose during my and Olivier's irc discussion.

The reason is mostly because some monitor resolutions doesn't look good on neither scale 1 or scale 2. In the X11 world, people tend to use xrandr scale hacks to scale things some way, and we but we want to do it a bit better on Wayland by at least only scaling down, instead of scaling up.

As far as I know, this is similar to how certain HiDPI aware OS:es does it (draw larger and scale down) and (if I haven't misunderstood) still how QT does it client side.

> 
> From the beginning, wl_output's modes are the video modes on the hardware
> which means that if a client aims for perfect fit with a fullscreen,
> directly scanned out buffer, it needs to make the buffer size exactly the
> advertized mode size. I think this is the fundamental feature we need to
> keep intact. It is a bypass of the shell protocols unfortunately, I'm not
> sure we can make that go away.

The problem is that with fractional scaling, it's not possible to align things so they still match a pixel grid. With integer scaling, we don't have that problem as when we say "use logical size 100x100", a scale two client will just create a 200x200 size buffer. When we use fractional scaling (for example 1.7391304347826086), how should a client create a 173.913x173.913 pixels large buffer?

By just passing scale 2, we avoid all the protocol issues, and deal with it exclusively in the compositor only during the paint stage.

The ways we can deal with this to avoid any scaling is either to dive into dealing with half pixels (by adding policy about rounding and what not I suppose) and all the related issues, or we allow a way to bypass the scaling factors when we know we won't have any half pixel issues (for example, in the mutter implementation, a fullscreen window will always align perfectly with real pixels).

> 
> If RandR information in Xwayland needs to be different from the hardware
> video modes, I believe that should be computed in Xwayland, otherwise we
> screw up all Wayland clients; or, sending different parameters to one client
> (Xwayland) vs. others in a non-Xwayland-specific interface is far too subtle
> to rely on IMO.

Yea, the fact that we pass the scaled mode to Xwayland now is just because Xwayland simply works that way right now. I opened this so we can eventually actually pass real mode values to wl_output while not having X11 clients go nuts.

A simple "wp_xwayland_configure_screen" that takes a bunch of logical coordinate space rects might even do. If we pass any any fractional scales we'd have to deal with precision loss errors.

We could possibly set the current xrandr configuration to the current logical pixel configuration, while listing the actual modes as alternatives, then allow clients to override the mode under certain circumstances (see below).

> 
> I think it would be ok for shell protocols to offer alternative fullscreen
> window sizing methods that worked in the logical pixels instead of output
> hardware pixels - essentially xdg_shell already does that AFAIU. Fitting
> this together with the wl_output size is awkward, but I think it is doable
> with wl_viewport: you make the buffer with the wl_output size and use
> wl_viewport to make the window size match what the shell protocol asks for.
> That might allow for fractional scaling combined with a direct scanout path
> for e.g. native Wayland games.

Hmm. Interesting idea to use wp_viewporter (previously wl_viewport) together with wl_output modes. One potential issue would be if a client would want to combine multiple outputs into one large texture that is drawn. I guess setting up multiple viewports of the same wl_surface would work though.

> 
> But all that seems fundamentally incompatible with Xwayland. I think games
> there need to just rely on display hardware doing the scaling since drawing
> a buffer matching the hardware mode does not seem possible for output scales
> other than one.
> 
> Is there something in the above that just cannot be made to work with Mutter
> or denies features you want like the fractional scaling?

I can't think of anything that would make that impossible. The issue however is that we still wont change any mode by just using wp_viewporter. For example, maybe a game/video player wants to use another framerate, or a video player wants to enable interlacing.

> 
> Should we look forward to deprecating wl_buffer scale from the protocol (as
> in, recommend it to be always left to one) if we use wl_viewport as a
> recommended solution?


I think it's fairly incompatible indeed, but for isolated use cases, I think it could be made to work - not without issues though. A potential way it could work is as follows:

1) Game A starts, the current mode is the logical mode
( The following two might not happen )
2) Game A doesn't change any mode and starts, and will be scaled up by the compositor
3) Game A allows the user to change resolution, and will list the native resolution among the choices - the user selects the native resolution
)
4) Game A sets the native resolution
5) Xwayland pretends the resolution change actually succeeded, knowing what client did it
6) Game A will resize/create a new Window and make it fullscreen
7) Xwayland will see that an X11 client created a window that matches the fake resolution and will issue a fullscreen-with-mode-change (or some other equivalent method using wp_viewporter or something else)
8.1) If the mode change was approved given compositor policy, the output mode will be changed, and the content of the X11 client can now shown without being scaled using the native resolution
8.2) If the mode change as denied, this would have to be communicated too (I guess we could rely on wl_output updates here though) 

For HiDPI aware clients, I wonder if we can't introduce a EWMH kind of thing for multi DPI aware clients that can do the equivalent of setting wl_surface::buffer_scale while making it possible to let the compositor know how popups should be transformed. That is not strictly related to this issue though.

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/xorg/xserver/issues/704.