Bug 98894 - Add an API for taking screenshots and recording screencasts
Summary: Add an API for taking screenshots and recording screencasts
Status: RESOLVED MOVED
Alias: None
Product: Wayland
Classification: Unclassified
Component: wayland (show other bugs)
Version: unspecified
Hardware: Other All
: medium enhancement
Assignee: Wayland bug list
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-11-29 00:20 UTC by Christian Stadelmann
Modified: 2018-06-08 23:49 UTC (History)
4 users (show)

See Also:
i915 platform:
i915 features:


Attachments

Description Christian Stadelmann 2016-11-29 00:20:31 UTC
Currently, some wayland compositors have their own screenshot and/or screencast feature, such as weston. Others, such as gnome-shell, have an additional public DBus API [1]. For KWin, a similar feature is in the works [2].

This situation is bad because it looks like application developers need to write a separate implementation for their tools for every desktop shell / environment. For the sake of simplicity, can you please add one API (probably an extension to the wayland protocol) for all desktop environments?

[1] https://git.gnome.org/browse/gnome-shell/tree/data/org.gnome.Shell.Screencast.xml
[2] https://blog.martin-graesslin.com/blog/2016/11/from-window-killing-to-screenshot/

Some notes:
* Taking screenshots and recording screencasts should be an optional feature.
* The wayland compositor should be able to decide (based on rules or user decisions) whether to allow or deny the request. For this, applications need to have a way to provide information about themselves (like application name and icon) and why they want to take a screenshot/screencast.
* Applications must handle being denied to take a screenshot/screencast
* there should be a way to select regions of the screen(s) only. I don't care whether this should be implemented on client or server (compositor) side.
Comment 1 Pekka Paalanen 2016-11-29 07:58:36 UTC
(In reply to Christian Stadelmann from comment #0)
> * Taking screenshots and recording screencasts should be an optional feature.

Right.

> * The wayland compositor should be able to decide (based on rules or user
> decisions) whether to allow or deny the request. For this, applications need
> to have a way to provide information about themselves (like application name
> and icon) and why they want to take a screenshot/screencast.

This is the very problem. If an application provides identification info itself, it can lie. This is actually the root problem for authenticating any privileged actions, and I don't think there exists a commonly agreed solution yet.

A simple solution is to have the compositor itself launch only trusted binaries (e.g. installed to the system by root, pre-declared as safe), or verifying the app is such a trusted binary which is somewhat more complicated. However, the ultimate solution would probably revolve around secured containers, where the application id is provided by the system, not the app.

Any authenticated application also needs to avoid exploitation. E.g. a command line tool that captures the whole desktop could be simply launched by a malicious app.

Giving the user a notification and inferring from user actions whether the operation was really initiated and accepted by the user would probably go a long way, but it would also be important to be able to tell the user reliably which program and for what purpose is trying a privileged action.

> * Applications must handle being denied to take a screenshot/screencast

Definitely.

> * there should be a way to select regions of the screen(s) only. I don't
> care whether this should be implemented on client or server (compositor)
> side.

Server-side. It cannot be client-side for security reasons. The user must be able to trust that the app is not getting anything outside of the region he selected.

There is also the choice of per-window vs. from the composite (the whole desktop).

Someone might also want to see a preview of the screenshot before passing it on to the application, to verify it does not contain sensitive information by accident.

Whether screenshooting and screencasting should be a Wayland extension is also an open question, as you can see from the various implementations.
Comment 2 Christian Stadelmann 2016-11-29 10:49:59 UTC
(In reply to Pekka Paalanen from comment #1)
> > * The wayland compositor should be able to decide (based on rules or user
> > decisions) whether to allow or deny the request. For this, applications need
> > to have a way to provide information about themselves (like application name
> > and icon) and why they want to take a screenshot/screencast.
> 
> This is the very problem. If an application provides identification info
> itself, it can lie. This is actually the root problem for authenticating any
> privileged actions, and I don't think there exists a commonly agreed
> solution yet.

Afaik, applications can't lie about /proc/[pid]/exe and the compositor could use .desktop files from /usr/share/applications for identification (with both name and icon). Also, there could be a flag e.g. in these .desktop files for allowing screenshots / screencasts.
Comment 3 James Henstridge 2018-02-21 14:23:21 UTC
> Afaik, applications can't lie about /proc/[pid]/exe and the compositor could
> use .desktop files from /usr/share/applications for identification (with both
> name and icon).

It is pretty easy to spoof the result of readlink() on /proc/[pid]/exe though.  With user and mount namespaces, a regular user could bind mount a different executable over a trusted one before executing it.

This could be detected by comparing stat() information, but that would also give false positives when upgrading a running application.

There's not an obvious generic solution.
Comment 4 Jonas Ådahl 2018-03-12 09:38:53 UTC
(In reply to James Henstridge from comment #3)
> This could be detected by comparing stat() information, but that would also
> give false positives when upgrading a running application.
> 
> There's not an obvious generic solution.

There is one, except I only know of GNOME who has the screenshot part implemented: https://github.com/flatpak/xdg-desktop-portal/blob/master/data/org.freedesktop.portal.Screenshot.xml
Comment 5 David Woodhouse 2018-04-30 14:27:23 UTC
At https://bitbucket.org/pidgin/main/pull-requests/330 I've added screenshare support to Pidgin based on ximagesrc which allows the user to share single windows or full screens, but it's not going to work with Wayland.
It would be great if there was a way to fix that...
Comment 6 Vladimir Stoiakin 2018-06-07 18:54:24 UTC
I am not a developer, so my suggestion can be naive.
What about to use simple password authentication for apps?
For example, some user wants to record a screencast. He sends a request to a compositor with a public API. The compositor allows to do it (if configured so), generates a one-time secret password, and draws it in the final screen buffer. The user reads the password on the screen, and enters it in the program which is going to do the job. With this password the program can start recording.
In this scenario the user knows exactly which program will get access to the screen (he relies on the shell which starts user programs), and the process of recording can be fully controlled by the compositor.
Comment 7 Christian Stadelmann 2018-06-07 19:21:45 UTC
(In reply to Vladimir Stoiakin from comment #6)
> I am not a developer, so my suggestion can be naive.
> What about to use simple password authentication for apps?
> For example, some user wants to record a screencast. He sends a request to a
> compositor with a public API. The compositor allows to do it (if configured
> so), generates a one-time secret password, and draws it in the final screen
> buffer. The user reads the password on the screen, and enters it in the
> program which is going to do the job. With this password the program can
> start recording.
> In this scenario the user knows exactly which program will get access to the
> screen (he relies on the shell which starts user programs), and the process
> of recording can be fully controlled by the compositor.

In the modern world of software development, the goal is to have unobtrusive "authentication" through implicit actions, at least for compartmentalization on desktops. Take any application, for example a web browser. Let's assume this application is somehow confined (e.g. using bwrap/flatpak or running under a different user ID as on android) and is not allowed to access your files by default. Sometimes, you want it to get access to your files, e.g. to open or save a document. This happens through the file chooser: If you choose a file using the file chooser it is assumed that you want this application to access the selected document, so access is granted for that file/folder. This is called a portal.

In case you are interested in this technology, please have a look at https://github.com/flatpak/xdg-desktop-portal/ and also this great explanation by Alexander Larsson: https://blogs.gnome.org/alexl/2017/01/24/the-flatpak-security-model-part-3-the-long-game/
For the example above, this portal is called FileChooser portal.

This bug report is asking to add an API to the wayland protocol so that clients (applications) can request to take a screenshot and the compositor / desktop shell can decide whether to grant this request. It is also (more importantly) about adding one API for any wayland compositor so that client application developers don't have to write separate code for every single compositor they want to support. Btw: xdg-desktop-portal already has an API for a Screenshot and a ScreenCast portal which may be interpreted as a fix to this bug. It also fixes the issues raised by Pekka Paalanen in comment #1. See also the comment #4 by Jonas Ådahl.
Comment 8 GitLab Migration User 2018-06-08 23:49:38 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/wayland/wayland/issues/32.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.