Bug 16799 - hibernate broken on radeon with latest libdrm git
Summary: hibernate broken on radeon with latest libdrm git
Status: RESOLVED FIXED
Alias: None
Product: xorg
Classification: Unclassified
Component: Driver/Radeon (show other bugs)
Version: git
Hardware: Other All
: medium normal
Assignee: xf86-video-ati maintainers
QA Contact: Xorg Project Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-07-21 12:34 UTC by Stefan Becker
Modified: 2008-07-28 00:47 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments
Add suspend/resume hooks to radeon DRM driver (1008 bytes, patch)
2008-07-22 01:27 UTC, Stefan Becker
no flags Details | Splinter Review
Add suspend/resume hooks to radeon DRM driver (1.40 KB, patch)
2008-07-22 03:09 UTC, Stefan Becker
no flags Details | Splinter Review

Description Stefan Becker 2008-07-21 12:34:00 UTC
It seems the latest vblank changes break hibernate support (at least) on radeon. The system freezes when the kernel gets told to dump to swap.

The last good commit is:

commit 7cfdba2b30e40efc688f1704bd4f4141dc6f9a6c
Author: Dave Airlie <airlied@redhat.com>
Date:   Fri Jul 18 14:30:57 2008 +1000

    radeon: remove microcode version

I'm not sure if this needs fixing in the driver or the drm kernel module. But I can only test on radeon, so that's why I report it here first.


I'm using Fedora 9 on a Lenovo T60 (RV515). I have update mesa, libdrm and xf86-video-ati from git:

mesa:
commit 77497eb73b9aa349f41f3bcb493d84610e302371
Author: Brian Paul <brian.paul@tungstengraphics.com>
Date:   Mon Jul 21 09:01:21 2008 -0600

    mesa: revert building glslcompiler by default

xf86-video-ati:
commit f9034214f070fe3054cd9cefd7034252de234f38
Author: Michel Dänzer <michel@tungstengraphics.com>
Date:   Mon Jul 21 09:09:02 2008 +0200

    Call DRM modeset ioctl after the IRQ has been initialized.

    This lets the DRM know it can safely disable the vblank interrupts.
Comment 1 Michel Dänzer 2008-07-21 14:04:40 UTC
Are you using compiz? If so, are you setting vblank_mode to 2 or 3 in /etc/drirc?
Comment 2 Jesse Barnes 2008-07-21 14:36:49 UTC
Weird... when you say hibernate you mean suspend to disk, right?  The sequence there should be:
  1) server & driver do LeaveVT
  2) drm driver will suspend (if present, maybe not on radeon though?)
  3) machine sleeps
  4) machine boots normally
  5) memory image is restored
  6) drm driver will resume (again maybe not on radeon)
  7) server & driver do EnterVT
Does the radeon X driver do any pre/post modeset ioctl calls?  Also on resume does it re-enable vblank interrupts?
Comment 3 Stefan Becker 2008-07-21 22:20:34 UTC
(In reply to comment #1)
> Are you using compiz? If so, are you setting vblank_mode to 2 or 3 in
> /etc/drirc?

It also happens when compiz is not running, e.g. machine has rebooted and only X server plus GDM is running. 

Comment 4 Stefan Becker 2008-07-21 22:24:38 UTC
(In reply to comment #2)
> Weird... when you say hibernate you mean suspend to disk, right?  The sequence
> there should be:
>   1) server & driver do LeaveVT

Should that disable vblank interrupts?

>   2) drm driver will suspend (if present, maybe not on radeon though?)

radeon DRM driver does not implement suspend/resume currently.

> Does the radeon X driver do any pre/post modeset ioctl calls?  Also on resume
> does it re-enable vblank interrupts?

I think I'm going to try to hack suspend/resume functions into the DRM code just to make sure...
Comment 5 Michel Dänzer 2008-07-21 23:46:24 UTC
(In reply to comment #4)
> (In reply to comment #2)
> > Weird... when you say hibernate you mean suspend to disk, right?  The sequence
> > there should be:
> >   1) server & driver do LeaveVT
> 
> Should that disable vblank interrupts?

Possibly, but with only GDM, there shouldn't be anything waiting for vblank.

> > Does the radeon X driver do any pre/post modeset ioctl calls?  

Sure... on each modeset as well as across LeaveVT/EnterVT.

(Of course, only if the driver was built against libdrm headers with modeset ioctl definitions)

> > Also on resume does it re-enable vblank interrupts?

Yes. But if I'm reading the initial report correctly, the problem occurs at suspend time.

FWIW, suspend to RAM still works fine for me.
Comment 6 Stefan Becker 2008-07-22 00:01:01 UTC
I just found out that suspend-to-RAM works fine.

After a successful suspend-to-RAM also suspend-to-Disk (hibernate) works OK *UNTIL* you restart the X server. I guess suspend-to-RAM disables vblank interrupts and they don't get restored until the X server is restarted...

Suspend-to-Disk actually doesn't freeze the machine, I just didn't wait long enough. Suspend-to-Disk returns after about 2 minutes, but the machine has become unusable (/ is gone). I guess the pm tools can't recover from a freeze failure at this state anymore.
Comment 7 Michel Dänzer 2008-07-22 01:04:12 UTC
(In reply to comment #6)
> After a successful suspend-to-RAM also suspend-to-Disk (hibernate) works OK
> *UNTIL* you restart the X server. I guess suspend-to-RAM disables vblank
> interrupts and they don't get restored until the X server is restarted...

Not really, vblank interrupts are still working fine here after suspend to RAM.

Given the commit that introduced the problem for you, it seems likely the problem is related to vblank interrupts occurring at suspend time. So a DRM suspend hook disabling them could indeed be the proper solution, but isn't generally practical without kernel modesetting, as some platforms (like the PowerBook I'm using) still require radeonfb for console. In the meantime, maybe xf86-video-ati could disable interrupts in LeaveVT and re-enable them in EnterVT as a workaround. If you don't know how to try this, I can try whipping up a test patch.
Comment 8 Stefan Becker 2008-07-22 01:27:54 UTC
Created attachment 17814 [details] [review]
Add suspend/resume hooks to radeon DRM driver

(In reply to comment #7)
> PowerBook I'm using) still require radeonfb for console. In the meantime, maybe
> xf86-video-ati could disable interrupts in LeaveVT and re-enable them in
> EnterVT as a workaround. If you don't know how to try this, I can try whipping
> up a test patch.

The attached horrible quick-and-dirty hack for libdrm fixes the problem for me.

If you have something better to test please go ahead...
Comment 9 Stefan Becker 2008-07-22 03:09:45 UTC
Created attachment 17819 [details] [review]
Add suspend/resume hooks to radeon DRM driver

As discussed on #radeon: updated suspend/resume hooks that clear/restore the radeon interrupt register contents.
Comment 10 Stefan Becker 2008-07-22 22:44:14 UTC
Still investigating an alternative implementation...

The X11 driver uses RADEONDRISetVBlankInterrupt() which issues the ioctl (DRM_RADEON_SETPARAM, RADEON_SETPARAM_VBLANK_CRTC, value). That ends up in the DRM code in radeon_irq.c:

int radeon_vblank_crtc_set(struct drm_device *dev, int64_t value)
{
	drm_radeon_private_t *dev_priv = (drm_radeon_private_t *) dev->dev_private;
...
	dev_priv->vblank_crtc = (unsigned int)value;
	return 0;
}

But vblank_crtc is not used anywhere. So when the X driver wants to disable vblank interrupts nothing really happens.

Shouldn't the above function update the interrupt registers?
Comment 11 Stefan Becker 2008-07-23 00:28:51 UTC
Adding interrupt register to radeon_vblank_crtc_set() didn't fix the problem. When I checked when it is called I could only see calls with value 0, so I can only assume that something else is still enabling the interrupts.

So adding the suspend/resume hooks seems currently the only working option.
Comment 12 Michel Dänzer 2008-07-23 00:43:28 UTC
(In reply to comment #10)
> Shouldn't the above function update the interrupt registers?

No. This is an old scheme which was supposed to disable the vblank interrupts while there are no 3D clients. This was obviously useless with compiz, and generally quirky at best (e.g. I think direct rendering clients could end up already waiting for vblank before the X driver got around to enabling the interrupt). It's been superseeded by vblank-rework, of which the commit causing this problem for you is a part. From my POV the old interface could only be useful anymore for the 3D driver to find out which CRTC(s) it can wait for vblank on.

P.S. I'm already getting two copies of each update via the two lists, please don't add me to CC... also I think Jesse has been exonerated. ;)
Comment 13 Stefan Becker 2008-07-23 00:51:40 UTC
OK, can you then push the suspend/resume hook to drm git instead?
Comment 14 Michel Dänzer 2008-07-23 00:57:08 UTC
(In reply to comment #13)
> OK, can you then push the suspend/resume hook to drm git instead?

Will do once I've confirmed it doesn't break my setup, unless Dave objects - he didn't seem too hot on it on IRC yesterday.
Comment 15 Michel Dänzer 2008-07-28 00:47:24 UTC
Patch pushed to drm Git, thanks.

commit 514c05cebe31a62f827a76f348d35596bef97811
Author: Stefan Becker <stefan.becker@nokia.com>
Date:   Sat Jul 26 16:49:14 2008 +0200

    radeon: Add suspend/resume hooks for saving/clearing/restoring interrupts.

    Fixes http://bugs.freedesktop.org/show_bug.cgi?id=16799 .


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.