Bug 103611 - In a stress test of 500 cycles of s4, it fails before 80 cycles, PM: Failed to load hibernation image
Summary: In a stress test of 500 cycles of s4, it fails before 80 cycles, PM: Failed t...
Status: CLOSED WORKSFORME
Alias: None
Product: DRI
Classification: Unclassified
Component: DRM/Intel (show other bugs)
Version: DRI git
Hardware: x86-64 (AMD64) Linux (All)
: medium major
Assignee: Intel GFX Bugs mailing list
QA Contact: Intel GFX Bugs mailing list
URL:
Whiteboard: ReadyForDev
Keywords:
Depends on:
Blocks:
 
Reported: 2017-11-07 17:27 UTC by Hector Velazquez
Modified: 2018-04-25 09:33 UTC (History)
2 users (show)

See Also:
i915 platform: SKL
i915 features: power/suspend-resume

Attachments
suspend_stress_test.shell.sh file (13.02 KB, application/x-shellscript)
2017-11-07 17:27 UTC, Hector Velazquez 		no flags Details
dmesg log (211.07 KB, text/plain)
2017-11-07 17:28 UTC, Hector Velazquez 		no flags Details
kernel log (288.29 KB, text/plain)
2017-11-07 17:29 UTC, Hector Velazquez 		no flags Details
Output file (14.71 KB, text/plain)
2017-11-07 17:30 UTC, Hector Velazquez 		no flags Details
s4-log (7.69 MB, text/plain)
2018-04-12 15:35 UTC, Elizabeth 		no flags Details
View All

Description Hector Velazquez 2017-11-07 17:27:41 UTC 
Created attachment 135285 [details]
suspend_stress_test.shell.sh file

This test fails in a SKL:

Launch 500 iterations of suspend to disk

======================================
	     Issue description
======================================

In a stress test of 500 cycles of s4, it fails before getting to the 80 cycles because the system reboots...

======================================
	     Steps to reproduce
======================================
1)Setup an SKL with Ubuntu 17.04
  make sure setting up during ubuntu installation a separate swap partition (at least the same size of the ram...)
    my partition map is:
     sda    8:0    0 223.6G  0 disk
   ├─sda1   8:1    0   476M  0 part /boot/efi
   ├─sda2   8:2    0  39.1G  0 part [SWAP]
   └─sda3   8:3    0 181.2G  0 part /
  
2)Install the kernel 4.13.5-041305-generic_4.13.5-041305.201710050600_amd64 from kernel.org
  a)In a Terminal execute...
    I)  $sudo apt update
    II) $sudo apt upgrade
	
3)Install full graphic stack with the components and commits described below in the Graphic stack section...
  a)In a Terminal execute...
    I)  $sudo apt update
    II) $sudo apt upgrade
	
4)make sure to add this line: 'resume=/dev/sda2' in the grub to perform the S4...
  a)execute $sudo nano /etc/default/grub
  b)add the swap partition in the line: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash resume=/dev/sda2"
  c)sudo update-grub
     Note: you can verified the swap location with the 'swapon' command...
     NAME      TYPE       SIZE USED PRIO
     /dev/sda2 partition 39.1G   0B   -1
  d)Reboot the system to apply changes... 

5)Board is powered on, a Terminal is opened...
  a)Remove kernel boot command lines, except drm.debug=0xe and no_console_suspend
  c) Execute commands in a terminal:
     sudo rm /var/log/kern.log 
     sudo reboot now
     sudo -s
     chmod 755 suspend_stress_test.shell
	 Note: the script 'suspend_stress_test.shell' is an attachement...

6)Execute command:
  ./suspend_stress_test.shell --iterations=500 --mode=disk --suspend=58-62 --wake=28-32 --abort=none --display=error --file=/home/$USER/Desktop/s4-log
     
======================================
	    Spected Results
======================================

Suspend to DISK and resume states are reached each time. There is no error printed by the script, no instability (crash, freeze, black screen, mouse/keyboard not responding)
The system should be complete the 500 S4 cycles...



This is my configuration:

======================================
	    Graphic stack
======================================
Component: drm
    tag: libdrm-2.4.83-10-g2ecafca
    commit: 2ecafcae8a215d9994fb26a122d97bcb5437c5e8

Component: mesa
    tag: mesa-17.1.10
    commit: 834d6c60db266c7d7dfd973729f20379dd3da287

Component: macros
    tag: util-macros-1.19.1-2-g39f07f7
    commit: 39f07f7db58ebbf3dcb64a2bf9098ed5cf3d1223

Component: xproto
    tag: xproto-7.0.31-6-gab86666
    commit: ab8666661fc68f075b8d6ffabe22c6b577c30ac1

Component: glproto
    tag: glproto-1.4.17-5-g500b137
    commit: 500b137bbdba1cd8ae943849fef6e79fdd4eca40

Component: dri2proto
    tag: dri2proto-2.8-4-gb118dfb
    commit: b118dfbf91dcec6d82dfddc3f41031e23ea3c039

Component: xserver
    tag: xorg-server-1.19.0-433-gc2f2b25
    commit: c2f2b25ab55c67f9f3ad07c02fa746eae7c61196

Component: libXfont
    tag: libXfont2-2.0.1-14-g672bb94
    commit: 672bb944311392e2415b39c0d63b1e1902905bcd

Component: xf86-input-evdev
    tag: xf86-input-evdev-2.10.5-3-g8c1058f
    commit: 8c1058f2ebd9355068fb1b67329f3428ca64775b

Component: xf86-input-libinput
    tag: xf86-input-libinput-0.26.0
    commit: 2be6487de417473aac85ebd800392cdd8604c4a6

Component: xf86-video-fbdev
    tag: xf86-video-fbdev-0.4.4-11-g3cf9923
    commit: 3cf99231199bd5bd9e681e85d9da1f9eb736e3e7

Component: xf86-video-vesa
    tag: xf86-video-vesa-2.3.4-5-gb9f9c95
    commit: b9f9c95ca2383460aa283adeeee6e0a66eed722b

Component: xf86-video-vmware
    tag: xf86-video-vmware-13.0.2-55-g08ad438
    commit: 08ad438e2793248452dab100822cbfcaa05b9938

Component: xf86-video-qxl
    tag: xf86-video-qxl-0.1.5-9-gee8f904
    commit: ee8f904ab0d590c741e640e9548c472e6a58b3cc

Component: xf86-video-chips
    tag: xf86-video-chips-1.2.7-5-gc2711ee
    commit: c2711eedaac20af973721111a909a6f575078410

Component: x11proto
    tag: xproto-7.0.31-6-gab86666
    commit: ab8666661fc68f075b8d6ffabe22c6b577c30ac1

Component: libxtrans
    tag: xtrans-1.3.5-12-g2836667
    commit: 28366676effaa512e43bfd2276a317389a992511

Component: libX11
    tag: libX11-1.6.5-21-ge835a9d
    commit: e835a9dcc3362b5e92893be756dd7ae361e64ced

Component: libXext
    tag: libXext-1.3.3-7-ga07b4bb
    commit: a07b4bb8290d0c1bba7bcecd5bb6896fbe1b169c

Component: xrdb
    tag: xrdb-1.1.0-15-gae86081
    commit: ae86081a92522653ff1523c92524ff892f75d496

Component: xf86-video-intel
    tag: 2.99.917-789-g291fdcd
    commit: 291fdcdc09420d8e422617070731eb3da3d55b44

Component: xkbcomp
    tag: xkbcomp-1.4.0
    commit: 3fcd5f4490adee266eb4c0110d058c7f4e515e36

Component: xf86-input-wacom
    tag: xf86-input-wacom-0.34.2-12-g2456f82
    commit: 2456f821ba368fae5c084b44e86b8754187670c0

Component: pixman
    tag: pixman-0.33.6-24-g85467ec
    commit: 85467ec308f8621a5410c007491797b7b1847601

Component: libpciaccess
    tag: libpciaccess-0.13.5-1-ga167bd6
    commit: a167bd6474522a709ff3cbb00476c0e4309cb66f

Component: libinput
    tag: 1.8.1
    commit: cc9a4debd3889a3b3a5139576ea873eebcf7dde7

Component: xkeyboard-config
    tag: xkeyboard-config-2.22-1-ga6cba79
    commit: a6cba79b4ba1760b348db5145f597514e1773a16

Component: xf86-input-mouse
    tag: xf86-input-mouse-1.9.2-4-g3c8f243
    commit: 3c8f243b750a92d5837a449d344ff884dbd02b57

Component: xf86-input-keyboard
    tag: xf86-input-keyboard-1.9.0-3-g940f441
    commit: 940f44149d1037cfc14bbb3628044a2bd002c33e

Component: xf86-input-synaptics
    tag: xf86-input-synaptics-1.9.0-2-gafa8d7b
    commit: afa8d7bb469e7ce01d9239f0dd790d3d620bfbd6

Component: libva
    tag: 2.0.0.pre2
    commit: d6a40680bc41b5365eab4924bed951ddb55b5ab9

Component: libva-utils
    tag: 2.0.0.pre2
    commit: 4341558abc0b1cd705b8f1fb06dd2fe6d2774886

Component: intel-vaapi-driver
    tag: 2.0.0.pre2
    commit: bd195529797c180d9efead59986b9128f0557885

Component: cairo
    tag: 1.15.8-17-g5a9dba9
    commit: 5a9dba9929d9c9046705150f7e3c4783ff34e4dd

Component: intel-gpu-tools
    tag: intel-gpu-tools-1.20
    commit: 7f93a2632aae7c5865823b4a2fa4cd8c2a1c0977

Component: piglit
    tag: piglit-v1
    commit: 772955db4b29ee515d9553ebbaff49905effc430

======================================
	     Software
======================================
kernel version              : 4.13.5-041305-generic
hostname                    : SkullCanyon
architecture                : x86_64
os version                  : Ubuntu 17.04
os codename                 : zesty
kernel driver               : i915
bios revision               : 5.6
bios release date           : 09/29/2016
hardware acceleration       : enabled
swap partition              : enabled on (/dev/sda2)

======================================
	Graphic drivers
======================================
mesa                        : 17.1.10 (git-834d6c60
modesetting                 : enabled
modesetting compiled for    : 1.19.99.1 X.Org Video Driver
libdrm                      : 2.4.83
vaapi (intel-driver)        : Intel i965 driver for Intel(R) Skylake - 2.0.0.pre2 (2.0.0.pre2)
cairo                       : 1.15.9
intel-gpu-tools (tag)       : intel-gpu-tools-1.20
intel-gpu-tools (commit)    : 7f93a263

======================================
	     Hardware
======================================
platform                   : Skylake Canyon
motherboard id             : NUC6i7KYB
form factor                : Desktop
cpu family                 : Core i7
cpu family id              : 6
cpu information            : Intel(R) Core(TM) i7-6770HQ CPU @ 2.60GHz
gpu card                   : Intel Corporation Iris Pro Graphics 580 (rev 09) (prog-if 00 [VGA controller])
memory ram                 : 31.31 GB
max memory ram             : 32 GB
display resolution         : 1920x1080
cpu thread                 : 8
cpu core                   : 4
cpu model                  : 94
cpu stepping               : 3
socket                     : Other
signature                  : Type 0, Family 6, Model 94, Stepping 3
hard drive                 : 223GiB (240GB)
current cd clock frequency : 337500 kHz
maximum cd clock frequency : 675000 kHz
displays connected         : DP-1 DP-2 DP-3

======================================
	     Firmware
======================================
dmc fw loaded             : yes
dmc version               : 1.26
guc fw loaded             : SUCCESS
guc version wanted        : 6.1
guc version found         : 6.1
huc fw loaded             : yes

======================================
	     kernel parameters
======================================
quiet splash drm.debug=0xe pci=pcie_bus_safe intel_iommu=igfx_off i915.alpha_support=1 i915.enable_guc_loading=2 i915.enable_guc_submission=2 resume=/dev/sda2
Comment 1 Hector Velazquez 2017-11-07 17:28:54 UTC 
Created attachment 135286 [details]
dmesg log
Comment 2 Hector Velazquez 2017-11-07 17:29:24 UTC 
Created attachment 135287 [details]
kernel log
Comment 3 Hector Velazquez 2017-11-07 17:30:14 UTC 
Created attachment 135288 [details]
Output file
Comment 4 Hector Velazquez 2017-11-07 18:38:12 UTC 
Sorry, i forget an important step after step 3: 

3a)Install the latest Firmwares from https://01.org/linuxgraphics/downloads/firmware
  a)extract all packages in the download directory
    I) In a terminal with the command: tar -xvf FileName.bz2
	II)enter to the paths skl_GUC... and execute sudo ./install.sh for each..
  b)execute $sudo nano /etc/default/grub
  c)add the enabled firmware parameters in the line: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.enable_guc_submission=2 i915.enable_guc_loading=2 i915.alpha_support=1 resume=/dev/sda2"
  d)sudo update-grub

continue with step 4...
Comment 5 Elizabeth 2017-11-09 19:45:45 UTC 
I guess this is the moment when it failed:

From dmesg:
[    2.851311] PM: Starting manual resume from disk
[    2.851312] PM: Hibernation image partition 8:2 present
[    2.851313] PM: Looking for hibernation image.
[    2.852181] PM: Image signature found, resuming
[    2.852444] PM: Preparing processes for restore.
[    2.852445] Freezing user space processes ... (elapsed 0.001 seconds) done.
[    2.853598] OOM killer disabled.
[    2.853599] PM: Loading hibernation image.
[    2.853740] PM: Marking nosave pages: [mem 0x00000000-0x00000fff]
[    2.853741] PM: Marking nosave pages: [mem 0x00058000-0x00058fff]
[    2.853742] PM: Marking nosave pages: [mem 0x0009f000-0x000fffff]
[    2.853743] PM: Marking nosave pages: [mem 0x31174000-0x311befff]
[    2.853744] PM: Marking nosave pages: [mem 0x3a1c2000-0x3b2fdfff]
[    2.853793] PM: Marking nosave pages: [mem 0x3b2ff000-0xffffffff]
[    2.854909] PM: Basic memory bitmaps created
[    2.912424] PM: Using 3 thread(s) for decompression.
[    2.912424] PM: Loading and decompressing image data (813171 pages)...
[    2.984281] PM: Image loading progress:   0%
[    3.483199] random: crng init done
[    3.668234] PM: Image loading progress:  10%
[    3.808281] [drm] RC6 on
[    4.053623] PM: Image loading progress:  20%
[    4.471774] PM: Image loading progress:  30%
[    4.963610] PM: Image loading progress:  40%
[    5.458739] PM: Image loading progress:  50%
[    5.922511] PM: Image loading progress:  60%
[    6.056183] ata1.00: exception Emask 0x0 SAct 0x7fffffff SErr 0x0 action 0x0
[    6.056207] ata1.00: irq_stat 0x40000008
[    6.056218] ata1.00: failed command: READ FPDMA QUEUED
[    6.056232] ata1.00: cmd 60/18:90:28:1c:bb/01:00:19:00:00/40 tag 18 ncq dma 143360 in
[    6.056232]          res 41/40:18:28:1c:bb/00:01:19:00:00/40 Emask 0x409 (media error) <F>
[    6.056267] ata1.00: status: { DRDY ERR }
[    6.056277] ata1.00: error: { UNC }
[    6.056824] ata1.00: configured for UDMA/133
[    6.056842] sd 0:0:0:0: [sda] tag#18 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[    6.056843] sd 0:0:0:0: [sda] tag#18 Sense Key : Medium Error [current] 
[    6.056844] sd 0:0:0:0: [sda] tag#18 Add. Sense: Unrecovered read error - auto reallocate failed
[    6.056845] sd 0:0:0:0: [sda] tag#18 CDB: Read(10) 28 00 19 bb 1c 28 00 01 18 00
[    6.056846] print_req_error: I/O error, dev sda, sector 431692840
[    6.056865] Read-error on swap-device (8:0:431692848)
...
[    6.057381] Read-error on swap-device (8:0:431693120)
[    6.057400] ata1: EH complete
[    6.101616] PM: Read 3252684 kbytes in 3.18 seconds (1022.85 MB/s)
[    6.102283] PM: Error 251 resuming
[    6.102296] PM: Failed to load hibernation image, recovering.
[    6.162908] PM: Basic memory bitmaps freed
[    6.162909] OOM killer enabled.
[    6.162909] Restarting tasks ... done.
[    6.163240] video LNXVIDEO:00: Restoring backlight state
[    6.163320] PM: Hibernation image not present or could not be loaded
Comment 6 Jani Saarinen 2018-03-29 07:10:12 UTC 
First of all. Sorry about spam.
This is mass update for our bugs. 

Sorry if you feel this annoying but with this trying to understand if bug still valid or not.
If bug investigation still in progress, please ignore this and I apologize!

If you think this is not anymore valid, please comment to the bug that can be closed.
If you haven't tested with our latest pre-upstream tree(drm-tip), can you do that also to see if issue is valid there still and if you cannot see issue there, please comment to the bug.
Comment 7 Elizabeth 2018-04-12 15:35:46 UTC 
Created attachment 138802 [details]
s4-log

I tried to replicate this again with 4.16, it worked fine until the platform reached the 463 s4. After that, even when it came back successfully, the display was unresponsive till a hard reboot was applied.  

======================================
             Software
======================================
kernel version              : 4.16.0-041600-generic
hostname                    : gfx-desktop
architecture                : x86_64
os version                  : Ubuntu 17.10
os codename                 : artful
kernel driver               : i915
bios revision               : 5.6
bios release date           : 09/29/2016
hardware acceleration       : disabled
swap partition              : enabled on (/dev/sda2)

======================================
        Graphic drivers
======================================
mesa                        : 18.0.0 (git-dceb1ce807
libdrm                      : 2.4.91
cairo                       : 1.14.10

======================================
             Hardware
======================================
platform                   : Skylake Canyon
motherboard id             : NUC6i7KYB
form factor                : Desktop
cpu family                 : Core i7
cpu family id              : 6
cpu information            : Intel(R) Core(TM) i7-6770HQ CPU @ 2.60GHz
gpu card                   : Intel Corporation Iris Pro Graphics 580 (rev 09) (prog-if 00 [VGA controller])
memory ram                 : 31.31 GB
max memory ram             : 32 GB
display resolution         : 1600x900
cpu thread                 : 8
cpu core                   : 4
cpu model                  : 94
cpu stepping               : 3
socket                     : Other
signature                  : Type 0, Family 6, Model 94, Stepping 3
hard drive                 : 223GiB (240GB)
current cd clock frequency : 337500 kHz
maximum cd clock frequency : 675000 kHz
displays connected         : DP-1 DP-3

======================================
             Firmware
======================================
dmc fw loaded             : yes
dmc version               : 1.27
guc fw loaded             : fetch SUCCESS, load SUCCESS
guc version wanted        : wanted 9.33, found 9.33
guc version found         : wanted 9.33, found 9.33

======================================
             kernel parameters
======================================
splash drm.debug=0xe resume=/dev/sda2 i915.enable_guc=-1
Comment 8 Elizabeth 2018-04-12 15:39:05 UTC 
Also no "GPU Hang" nor "PM: Failed to load" errors where found in the log, it is around 500Mb, if needed I can attach it. Thank you.
Comment 9 Jani Saarinen 2018-04-25 09:33:04 UTC 
Imre, how do you see this suspend test?
Comment 10 Jani Saarinen 2018-04-25 09:33:45 UTC 
Resolving now based on feedback. Good enough?

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.