Bug 71802 - Unable to spin-down external USB HDD and switch off LED of USB-flash with udisks2
Summary: Unable to spin-down external USB HDD and switch off LED of USB-flash with udi...
Status: REOPENED
Alias: None
Product: udisks
Classification: Unclassified
Component: general (show other bugs)
Version: unspecified
Hardware: Other All
: medium normal
Assignee: David Zeuthen (not reading bugmail)
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-11-19 18:26 UTC by Norbert X
Modified: 2016-03-28 08:08 UTC (History)
2 users (show)

See Also:
i915 platform:
i915 features:


Attachments
Syslog/Message for my WD USB-HDD (from plugging-in to Safely remove) - Ubuntu 14.04 with UDisks 2.1.3 (4.30 KB, text/plain)
2014-02-04 19:16 UTC, Norbert X
Details
Syslog/Message for my WD USB-HDD (from plugging-in to Safely remove) - Ubuntu 12.04 with UDisks 1.0.4-5ubuntu2.1 (7.66 KB, text/plain)
2014-02-04 20:04 UTC, Norbert X
Details
log-files for my WD-HDD (Ubuntu 14.04 with udisks1 and udisks2) (14.72 KB, application/x-compressed-tar)
2014-02-05 21:19 UTC, Norbert X
Details
strace nautilus with udisks2, WD-HDD safely remove (ok - unmounted, fail - mounted) (322.27 KB, application/x-compressed-tar)
2014-03-08 21:11 UTC, Norbert X
Details

Description Norbert X 2013-11-19 18:26:04 UTC
I have Ubuntu 12.04.3 with GNOME 3.4 and udisks 1.0.4-5ubuntu2.1. With this software I can do a Safely remove of a USB-flash or external USB-HDD from nautilus (3.4.2-0ubuntu8) and from gnome-disk-utility (3.0.2-2ubuntu7).

In modern linux distros (such as Ubuntu 13.04, 13.10, Mageia 4, Fedora 19, OpenSuse 12.3) there is no Safely remove option in Nautilus (see https://bugs.launchpad.net/ubuntu/+source/udisks2/+bug/1067876 and https://bugs.freedesktop.org/show_bug.cgi?id=60293).

In gnome-disks Safely remove / Poweroff is appeared again (from 3.8, see https://bugzilla.gnome.org/show_bug.cgi?id=675542), but it does not spin-down external USB-HDD.

In some linux distros I can spin-down my disk with "udisks --detach /dev/sdX", but it is not user-friendly.

So please, enable spin-down on the udisks side. 

I understand that external SATA (and IDE) HDDs are hot-pluggable and hot-swappable, but it is not comfortable for me to detach rotating drive.
Comment 1 Norbert X 2013-12-24 09:13:35 UTC
Bug exists in Fedora 20 with gnome-disk-utility 3.10.0
UDisks 2.1.1 (built against 2.1.1).
Comment 2 Norbert X 2013-12-24 09:15:41 UTC
There is no Safely remove option in Nautilus 3.10.1 too.
Comment 3 Norbert X 2014-01-18 12:42:59 UTC
Tested again on Arch with 
* gnome-disk-utility 3.10.0 UDisks 2.1.1 (built against 2.1.1)
* Nautilus 3.10.1

- the bug exists with both applications. Please fix it.
Comment 4 David Zeuthen (not reading bugmail) 2014-01-20 20:52:54 UTC
The reason that

 $ udisks --detach /dev/sdX

spins down the disk properly but clicking the "Power off" menu item in the GNOME Disks application doesn't has to do with the fact that the udisks program is from udisks version 1 and was rewritten in udisks version 2.

udisks v1: http://cgit.freedesktop.org/udisks/tree/src/helpers/job-drive-detach.c?id=1.0.4

udisks v2: http://cgit.freedesktop.org/udisks/tree/src/udiskslinuxdrive.c?id=2.1.2#n1195

As you can see, both v1 and v2 does this by writing a '1' to the 'remove' sysfs attribute on the parent USB device, as per

 http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=253e05724f9230910344357b1142ad8642ff9f5a

and this makes most USB-attached disk drives actually power down - at least all the different devices that I've tested with.

However, what's missing in v2 (and present in v1) is the following steps

 1. sending the SCSI SYNCHRONIZE CACHE command
 2. sending START/STOP UNIT command
 3. unbinding the USB Mass Storage kernel driver

Notably, there's actually a TODO item in v2 for doing this:

 /* TODO: Send the eject? Send SCSI START STOP UNIT? */

Now, I don't think that 3. is necessary as it happens as part of writing to the 'remove' sysfs file. That leaves 1. and 2.

Here's what I'd like to you try. Does

 sg_start --stop /dev/sdX

do what you want? If so, we should add 1. and 2. to v2.
Comment 5 Norbert X 2014-01-21 17:32:38 UTC
Dear David!
First of all I would like to thank you for answer and recommendation!

I tested my USB-harddrives with sg_start utility. Thank you for hint, I did not know about SCSI utilities before.

I made such tests under Ubuntu 12.04.4, Fedora 20 and OpenSuSe 13.1 with all installed updates.
* Ubuntu 12.04.4 does not have udisks2, it has udisks 1.0.4-5ubuntu2.1, sg3-utils 1.33-1.
* Fedora 20 has udisks-1.0.4-12.fc20.i686, udisks2-2.1.2-1.fc20.i686, sg3_utils-1.37-2.fc20.i686.
* OpenSuSe 13.1 has udisks-1.0.4-13.1.3.i586, udisks2-2.1.1-2.1.3.i586, sg3_utils-1.36-3.1.2.i586.

For more adequate results I disabled auto-mount feature in GNOME with dconf-editor (as recommended here - https://help.ubuntu.com/community/Mount/USB#Configuring_Automounting).


My test results are the following:
1. For my USB-HDD (Seagate ST9750420A in USB 2.0 Tsunami e-data 2500 enclosure, 
  lsusb - 04fc:0c25 Sunplus Technology Co., Ltd SATALink SPIF225A)
    1.1. command “sg_start --stop /dev/sdX” really spin-downs my drive (sometimes on 2nd or 3rd attempt - I don’t know why), device remains in system and spin-up again only on my demand.
    1.2. command “udisks --detach /dev/sdX” spin-downs my drive (always on 1st attempt), device is completely removed from system after detach.

2. For my USB 3.0 WD My Passport Ultra (WD WD20NMVW,
  lsusb - 1058:0743 Western Digital Technologies, Inc.)
    2.1. command “sg_start --stop /dev/sdX” is ignored by this HDD (it make one click, but does not spin-down). Maybe it is because of proprietary/non-fully-compliant ATA-command set in WD’s controller.
    2.2. command “udisks --detach /dev/sdX” spin-downs my drive (on 1st attempt always), device is completely removed from system after detach.

So the results are identical for all three distros.
For me it seems, that “udisks --detach” works better and in 100% (on all distros and with proprietary WD HDD-controller).
So if you have time, please, do deeper comparison between udisks v.1, udisks v.2 and sg_start.

I'm ready to do more testing if you send me concrete instruction, but I am not expert and I do not know how to debug/trace/log ATA and udisks.
Comment 6 Programmist11180 2014-01-21 17:52:34 UTC
Debian, udisks2 v 2.1.2-1. When I do safely remove of Seagate Portable from Thunar, udisks2 doesn't switch off a disk power when detaching. But udisks1 switch off a disk power.
Comment 7 David Zeuthen (not reading bugmail) 2014-01-25 19:06:26 UTC
Thanks for testing. I'll look into making udisks2 sending SYNCHRONIZE CACHE and START/STOP UNIT commands. Stay tuned.
Comment 8 David Zeuthen (not reading bugmail) 2014-01-26 20:16:32 UTC
OK, I just made that change

 http://cgit.freedesktop.org/udisks/commit/?id=fcdd8f48b6ac9b1b6da82fdf5f59230fc2ea6feb

and tested it with a couple of different units. Notes

 - One of my devices (bus-powered) does not accept the START STOP UNIT
   command so we just continue if it fails. That is, this patch should
   not break existing behavior.

 - Another device (not bus-powered) used to spin down a couple of seconds
   after removing power to the USB port. With this patch it spins down
   immediately. (Which is actually nicer.)

 - Still works on USB sticks etc.

 - I didn't add SCSI SYNCHRONIZE CACHE as that command failed on all my
   devices. I also don't think it's necessary as the device drivers will
   issue something like this in part of the fsync(2) call that we do
   before this.

Please test if the patch works and report back - thanks!
Comment 9 David Zeuthen (not reading bugmail) 2014-01-26 21:52:38 UTC
> command “sg_start --stop /dev/sdX” really spin-downs my drive (sometimes
> on 2nd or 3rd attempt - I don’t know why), device remains in system and
> spin-up again only on my demand.

Btw, this is because the sg_start command opens the device node with O_RDWR which causes an uevent 'change' event to fire when the command completes. This in turn causes udev rules to fire which accesses the disk which causes it to spin up again. In the patch I committed to udisks to do this, we use O_RDONLY to avoid this.
Comment 10 Norbert X 2014-02-02 22:59:15 UTC
Thank you for your commits, David!

I installed all build-dependencies on my Ubuntu 14.04 system with 'apt-get install build-dep', did a 'git clone git://anongit.freedesktop.org/udisks', did './autogen.sh', did 'make', did 'sudo checkinstall make install', verified that I have udisks 2.1.3 installed with 'apt-cache policy', did 'mv /usr/local/share/polkit-1/actions/org.freedesktop.udisks2.policy /usr/share/polkit-1/actions/' and rebooted my machine.

Please note: automount is disabled.
What I get after reboot?


1. for USB-HDDs (both drives have EXT4 and NTFS partitions)
1.1. Seagate - 'Safely remove drive' option in Nautilus spinned-down it, gnome-disks spinned-down it after click on 'Power off the drive'. The 'udisksctl power-off -b /dev/sdX' works too. 
If I enable automount, the 'Safely remove drive'/'Power off the drive' spins-down my drive. It's very good!

1.2. WD - 'Safely remove drive' option in Nautilus spinned-down it, gnome-disks 'Power off the drive' and 'udisksctl power-off -b /dev/sdX' works too.
But there is a little difference - if I enable automount the 'Safely remove drive'/'Power off the drive' does not spin-down the disk (disk plates rotating, but device is removed from system), I reopen the bug because of this.

2. for USB-flashes
There is no 'Safely remove drive' option in Nautilus for my USB-flashes. After 'Eject' the parent device remains in system and may be powered-off by gnome-disks. For me it's a good compromise between udisks1 and udisks2 behavior.


I'm ready to test and collect logs of my WD USB-HDD. I can't understand why Seagate drive unmounts all partitions before power-off, but WD does not.
What logs can help you to understand the problem?
Comment 11 David Zeuthen (not reading bugmail) 2014-02-03 17:34:13 UTC
Hey, thanks for testing the patches.

(In reply to comment #10)
> 1.2. WD - 'Safely remove drive' option in Nautilus spinned-down it,
> gnome-disks 'Power off the drive' and 'udisksctl power-off -b /dev/sdX'
> works too.
> But there is a little difference - if I enable automount

I have an idea of what's wrong here but before I speculate on that, what exactly does "enable automount" mean? Are you referring to having the 'auto' option in the /etc/fstab file?

> the 'Safely remove
> drive'/'Power off the drive' does not spin-down the disk (disk plates
> rotating, but device is removed from system), I reopen the bug because of
> this.
> 
> 2. for USB-flashes
> There is no 'Safely remove drive' option in Nautilus for my USB-flashes.
> After 'Eject' the parent device remains in system and may be powered-off by
> gnome-disks. For me it's a good compromise between udisks1 and udisks2
> behavior.
> 
> 
> I'm ready to test and collect logs of my WD USB-HDD. I can't understand why
> Seagate drive unmounts all partitions before power-off, but WD does not.
> What logs can help you to understand the problem?

When the system is running with the WD USB-HDD, please include the output of 'gvfs-mount -li' from a non-root shell in a terminal in the desktop session.
Comment 12 Norbert X 2014-02-03 18:19:50 UTC
Thank you for reply, David!

>I have an idea of what's wrong here but before I speculate on that, what exactly does "enable automount" mean? Are you referring to having the 'auto' option in the /etc/fstab file?

By "enable automount" I mean these dconf-editor parameters: org.gnome.desktop.media-handling.automount and org.gnome.desktop.media-handling.automount-open (see comment #5). So if I set both parameters to true (in dconf-editor) I get all two partitions of my drive mounted in Nautilus on device connection.

>When the system is running with the WD USB-HDD, please include the output of 'gvfs-mount -li' from a non-root shell in a terminal in the desktop session.

The output of 'gvfs-mount -li' for my WD USB-HDD is as follows:
	Drive(3): WD My Passport 0743
	  Type: GProxyDrive (GProxyVolumeMonitorUDisks2)
	  ids:
	   unix-device: '/dev/sdc'
	  themed icons:  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	  symbolic themed icons:  [drive-harddisk-usb-symbolic]  [drive-harddisk-symbolic]  [drive-symbolic]  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	  is_media_removable=0
	  has_media=1
	  is_media_check_automatic=1
	  can_poll_for_media=0
	  can_eject=0
	  can_start=0
	  can_stop=1
	  start_stop_type=shutdown
	  sort_key=01hotplug/1391451089446989
	  Volume(0): WD_2TB_NTFS
	    Type: GProxyVolume (GProxyVolumeMonitorUDisks2)
	    ids:
	     class: 'device'
	     unix-device: '/dev/sdc2'
	     uuid: '32BA1ADB5E526052'
	     label: 'WD_2TB_NTFS'
	    themed icons:  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	    symbolic themed icons:  [drive-harddisk-usb-symbolic]  [drive-harddisk-symbolic]  [drive-symbolic]  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	    can_mount=1
	    can_eject=0
	    should_automount=1
	    sort_key=gvfs.time_detected_usec.1391451089661368
	  Volume(1): WD_2TB_EXT4
	    Type: GProxyVolume (GProxyVolumeMonitorUDisks2)
	    ids:
	     class: 'device'
	     unix-device: '/dev/sdc1'
	     uuid: '6d296a81-e054-4e09-a092-cee6a0041e0c'
	     label: 'WD_2TB_EXT4'
	    themed icons:  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	    symbolic themed icons:  [drive-harddisk-usb-symbolic]  [drive-harddisk-symbolic]  [drive-symbolic]  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	    can_mount=1
	    can_eject=0
	    should_automount=1
	    sort_key=gvfs.time_detected_usec.1391451089662205
	    Mount(0): WD_2TB_EXT4 -> file:///run/media/flash/WD_2TB_EXT4
	      Type: GProxyMount (GProxyVolumeMonitorUDisks2)
	      default_location=file:///run/media/flash/WD_2TB_EXT4
	      themed icons:  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	      symbolic themed icons:  [drive-harddisk-usb-symbolic]  [drive-harddisk-symbolic]  [drive-symbolic]  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	      can_unmount=1
	      can_eject=0
	      is_shadowed=0
	      sort_key=gvfs.time_detected_usec.1391451089941622


For my Seagate HDD I have:


	Drive(3): ST950042 0AS
	  Type: GProxyDrive (GProxyVolumeMonitorUDisks2)
	  ids:
	   unix-device: '/dev/sdc'
	  themed icons:  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	  symbolic themed icons:  [drive-harddisk-usb-symbolic]  [drive-harddisk-symbolic]  [drive-symbolic]  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	  is_media_removable=0
	  has_media=1
	  is_media_check_automatic=1
	  can_poll_for_media=0
	  can_eject=0
	  can_start=0
	  can_stop=1
	  start_stop_type=shutdown
	  sort_key=01hotplug/1391451261668976
	  Volume(0): NTFS
	    Type: GProxyVolume (GProxyVolumeMonitorUDisks2)
	    ids:
	     class: 'device'
	     unix-device: '/dev/sdc2'
	     uuid: '5F6174ED0C58C4B3'
	     label: 'NTFS'
	    themed icons:  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	    symbolic themed icons:  [drive-harddisk-usb-symbolic]  [drive-harddisk-symbolic]  [drive-symbolic]  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	    can_mount=1
	    can_eject=0
	    should_automount=1
	    sort_key=gvfs.time_detected_usec.1391451261990760
	    Mount(0): NTFS -> file:///run/media/flash/NTFS
	      Type: GProxyMount (GProxyVolumeMonitorUDisks2)
	      default_location=file:///run/media/flash/NTFS
	      themed icons:  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	      symbolic themed icons:  [drive-harddisk-usb-symbolic]  [drive-harddisk-symbolic]  [drive-symbolic]  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	      can_unmount=1
	      can_eject=0
	      is_shadowed=0
	      sort_key=gvfs.time_detected_usec.1391451263178059
	  Volume(1): GENTOO
	    Type: GProxyVolume (GProxyVolumeMonitorUDisks2)
	    ids:
	     class: 'device'
	     unix-device: '/dev/sdc1'
	     uuid: '52e2b7ea-21e0-4973-aeae-f5b7edfadf40'
	     label: 'GENTOO'
	    themed icons:  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	    symbolic themed icons:  [drive-harddisk-usb-symbolic]  [drive-harddisk-symbolic]  [drive-symbolic]  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	    can_mount=1
	    can_eject=0
	    should_automount=1
	    sort_key=gvfs.time_detected_usec.1391451261992313
	    Mount(0): GENTOO -> file:///run/media/flash/GENTOO
	      Type: GProxyMount (GProxyVolumeMonitorUDisks2)
	      default_location=file:///run/media/flash/GENTOO
	      themed icons:  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	      symbolic themed icons:  [drive-harddisk-usb-symbolic]  [drive-harddisk-symbolic]  [drive-symbolic]  [drive-harddisk-usb]  [drive-harddisk]  [drive]
	      can_unmount=1
	      can_eject=0
	      is_shadowed=0
	      sort_key=gvfs.time_detected_usec.1391451262121538
	

I hope this help you to understand the problem.
Comment 13 David Zeuthen (not reading bugmail) 2014-02-03 18:43:35 UTC
(Hmm, the drives have

	  can_stop=1
	  start_stop_type=shutdown

so everything should be good. My guess is that it's the NTFS partition that is the culprit. Specifically, unmounting it via udisks somehow fails.)

If you manually unmount the partitions, does powering down the drive work? Specifically, try this

 1. Unmount all partitions using Disks
    - if that fails, try 'umount /dev/sdXN' from a terminal. Does that work?
 2. Press the "Power off" button in Disks when everything has been unmounted.

Does that work?

Also, please include all messages from udisks from syslog (e.g. /var/log/messages or similar) when pressing the "Power off" button in Disks. It should contain some useful info.
Comment 14 Norbert X 2014-02-04 19:11:32 UTC
Today I tested 5 other drives with different paritioning schemes (1 primary + 1 extended partition with some logical partitions in it):
3 of them were successfully Safely removed (in Nautilus) and Powered off (in Disks), 
2 of them were successfully Safely removed (in Nautilus) and Powered off (in Disks) only after manual unmount.

>If you manually unmount the partitions, does powering down the drive work?
>Specifically, try this
>
> 1. Unmount all partitions using Disks
>    - if that fails, try 'umount /dev/sdXN' from a terminal. Does that work?
did it in Disks with my WD drive

> 2. Press the "Power off" button in Disks when everything has been unmounted.
did it in Disks - my WD drive is spinned down.

>Does that work?
So it works.
Comment 15 Norbert X 2014-02-04 19:16:33 UTC
Created attachment 93407 [details]
Syslog/Message for my WD USB-HDD (from plugging-in to Safely remove) - Ubuntu 14.04 with UDisks 2.1.3

>Also, please include all messages from udisks from syslog (e.g. /var/log/messages or similar) when pressing the "Power off" button in Disks. It should contain some useful info.

I attached full log-file for my WD USB-HDD. It contains device plugging-in, detection by automount feature, un-mounting both partiotion in Disks, Powering off in Disks, device plugging-out  as you recommended.
Comment 16 Norbert X 2014-02-04 20:04:02 UTC
Created attachment 93414 [details]
Syslog/Message for my WD USB-HDD (from plugging-in to Safely remove) - Ubuntu 12.04 with UDisks 1.0.4-5ubuntu2.1

If that can help I prepared syslog/messages for my WD-HDD in Ubuntu 12.04 too. Sequence is identical to comment 15.
Comment 17 David Zeuthen (not reading bugmail) 2014-02-04 20:24:50 UTC
Let me see if I understand this correctly:

 1. Manually unmount, then power-off via the Disks GUI works as expected.
 2. The "safely remove drive" button in Nautilus does not work as expected.

Is that about right?
Comment 18 Norbert X 2014-02-04 20:35:43 UTC
It seems that my english is not perfect. I'm sorry.

Let's consider only WD-HDD.

>Let me see if I understand this correctly:
> 1. Manually unmount, then power-off via the Disks GUI works as expected.
Yes.
> 2. The "safely remove drive" button in Nautilus does not work as expected.
No. It works as expected on other drives, but not WD.
>Is that about right?
Yes and no.

Safely remove (in Nautilus) and Power off (in Disks) works as expected only if I manually unmount all WD partitions before Safe removal / Power off. 

If I unmount all WD partitions manually and then do Safe removal / Power off HDD spins down.
Comment 19 David Zeuthen (not reading bugmail) 2014-02-04 20:54:20 UTC
OK, I'm interested in output of /var/log/messages when it doesn't work with the WD. Or is that what you posted in comment 15?
Comment 20 Norbert X 2014-02-04 21:31:32 UTC
>OK, I'm interested in output of /var/log/messages when it doesn't work with the WD. Or is that what you posted in comment 15?
I tested it again - the log in comment 15 represent both situations - when drive spinned-down and when it does not. Only timestamps differ, messages order is the same.


With udisks1 this WD-HDD spins-down (by Safely remove in Nautilus or Power off in Disks) even if all partitions mounted, with udisks2 it does not. 
There is a little difference between udisks1 and udisks2 "powering off drive" algorithms. What other log-file can I attach to help you to understand the problem?
Comment 21 David Zeuthen (not reading bugmail) 2014-02-05 19:46:34 UTC
(In reply to comment #20)
> >OK, I'm interested in output of /var/log/messages when it doesn't work with the WD. Or is that what you posted in comment 15?
> I tested it again - the log in comment 15 represent both situations - when
> drive spinned-down and when it does not. Only timestamps differ, messages
> order is the same.
> 
> 
> With udisks1 this WD-HDD spins-down (by Safely remove in Nautilus or Power
> off in Disks) even if all partitions mounted, with udisks2 it does not. 
> There is a little difference between udisks1 and udisks2 "powering off
> drive" algorithms. What other log-file can I attach to help you to
> understand the problem?

Your testing with udisks1 is on an older version of Ubuntu (12.04 vs. 14.04) isn't it? If so, any chance you can install the udisks1 package on the same OS as you tested with udisks2? The packages are parallel-installable (and called 'udisks' and 'udisks2') so it should be as simple as 'apt-get install udisks' and then run "udisks --detach /dev/sdX".

I'm asking for this because I think this is due to a kernel and/or ntfs-3g problem.

Another thing to try would be to see if it happens if the filesystem type is *not* ntfs, e.g. try with ext4. (I realize this may not be possible as you may not want to reformat the disk.)

To recap, the only difference now from udisks1 is that udisks2 does not send SYNCHRONIZE CACHE before START STOP UNIT (it didn't work on any of my devices when I make the recent udisks2 changes). If it turns out that udisks1 works as expected on 14.04, I will try to add this change to see if it makes the difference...
Comment 22 Norbert X 2014-02-05 21:19:22 UTC
Created attachment 93492 [details]
log-files for my WD-HDD (Ubuntu 14.04 with udisks1 and udisks2)

Thank you for reply, David.

>Your testing with udisks1 is on an older version of Ubuntu (12.04 vs. 14.04) isn't it? If so, any chance you can install the udisks1 package on the same OS as you tested with udisks2? The packages are parallel-installable (and called 'udisks' and 'udisks2') so it should be as simple as 'apt-get install udisks' and then run "udisks --detach /dev/sdX".
I have already installed udisks1 (1.0.4-8ubuntu1) on Ubuntu 14.04 and it works as expected on all my drives if I unmount all partitions manually (from console, Nautilus or Disks - it does not matter).

>I'm asking for this because I think this is due to a kernel and/or ntfs-3g problem.
For me it seems that we have timing/race issue in udisks2.

>Another thing to try would be to see if it happens if the filesystem type is *not* ntfs, e.g. try with ext4. (I realize this may not be possible as you may not want to reformat the disk.)
I have 500Gb of data on my NTFS partition, so I do not want to reformat my HDD. I'm sorry for this.

>To recap, the only difference now from udisks1 is that udisks2 does not send SYNCHRONIZE CACHE before START STOP UNIT (it didn't work on any of my devices when I make the recent udisks2 changes). If it turns out that udisks1 works as expected on 14.04, I will try to add this change to see if it makes the difference...
If there are no other differences - please add SYNCHRONIZE CACHE, I'm ready to do a test and report back.


I prepared an archive of log-files for my WD-HDD on Ubuntu 14.04 with udisks1 and udisks2 installed.
The log-files are: "tailf /var/log/syslog", "gvfs-mount -o", "udisksctl monitor", "udisks --monitor-detail".
The test-cases are:
1. udisks2 Safely remove from Disks (mounted) = FAIL - not spinned down
    connect, auto-mount by Nautilus, clicked Power off in Disks, disconnect
2. udisks2 Safely remove from Disks (unmounted) = SUCCESS - spinned down
    connect auto-mount by Nautilus, unmount in Nautilus, clicked Safely remove drive in Nautilus, disconnect
3. udisks2 Safely remove from Nautilus (mounted) FAIL - not spinned down
    connect, auto-mount by Nautilus, clicked Safely remove drive in Nautilus, disconnect
4. udisks2 Safely remove from Nautilus (unmounted) =  SUCCESS - spinned down
    connect, auto-mount by Nautilus, unmount in Nautilus, clicked Safely remove drive in Nautilus, disconnect
5. udisks --detach Safely remove (unmounted) = SUCCESS - spinned down
    connect, auto-mount by Nautilus, unmount in Nautilus, sent "udisks --detach /dev/sdX" in console, disconnect

On FAIL-cases there are interesting lines in "udisks --monitor-detail"
    (udisks:2365): udisks-WARNING **: Couldn't call GetAll() to get properties for /org/freedesktop/UDisks/devices/sdc2: Method "GetAll" with signature "s" on interface "org.freedesktop.DBus.Properties" doesn't exist
may be it cause problems. I do not know.
  
I'm ready to test your new commits.
Comment 23 David Zeuthen (not reading bugmail) 2014-02-05 22:28:12 UTC
(In reply to comment #22)
> I have 500Gb of data on my NTFS partition, so I do not want to reformat my
> HDD. I'm sorry for this.

Fair enough. I'll try to see if I can repro on my hardware.

> If there are no other differences - please add SYNCHRONIZE CACHE, I'm ready
> to do a test and report back.

Will do - thanks for testing!

> I prepared an archive of log-files for my WD-HDD on Ubuntu 14.04 with
> udisks1 and udisks2 installed.

Thanks for doing that - it's very helpful!

> On FAIL-cases there are interesting lines in "udisks --monitor-detail"
>     (udisks:2365): udisks-WARNING **: Couldn't call GetAll() to get
> properties for /org/freedesktop/UDisks/devices/sdc2: Method "GetAll" with
> signature "s" on interface "org.freedesktop.DBus.Properties" doesn't exist
> may be it cause problems. I do not know.

This is likely not a problem.
Comment 24 Programmist11180 2014-02-06 13:02:47 UTC
When I try execute (udisks2 installed):

$ udisksctl power-off -b /dev/sde

It says: "Unknown command `power-off'"

What do I do not so?
Comment 25 Norbert X 2014-02-06 13:17:44 UTC
Hello, Programmist11180!
From comment #8 to #23 we talk about unrelease udisks2 version from git-repository.

If you are on Ubuntu (or Mint, or other Debian/Ubuntu based distro) you can follow my instruction (in comment #10) to compile udisks2 from latest sources and test "udisksctl power-off -b /dev/sde" again. 

I have compiled deb-package for 14.04 i686 - I can post it here if you have any problems with compilation.
Comment 26 Programmist11180 2014-02-08 15:41:03 UTC
Hello nrbrtx.
I have compiled and installed udisks 2.1.3. And I have a problem. udisksctl not work.

$ udisksctl power-off -b /dev/sde

(process:3807): GLib-GObject-WARNING **: invalid (NULL) pointer instance

(process:3807): GLib-GObject-CRITICAL **: g_signal_handlers_disconnect_matched: assertion 'G_TYPE_CHECK_INSTANCE (instance)' failed

(process:3807): GLib-GObject-WARNING **: invalid (NULL) pointer instance

(process:3807): GLib-GObject-CRITICAL **: g_signal_handlers_disconnect_matched: assertion 'G_TYPE_CHECK_INSTANCE (instance)' failed

(process:3807): GLib-GObject-WARNING **: invalid (NULL) pointer instance

(process:3807): GLib-GObject-CRITICAL **: g_signal_handlers_disconnect_matched: assertion 'G_TYPE_CHECK_INSTANCE (instance)' failed

(process:3807): GLib-GObject-WARNING **: invalid (NULL) pointer instance

(process:3807): GLib-GObject-CRITICAL **: g_signal_handlers_disconnect_matched: assertion 'G_TYPE_CHECK_INSTANCE (instance)' failed

(process:3807): GLib-GObject-WARNING **: invalid (NULL) pointer instance

(process:3807): GLib-GObject-CRITICAL **: g_signal_handlers_disconnect_matched: assertion 'G_TYPE_CHECK_INSTANCE (instance)' failed

(process:3807): GLib-GObject-CRITICAL **: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
Error connecting to the udisks daemon: Ошибка вызова StartServiceByName для org.freedesktop.UDisks2: Время ожидания истекло


/var/log/syslog

Feb  8 18:54:37 debian-terminal dbus[2839]: [system] Activating service name='org.freedesktop.UDisks2' (using servicehelper)
Feb  8 18:54:37 debian-terminal udisksd[3540]: udisks daemon version 2.1.3 starting
Feb  8 18:55:02 debian-terminal dbus[2839]: [system] Failed to activate service 'org.freedesktop.UDisks2': timed out
Feb  8 18:55:07 debian-terminal dbus[2839]: [system] Activating service name='org.freedesktop.UDisks2' (using servicehelper)
Feb  8 18:55:07 debian-terminal udisksd[3810]: udisks daemon version 2.1.3 starting
Feb  8 18:55:32 debian-terminal dbus[2839]: [system] Failed to activate service 'org.freedesktop.UDisks2': timed out

$ ps aux | grep udisks
nikts     3531  0.0  0.0 205256  3312 ?        Sl   18:54   0:00 /usr/lib/gvfs/gvfs-udisks2-volume-m
root      3540  0.0  0.1 366324  4912 ?        Sl   18:54   0:01 /usr/local/lib/udisks2/udisksd --no
root      3810  0.0  0.1 366324  4916 ?        Sl   18:55   0:01 /usr/local/lib/udisks2/udisksd --no
Comment 27 Norbert X 2014-02-10 19:33:04 UTC
Hello, Programmist11180!

Did you completely followed my instruction in comment #10? Have you rebooted PC after udisks 2.1.3 installation?
Do you have Ubuntu 14.04 as me?

Does power-off work from gnome-disks or Nautilus?
Comment 28 Programmist11180 2014-02-10 20:28:44 UTC
nrbrtx, I have Debian Wheezy (with some Sid), and Thunar filemanager.
The paths in built udisks and udisks from repository differs.
This solve the problem
sudo cp /usr/local/etc/dbus-1/system.d/org.freedesktop.UDisks2.conf /etc/dbus-1/system.d/

But there is an another problem:
$ udisksctl power-off -b /dev/sde
Error looking up object for device /dev/sde

The 'eject' option not available in Thunar with built udisks. Therefore I can't check power-off.
Comment 29 Norbert X 2014-02-10 20:36:43 UTC
Programmist11180, you can always detect which devices are connected with
"fdisk -l" (called by root) or watch syslog on device connection. It seems that /dev/sde device does not exist. Also you can check which storage devices are connected in gnome-disks (previously palimsest).

Usually /dev/sda is your hard disk, the removable devices start from /dev/sdb.
Comment 30 Programmist11180 2014-02-11 05:23:30 UTC
sudo cp /usr/local/lib/libudisks2.so.0.0.0 /usr/lib/x86_64-linux-gnu/libudisks2.so.0.0.0

And after reboot 'eject' became available in Thunar.

This is my small test:
1. Seagate Portable. Filesystem - one big NTFS volume. Eject from Thunar - no power-off. Disable volume and then eject from Thunar - disk power-offs. Umount volume and udisksctl power-off -b /dev/sde also spin-down disk.

2. U3 flash. Filesystem - one big FAT32. Eject from Thunar - flash power-offs.

3. WD Passport. Filesystem - one big NTFS. No power-off in all cases. Also no power-off in old udisks. I think that this disk may not support this feature.
Comment 31 Norbert X 2014-03-08 21:11:33 UTC
Created attachment 95371 [details]
strace nautilus with udisks2, WD-HDD safely remove (ok - unmounted, fail - mounted)

Dear, David!
I look udisks2 git-repo, it seems that there are no changes since last comments.

I prepared two strace logs for nautilus with my WD-HDD connected:
* nautilus_ok.txt represent case no 4 of my comment #22
* nautilus_fail.txt represent case no 3 of my comment #22

I hope that this log-files will help determine the root of the safely-remove problems. Also I still hope that we can get udisks1 functionality in new udisks2.
Comment 32 David Zeuthen (not reading bugmail) 2014-03-10 21:53:31 UTC
Hi, sorry for the lack of updates. Nothing special, just been busy with work, I'm still planning on adding the SYNCHRONIZE CACHE command. Hopefully I'll get to look at it soon. Thanks.
Comment 33 David Zeuthen (not reading bugmail) 2014-03-26 06:01:02 UTC
Hi, finally got around to making udisks also send the SYNCHRONIZE CACHE command. Please check if it works. The patch is on master and here:

 http://cgit.freedesktop.org/udisks/commit/?id=429892f2ec39d66732bee0f78d093eb6d2c5433f

Thanks!
Comment 34 Programmist11180 2014-03-28 15:04:53 UTC
I don't see a difference after applying this patch.
Comment 35 Norbert X 2014-03-29 18:53:08 UTC
Thank you for your commit, David!

But as Programmist11180 I don't see any differences after applying this patch. My WD HDD acts as before.

How I can get full debug log about low-level actions which are performed during Safely remove from Nautilus or Disks?
Comment 36 Phillip Susi 2014-05-20 13:45:21 UTC
FYI, when you unbind the scsi disk driver by writing the 1 to the delete knob in sysfs, it takes care of issuing the synchronize cache and start/stop unit commands.  You can see this in dmesg.  You mentioned the usb mass storage driver, which I would not have thought of.  I'd bet that is the issue.  My guess is that most drives appear to work because they shut off their LED when given the STOP UNIT command when the scsi disk driver unbinds, but if the usb mass storage driver is still bound, some drives decide to keep the LED on.
Comment 37 jc 2016-03-28 08:08:15 UTC
Same issue here, reported there:
https://bugzilla.redhat.com/show_bug.cgi?id=1278944

It is not consistent between disks, even when there are from the same vendor or the same model.
Of course, the controller may be totally different.

Note that it all cases, they spin down correctly under Mac OS or Windows.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.