Created attachment 116391 [details]
Failed to mount /sysroot screenshoot

Created attachment 116392 [details]
related rdsosreport

Last "bootable" is systemd-219-15.fc23 - as part of
- Test compilation:
  http://goo.gl/Gm4ffO
  ISO/Rawhide-Live-Xfce.iso

https://bugzilla.redhat.com/show_bug.cgi?id=1229665

Well, this is the mount tool saying that it doesn't known the "nfs" file system type. That might have a number of different reasons, such as mount.nfs or the kernel nfs modules missing from the initrd or the kernel in general. Either way it's unlikely to be a systemd issue. Please ask the initrd, util-linux or kernel maintainers for help on this.

mount: unknown filesystem type 'nfs'

... means ENODEV from mount(2) syscall and it's obviously wrong, for NFS we use  /sbin/mount.nfs helper. The helper is probably not available (IMHO initrd setup mistake)

Related:

"fstab-generator: don't accept missing root=, but accept root=none"
http://cgit.freedesktop.org/systemd/systemd/commit/?id=093c2cfe3b1ae6081f12927ae7906d90d6623534

https://bugzilla.redhat.com/show_bug.cgi?id=1229665#c15

I have no idea how the fedora cd actually boots, so I am just guessing here...

I would think that the systemd code used to bomb out with a return 0 and a debug message and fedora did some special magic afterwards. Now we are trying to mount and fail on the unexpected syntax. Labels, uuids and whatnot are all supported, but I don't think we handle live:CDLABEL=, but I might be wrong there.

If that is indeed the case then I'd argue that fedora is doing it wrong and should hand us something that we can actually mount...

I look forward to all your comments.

(In reply to Tobias Hunger from comment #9)
> I have no idea how the fedora cd actually boots, so I am just guessing
> here...
> 
> I would think that the systemd code used to bomb out with a return 0 and a
> debug message and fedora did some special magic afterwards. Now we are
> trying to mount and fail on the unexpected syntax. Labels, uuids and whatnot
> are all supported, but I don't think we handle live:CDLABEL=, but I might be
> wrong there.
> 
> If that is indeed the case then I'd argue that fedora is doing it wrong and
> should hand us something that we can actually mount...

... or disable the generator entirely.

"I would think that the systemd code used to bomb out with a return 0 and a debug message and fedora did some special magic afterwards. Now we are trying to mount and fail on the unexpected syntax. Labels, uuids and whatnot are all supported, but I don't think we handle live:CDLABEL=, but I might be wrong there."

Yes, this is basically what I'm saying. The offending commit is http://cgit.freedesktop.org/systemd/systemd/commit/?id=093c2cfe3b1ae6081f12927ae7906d90d6623534 . Before that commit, systemd would not try and mount root at all; it would read the 'root=live:CDLABEL=...' , conclude it didn't understand it, and pass on it. Which was fine. That's what we wanted it to do. There is a dracut module which parses it later and knows how to interpret it properly. After that commit, systemd *does* try and mount root - as an NFS share, which is not at all what it is, so it fails.

I can't comment on what the 'correct' fix is, but it's clearly a systemd behaviour change that caused the problem here.

So why do you even run that generator in your initrd when you do not want to see it used?

Hi,

i've investigated this a bit.

In my case i'm booting live iso by special dracut module which does:
1. mount iso
2. mount squashfs
3. mount aufs combined with tmpfs and squashfs to /union dir (plus fstab entry)
4. mout rbind /union to /sysroot
5. dracut does the pivot

In case of systemd-219 this was working, but with 220 not.

Problem lies with systemd-fstab-generator as in case of 219 there is no /run/systemf/generator/sysroot.mount even if i run /lib/systemd/system-generator/systemd-fstab-generator. In this case system-fstab-generator creates file:

cat /run/systemd/generator/run-initramfs-union.mount

[Unit]
SourcePath=/etc/fstab
Documentation=man:fstab(5) man:systemd-fstab-generator(8)
Before=local-fs.target

[Mount]
What=aufs
Where=/run/initramfs/union
Type=aufs

 
While in 220 there is no /run/systemd/generator/run-initramfs-union.mount but this:

cat /run/systemd/generator/sysroot.mount

[Unit]
SourcePath=/proc/cmdline
Documentation=man:fstab(5) man:systemd-fstab-generator(8)
Before=initrd-root-fs.target

[Mount]
What=live:UUID=2015-06-13-09-16-55-00
Where=/sysroot
Options=ro

So sysroot.mount tries to mount ISO with false output:
mount: unknown filesystem type 'nfs'

Resolved,
- dmsquash-live: add dmsquash-generator
  http://git.kernel.org/cgit/boot/dracut/dracut.git/commit/?id=2c66c3b
- NEWS: version 043
  http://git.kernel.org/cgit/boot/dracut/dracut.git/commit/?id=eab0354

https://www.kernel.org/pub/linux/utils/boot/dracut/dracut-043.tar.xz
https://www.kernel.org/pub/linux/utils/boot/dracut/dracut-043.tar.sign
https://www.kernel.org/pub/linux/utils/boot/dracut/sha256sums.asc


Thank you Dr. Acut.

[liveuser@localhost ~]$ cat /proc/cmdline 
BOOT_IMAGE=vmlinuz0 initrd=initrd0.img root=live:CDLABEL=Rawhide-Live-Base-043 rootfstype=auto ro rd.live.image

$ rpm -q anaconda dracut systemd
anaconda-23.10-1.fc23.x86_64
dracut-043-1.fc23.x86_64
systemd-220-9.fc23.x86_64

$ uname -a
Linux localhost 4.1.0-0.rc8.git0.1.fc23.x86_64 #1 SMP Mon Jun 15 13:14:22 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux