Bug 103652 - fontconfig requires nano second precison for fontcache
Summary: fontconfig requires nano second precison for fontcache
Status: RESOLVED MOVED
Alias: None
Product: fontconfig
Classification: Unclassified
Component: fc-cache (show other bugs)
Version: 2.11
Hardware: Other All
: medium normal
Assignee: fontconfig-bugs
QA Contact: Behdad Esfahbod
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-11-09 15:28 UTC by Laurent Bigonville
Modified: 2018-08-20 21:46 UTC (History)
3 users (show)

See Also:
i915 platform:
i915 features:


Attachments
output of FC_DEBUG=16 fc-cache (33.10 KB, text/plain)
2018-04-04 10:40 UTC, Jean-Baptiste Lallement
Details
cache: If nsec is zero, don't use it for comparisons (1.42 KB, patch)
2018-04-04 13:15 UTC, Iain Lane
Details | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Laurent Bigonville 2017-11-09 15:28:49 UTC
Hi,

The following bug has been reported in debian:

===

Dear Maintainer,
fontconfig now requires nano second mtime precision on files in order to
validate their freshness. squashfs does not seem to support nano second
mtime precision causing any Desktop ISO's made with live-build to regenrate
the font cache at the desktop startup leading to a very slow time to desktop.

===

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=880574
Comment 1 Akira TAGOH 2017-11-10 03:37:29 UTC
Hmm, what does "not seem to support" really mean here? are you generating a cache on non-squashfs and using it on squashfs then?
Comment 2 Laurent Bigonville 2017-11-10 08:40:51 UTC
I believe that what bug reporter means is that the cache is added to the squashfs during the build of the livecd and at boot when FC tries to check the freshness of the cache it decides that it needs to be updated and then write a fresh one to the unionfs (I suppose)
Comment 3 Akira TAGOH 2017-11-10 09:39:56 UTC
Can you give me a log of FC_DEBUG=16 fc-cache prior to boot the desktop?
Comment 4 Jean-Baptiste Lallement 2018-04-04 10:40:22 UTC
Created attachment 138574 [details]
output of FC_DEBUG=16 fc-cache

I attached the output of FC_DEBUG=16 fc-cache before booting to an Ubuntu Desktop Live Session. This is a follow-up of the issue reported in LP (https://bugs.launchpad.net/ubuntu/+source/bluez/+bug/1754836) where, according to latest comments), the font cache is regenerated when the user session starts. 
Let me know if you need more information.
Comment 5 Iain Lane 2018-04-04 13:15:41 UTC
Created attachment 138578 [details] [review]
cache: If nsec is zero, don't use it for comparisons

I made a (stupid?) patch. Is this a reliable way to go about falling back from nsec to non-nsec? I couldn't find a way to actually ask the question of the system directly.

We're particularly interested because this is a big cause of live image boot slowness on Ubuntu. This patch works as intended; the cache isn't regenerated on live system boot any more.
Comment 6 Alistair Buxton 2018-04-04 21:33:55 UTC
The problem as I understand it:

1. You have /usr/share/fonts/* on ext4 or similar FS with nanos.
2. /usr/share/fonts has a timestamp like 1522818676.601423551.
3. You run fontconfig and it stores the timestamp in the cache file.
4. You build a squashfs from the FS. Squashfs does not support nanos.
5. The timestamp of /usr/share/fonts inside the squashfs becomes 1522818676.0.
6. Installer boots and fontconfig sees the mismatched time stamps so rebuilds the cache in the live user's home directory - which is wiped at reboot.
7. During install the squashfs is copied back to an ext4 FS.
8. User reboots and logs in.
9. /usr/share/fonts is now on a filesystem which supports nanos, but the original time stamp nanos has been lost.
10. Fontconfig decides the cache is out of date and rebuilds it again, in the new user's home directory. This happens each time a newly created user logs in, until such time as a postinst script rebuilds the system font cache.

Note steps 9 and 10 only happen on Xubuntu and maybe some other flavours, but not Ubuntu.

This means that even if you could ask the system whether the filesystem supports nanos, that wouldn't be enough. It is possible for the timestamp information to be lost/mutated through squashfs, and then copied back to a filesystem with nanos, and then there is no way to determine whether that has happened or not.
Comment 7 Jean-Baptiste Lallement 2018-04-05 10:16:04 UTC
Your analysis is correct. 

point 9 doesn't happen on Ubuntu (and maybe other flavours) because at the end of te installation process, some packages are removed from the target filesystem that triggers the fontconfig trigger (which runs the postinst script that regenerate the cache) So on next boot the timestamps won't have nanoseconfs but will match.
Comment 8 Iain Lane 2018-04-05 12:17:04 UTC
Aren't this:

(In reply to Alistair Buxton from comment #6)
> 5. The timestamp of /usr/share/fonts inside the squashfs becomes
> 1522818676.0.
> 6. Installer boots and fontconfig sees the mismatched time stamps so
> rebuilds the cache in the live user's home directory - which is wiped at
> reboot.
> 7. During install the squashfs is copied back to an ext4 FS.

and this:

> 8. User reboots and logs in.
> 9. /usr/share/fonts is now on a filesystem which supports nanos, but the
> original time stamp nanos has been lost.
> 10. Fontconfig decides the cache is out of date and rebuilds it again, in
> the new user's home directory. This happens each time a newly created user
> logs in, until such time as a postinst script rebuilds the system font cache.

actually the same thing happening twice? The nanosecond portion of the timestamp is lost when the squashfs is built, and it's still not there when it's copied to the target system, but the fontconfig cache has nanos in it.

In that case, my proposed fix fixes this. I tried with a modified Xubuntu ISO, and after installation:

laney@bionic:~$ ls -l .cache
total 24
-rw-rw-r-- 1 laney laney    5 Apr  5 12:52 blueman-applet-1000
drwxr-xr-x 2 laney laney 4096 Apr  5 12:52 gstreamer-1.0
-rw-r--r-- 1 laney laney    0 Apr  5 13:05 motd.legal-displayed
drwx------ 2 laney laney 4096 Apr  5 12:53 obexd
drwx------ 2 laney laney 4096 Apr  5 12:52 sessions
drwxrwxr-x 2 laney laney 4096 Apr  5 13:01 update-manager-core
-rw-rw-r-- 1 laney laney  381 Apr  5 12:52 xfce4-indicator-plugin.log

(laney@bionic:~$ stat /usr/share/fonts
...
Modify: 2018-04-04 06:44:44.000000000 +0100
laney@bionic:~$ FC_DEBUG=16 fc-cache
FC_DEBUG=16
FcCacheTimeValid dir "/usr/share/fonts" cache checksum 1522820684.549609025 dir checksum 1522820684.0
tv_nsec == 0, ignoring nanoseconds)
Comment 9 Alistair Buxton 2018-04-05 13:16:13 UTC
Yes, your current patch handles both cases.

My point was that if you "improve" the patch to check whether the filesystem supports nanos it would no longer handle the second case.
Comment 10 GitLab Migration User 2018-08-20 21:46:11 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/fontconfig/fontconfig/issues/44.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.