Bug 73291

Summary: poppler does not show fl ligature
Product: fontconfig Reporter: Christopher Yeleighton <giecrilj>
Component: fc-matchAssignee: fontconfig-bugs
Status: RESOLVED FIXED QA Contact: Behdad Esfahbod <freedesktop>
Severity: major    
Priority: medium CC: akira, fabian+debian, jan.steffens, js102, preining, psychonaut
Version: unspecified   
Hardware: x86-64 (AMD64)   
OS: Linux (All)   
URL: http://www.cs.dartmouth.edu/~doug/mdmspe.pdf
See Also: https://bugs.freedesktop.org/show_bug.cgi?id=80093
Whiteboard:
i915 platform: i915 features:
Attachments: a screenshot of okular -> file -> properties -> fonts

Description Christopher Yeleighton 2014-01-04 23:18:52 UTC
<URL: http://www.cs.dartmouth.edu/~doug/mdmspe.pdf >

In Okular:
 by constructing input on the   y

In GhostScript:
  by constructing input on the fly

Note: the result of copying the text out of Okular is correct.
Comment 1 Albert Astals Cid 2014-01-05 00:31:56 UTC
Works here.

Which poppler do you have?

Which freetype do you have?

Can you attach a screenshot of okular -> file -> properties -> fonts?
Comment 2 Christopher Yeleighton 2014-01-05 01:46:48 UTC
(In reply to comment #1)
> Works here.
> 
> Which poppler do you have?

libpoppler43-0.24.3

> 
> Which freetype do you have?

libfreetype6-2.5.0.1
Comment 3 Christopher Yeleighton 2014-01-05 01:49:28 UTC
Created attachment 91503 [details]
a screenshot of okular -> file -> properties -> fonts

Note that the selected substitution font face Tex Gyre Termes does have the ligatures in place (although I do not know which font is used to typeset the fragment with blanked ligatures).
Comment 4 Christopher Yeleighton 2014-01-05 01:58:13 UTC
But since Ghostscript loads NimbusRomNo9L-* early, I would say the document is typeset in Times except for embedded formulae in Computer Modern and API names in Courier.
Comment 5 Adrian Johnson 2014-01-05 02:10:17 UTC
What is the output of

pdffonts -subst mdmspe.pdf
Comment 6 Christopher Yeleighton 2014-01-05 02:15:55 UTC
name                                 object ID substitute font                      substitute font file
------------------------------------ --------- ------------------------------------ ------------------------------------
Times-Italic                              7  0 TeXGyreTermes-Italic                 /usr/share/fonts/texlive-tex-gyre/texgyretermes-italic.otf
Times-Roman                               8  0 TeXGyreTermes-Regular                /usr/share/fonts/texlive-tex-gyre/texgyretermes-regular.otf
Courier                                   9  0 Courier                              /usr/share/fonts/Type1/cour.pfa
Times-Bold                               10  0 TeXGyreTermes-Bold                   /usr/share/fonts/texlive-tex-gyre/texgyretermes-bold.otf
Times-Roman                              21  0 TeXGyreTermes-Regular                /usr/share/fonts/texlive-tex-gyre/texgyretermes-regular.otf
Comment 7 Adrian Johnson 2014-01-05 03:26:58 UTC
The font in the PDF uses the name "fi" for the ligature. FT_Get_Char_Index() is used to find the glyph in the substitute font. The problem is texgyretermes-bold.ttx uses the name "f_f" for the fi ligature.
Comment 8 Adrian Johnson 2014-01-05 03:28:21 UTC
(In reply to comment #7)
> The font in the PDF uses the name "fi" for the ligature. FT_Get_Char_Index()
> is used to find the glyph in the substitute font. The problem is
> texgyretermes-bold.ttx uses the name "f_f" for the fi ligature.

Oops. I meant FT_Get_Name_Index() is used to lookup the glyph.
Comment 9 Christopher Yeleighton 2014-01-05 10:33:10 UTC
(In reply to comment #7)
> The font in the PDF uses the name "fi" for the ligature. FT_Get_Char_Index()
> is used to find the glyph in the substitute font. The problem is
> texgyretermes-bold.ttx uses the name "f_f" for the fi ligature.

I do not know what texgyretermes-bold.ttx is.  The font texgyretermes-bold.otf uses the name "f_i" for the ligature "fi".
Comment 10 Adrian Johnson 2014-01-05 10:37:17 UTC
That was a typo. The ttx file is the font converted to xml format using the ttx utility.
Comment 11 Christopher Yeleighton 2014-01-05 12:56:39 UTC
The font Termes is incompatible with Times and should not be used as a replacement.
Comment 12 Florian Kulzer 2014-01-26 15:32:53 UTC
I have the same problem on Debian/Sid (unstable), which uses
libpoppler37. If I open mdmspe.pdf with Okular, Evince or GIMP then the
"fl" and "fi" ligatures are missing.

If I downgrade Okular to the version in Debian testing, which still uses
libpoppler19, then these ligatures are displayed correctly.

Both versions of Okular use the same font substitution scheme, identical
to the screenshot that Christopher attached to Comment 3. The ligatures
in texgyretermes-{roman,bold}.otf are displayed with libpoppler19 and
are missing with libpoppler37.

I hope this information helps to track down the source of the problem.
Comment 13 John M 2014-03-05 00:38:58 UTC
libpoppler19 is no longer in Debian/Jessie (testing) or Debian/Sid (unstable).  With libpoppler37, the fi and fl ligatures are not displayed in evince.

The fi ligatures in "identifier", "suffix", etc., are not displayed in (Appendix A of) the following file:

www.open-std.org/jtc1/sc22/wg14/www/docs/n1494.pdf

Both gv and acroread display the ligatures properly; imagemagick's display has a massive memory leak trying to display the file (i.e., around 5GiB of RAM and 100% CPU).

Debian's reportbug recommends to report the bug directly to bugs.freedesktop.org.

$ lsb_release -rd
Description:	Debian GNU/Linux testing (jessie)
Release:	testing

$ apt-cache policy libpoppler37 gv acroread:i386
libpoppler37:
  Installed: 0.22.5-4
  Candidate: 0.22.5-4
  Version table:
 *** 0.22.5-4 0
        500 http://ftp.us.debian.org/debian/ testing/main amd64 Packages
        200 http://ftp.us.debian.org/debian/ unstable/main amd64 Packages
        100 /var/lib/dpkg/status
gv:
  Installed: 1:3.7.4-1
  Candidate: 1:3.7.4-1
  Version table:
 *** 1:3.7.4-1 0
        500 http://ftp.us.debian.org/debian/ testing/main amd64 Packages
        200 http://ftp.us.debian.org/debian/ unstable/main amd64 Packages
        100 /var/lib/dpkg/status
acroread:i386:
  Installed: 9.5.5-dmo1
  Candidate: 9.5.5-dmo1
  Version table:
 *** 9.5.5-dmo1 0
        100 http://www.deb-multimedia.org/ testing/non-free i386 Packages
        100 /var/lib/dpkg/status
Comment 14 Akira TAGOH 2014-03-05 02:51:21 UTC
Well, I'm not sure why this issue was reassigned to fontconfig honestly. fixing in fontconfig is the wrong direction IMHO for this kind of rendering bug. that is just a workaround and there should be the real solution in fonts and/or renderer.
Comment 15 Christopher Yeleighton 2014-03-08 11:17:39 UTC
(In reply to comment #14)
> Well, I'm not sure why this issue was reassigned to fontconfig honestly.
> fixing in fontconfig is the wrong direction IMHO for this kind of rendering
> bug. that is just a workaround and there should be the real solution in
> fonts and/or renderer.

Comment #11.
Comment 16 Akira TAGOH 2014-03-13 07:15:52 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > Well, I'm not sure why this issue was reassigned to fontconfig honestly.
> > fixing in fontconfig is the wrong direction IMHO for this kind of rendering
> > bug. that is just a workaround and there should be the real solution in
> > fonts and/or renderer.
> 
> Comment #11.

That sounds like opposed to what they say. then that should be a font bug. we could comment it out until they fix though.
Comment 17 Akira TAGOH 2014-03-18 02:55:55 UTC
commented out the lines related to TeX Gyre Termes font. please be aware this is a workaround and contact the font author to get it fixed.
Comment 18 Yaron S. 2014-04-08 20:14:16 UTC
I have the same issue, and I'm running fedora 20:

poppler-0.24.3-3.fc20.x86_64
okular-4.12.3-1.fc20.x86_64
fontconfig-2.11.0-1.fc20.x86_64

a pdf that references Times and pickes up Tex Gyre Termes
instead shows missing glyphs when rendering.

The fonts on my system are picked up (using the pdffonts command above) 
from a texlive 2013 install I have in place (not from the fedora repo).
Strangely, removing the otf version of Termes made okular pick
up the type1 version (It just happened to be there), and this
renders find - no missing letters on rendering.

Here's the output of pdffonts after removing the otf files,
with the file rendering properly.

```
name                                 object ID substitute font                      substitute font file
------------------------------------ --------- ------------------------------------ ------------------------------------
Symbol                                    9  0 Symbol                               /usr/share/fonts/wine-symbol-fonts/symbol.ttf
Times-Roman                               8  0 TeXGyreTermes                        /mnt/debian/texlive/2013/texmf-dist/fonts/type1/public/tex-gyre/qtmr.pfb
Times-Italic                              7  0 TeXGyreTermes Italic                 /mnt/debian/texlive/2013/texmf-dist/fonts/type1/public/tex-gyre/qtmri.pfb
Times-Bold                                6  0 TeXGyreTermes Bold                   /mnt/debian/texlive/2013/texmf-dist/fonts/type1/public/tex-gyre/qtmb.pfb
```
Comment 19 Christopher Yeleighton 2014-04-08 20:36:23 UTC
(In reply to comment #17)
> commented out the lines related to TeX Gyre Termes font. please be aware
> this is a workaround and contact the font author to get it fixed.

What makes Tex Gyre Termes so special that it must get fixed, rather than replaced with a better one?
That said, I tried to notify the authors at GUST-L but my letter was not published or replied to.  Maybe they will catch up with reality when they see their font dropped, or maybe they will not.
Comment 20 Ivo Anjo 2014-05-13 10:14:10 UTC
I'm also seeing this with latest Ubuntu 14.04 stable.

Sorry if I'm asking a stupid question, but the bug is marked as fixed but where is it fixed and how can I access the fix?
Comment 21 Tim 2014-05-14 09:02:34 UTC
(In reply to comment #20)
> I'm also seeing this with latest Ubuntu 14.04 stable.
> 
> Sorry if I'm asking a stupid question, but the bug is marked as fixed but
> where is it fixed and how can I access the fix?

Hi,
as far as I understood the font "Tex Gyre Termes" (is it a font, or what is it?) is configured to be used instead of (the font?) "Times" because it is similar?

Not knowing much about fontconfig, I found Termes configured in /etc/fonts/conf.d/30-metric-aliases.conf und /etc/fonts/conf.d/30-fonts-texgyre-aliases.conf
In both files I commented out the parts with Tex Gyre Termes relating to Times (using <!-- ... --> ) and afterwards everything is displayed fine.
So the workaround is good for me, I hope it helps other, too, but it sure doesn't FIX the bug and I'm also not sure if all those commenting out is necessary. Maybe someone could comment on this...
Comment 22 nfxjfg 2014-05-26 14:06:56 UTC
If this is a regression, then what change caused it?
Comment 23 Akira TAGOH 2014-05-27 03:05:05 UTC
(In reply to comment #22)
> If this is a regression, then what change caused it?

Not really I'd say. the change for this workaround is mentioned above. if you want a certain link to the commit:
http://cgit.freedesktop.org/fontconfig/commit/?id=c6aa4d4bfcbed14f39d070fe7ef90a4b74642ee7

It is in 2.11.1 BTW.
Comment 24 Christopher Yeleighton 2014-05-27 16:35:53 UTC
(In reply to comment #22)
> If this is a regression, then what change caused it?

The adoption of Termes as the default substitution for Times, I guess.
Comment 25 Norbert Preining 2014-06-28 06:26:14 UTC
Ok, I reopen this bug against poppler, as poppler seems to implement something that is in contradiction to what Adobe recommends.

I refer you to the Adobe Glyph List Specification
http://sourceforge.net/adobe/aglfn/wiki/AGL%20Specification/
which clearly states in Section 6. Assigning glyph names for new fonts:

*******
For glyphs that represent ligatures of standard Unicode characters, there are two suggested formats for their glyph names, as follows:

    1. Descriptive. The decomposition is expressed by joining the glyph names of the standard Unicode characters, in order, using an underscore (U+005F; LOW LINE). The glyph names of the characters should specify the "uni" or "u" prefixes and use uppercase hexadecimal digits, as described above, or with a name from AGL. For example, the "o f f i" ligature should be named "o_f_f_i."

    2. UV with "uni" prefix. ....
*******

Thus, poppler should check for f_i *as*well*as* fi (for backward compatibility).

Thanks

Norbert Preining
TeX Live Team
Comment 26 Fabian Greffrath 2014-08-27 15:06:43 UTC
I have attached a new patch to #80093 that could use some review.

Thanks

Fabian
Comment 27 Fabian Greffrath 2014-10-01 16:20:20 UTC
I am re-assigning this bug report to fontconfig.

Dear fontconfig-devs,

now that a proper fix for the missing ligature glyphs in Tex Gyre Termes has made it into poppler [0,1], could you please revert the commit [2] to fontconfig that disabled Termes as a metric alias for Times? 

Thank you so much!

Fabian

[0] https://bugs.freedesktop.org/show_bug.cgi?id=80093#c16
[1] http://cgit.freedesktop.org/poppler/poppler/commit/?id=01723aa17e836e818158dbdc56df642a290be300
[2] http://cgit.freedesktop.org/fontconfig/commit/conf.d/30-metric-aliases.conf?id=c6aa4d4bfcbed14f39d070fe7ef90a4b74642ee7
Comment 28 Akira TAGOH 2014-10-03 03:29:01 UTC
revert the change in git. thanks!

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.