Bug 58641 - font "Wingdings" is displayed incorrectly and the bug in fontconfig crashes xfd
Summary: font "Wingdings" is displayed incorrectly and the bug in fontconfig crashes xfd
Status: RESOLVED FIXED
Alias: None
Product: fontconfig
Classification: Unclassified
Component: library (show other bugs)
Version: unspecified
Hardware: x86-64 (AMD64) Linux (All)
: medium major
Assignee: fontconfig-bugs
QA Contact: Behdad Esfahbod
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-12-22 04:52 UTC by Dale Wang
Modified: 2015-01-21 22:25 UTC (History)
5 users (show)

See Also:
i915 platform:
i915 features:


Attachments
The screen shot of the incorrect font display. (128.95 KB, image/jpeg)
2012-12-22 04:52 UTC, Dale Wang
Details
The Wingdings font copied from M$ Windows. (deleted)
2012-12-22 04:55 UTC, Dale Wang
Details

Description Dale Wang 2012-12-22 04:52:53 UTC
Created attachment 71960 [details]
The screen shot of the incorrect font display.

libXft has some problems when it displayes the font "Wingdings" widely used in M$ Windows and Office. "Wingdings" also crashed the little xfd program.

All the characters in the font "Wingdings" are displayed as a little square. When I try to use the tool program "xfd" to inspect on the font, "xfd" didn't response correctly. "xfd" can not display the characters in this font. All characters in the font are displayed as a little square and when I press the "Next" button, the xfd loses response and the cpu usage of xfd is very high.

This bug also affects other programs like conky, Font Viewer of Gnome(Font Viewer in Gnome can not display the characters in Wingdings correctly), WPS For Linux and so on.

But when I select the font Wingdings in LibreOffice, it works and displays correctly.

* package version(s)
libxft 2.3.1-1
xorg-xfd 1.1.1-1
*Linux version:Arch Linux amd64

*Steps to reproduce:
1.Install the font file "Wingdings" from the url(https://bugs.archlinux.org/task/32932?getfile=9775)[I reported the bug before in Arch, but it seems that the problem is about xorg]
3.Run "xfd -fa Wingdings"
4.All the characters are little squares
5.Press the button "Next"
6.xfd doesn't response.

The picture in attachment is the screen shot.
Comment 1 Dale Wang 2012-12-22 04:55:38 UTC
Created attachment 71961 [details]
The Wingdings font copied from M$ Windows.
Comment 2 Dale Wang 2012-12-22 04:57:34 UTC
Comment on attachment 71961 [details]
The Wingdings font copied from M$ Windows.

This font is widely used in M$ Windows and Office suite. Some website uses this font to display some itemize bullets. It is a important font that is needed in WPS for Linux(A Office Suit) to correctly render office documents.
Comment 3 Dale Wang 2012-12-29 10:02:48 UTC
Downgrading fontconfig to 2.8.0 solves the problem.
A bug in fontconfig causes xfd cannot get the "range" information correctly.
Comment 4 Dale Wang 2012-12-29 10:28:26 UTC
I am sure that it is a bug in fontconfig.
I am an Arch Linux user and I suffers from this bug for a long time.
At first I suspected it was a bug in LibXft, but today I *downgraded my fontconfig from 2.10.2 to 2.8.0* and the bug disappear!
So apparently, it is a bug in fontconfig.

Please debug about it~
Comment 5 Behdad Esfahbod 2012-12-29 23:02:15 UTC
I can confirm this.  Please don't attach proprietary fonts to bugzilla.
Comment 6 Dale Wang 2012-12-30 02:02:49 UTC
(In reply to comment #5)
> I can confirm this.  Please don't attach proprietary fonts to bugzilla.

I am sorry for attaching improper files. I make it obsolete now but I can not delete it. If the administritor reads this comment, please help to delete that font file. Or if you have the administrator authority, please help delete it. Thank you for your concern. :)
Comment 7 Alan Coopersmith 2012-12-30 18:21:08 UTC
The content of attachment 71961 [details] has been deleted by
    Alan Coopersmith <alan.coopersmith@oracle.com>
who provided the following reason:

Deleted as requested to avoid license issues

The token used to delete this attachment was generated at 2012-12-30 18:20:45 UTC.
Comment 8 Akira TAGOH 2013-01-04 02:03:29 UTC
Given that it doesn't work on even 2.9.0, is it maybe related to Bug#35517?
Comment 9 Behdad Esfahbod 2013-01-04 02:21:59 UTC
I have no idea.  That said, the font in question doesn't seem to have the MS Symbol encoding either.  Donno.  Someone should bisect.
Comment 10 Dale Wang 2013-01-06 08:27:45 UTC
(In reply to comment #9)
> I have no idea.  That said, the font in question doesn't seem to have the MS
> Symbol encoding either.  Donno.  Someone should bisect.

It seems that MS fonts, Wingdings & Symbol, have their own code pages.Those fonts map common characters(like 'C','D')'s ASCII codes to some different symbols. 

In the software WPS for Linux, they uses those fonts to correctly display Office file from M$ Office. It seems that M$ still uses those fonts in their Office suit. They find that 
----------------(in Chinese)
老版本,Symbol字体,绘制0x22能够正确的绘制倒三角,绘制0xF022也能正确绘制倒三角
老版本,Symbol [urw]字体,绘制0x22能够正确的绘制倒三角,绘制0xF022也能正确绘制倒三角
新版本,Symbol字体,绘制0x22不能正确绘制倒三角,0xF022也能正确绘制倒三角
新版本,Symbol [urw]字体,绘制0x22能够正确的绘制倒三角,绘制0xF022也能正确绘制倒三角
----------------(The following is the summary)
For Symbol font from M$ Window
In fontconfig 2.8.0 under Ubuntu 12.04(marked as Old Version) and fontconfig 2.10 under Ubuntu 12.10(marked as New Version),

Old Version can draw both characters 0x22 and 0xF022 correctly;
New Version con only draw character 0xF022 correctly, character 0x22 is displayed as a little square.
Comment 11 Behdad Esfahbod 2013-01-06 11:55:40 UTC
Thanks.  I'll bisect and see what regressed this.
Comment 12 Behdad Esfahbod 2013-01-08 00:06:00 UTC
The change indeed was a result of removing the Apple Roman encoding.  However, I don't think it's incorrect.

The font has two cmap tables:

  1. Apple Roman.  This one incorrectly maps Latin characters to the wingding symbols.

  2. MS Symbol.  This one maps private-use-area characters to the symbol.

Now, for some reason we are NOT reading the second table, so previously we were relying on the first one, which we don't read anymore.

The problem is, symbol fonts have no defined encoding.  One would never know what the glyph at position one really is.  So I think it's correct for fontconfig to NOT enumerate those glyphs.
Comment 13 Akira TAGOH 2013-01-08 02:28:06 UTC
Thanks for investigation.

(In reply to comment #12)
> The problem is, symbol fonts have no defined encoding.  One would never know
> what the glyph at position one really is.  So I think it's correct for
> fontconfig to NOT enumerate those glyphs.

I think there are similar issue about encoding on some BDF fonts which fontconfig can't deal with. so just wonder if fontconfig provides any way to embed the encoding information into the configuration as the meta data?

As you have implemented the bitmap scaling, such fonts is still somewhat usable?
Comment 14 Behdad Esfahbod 2013-01-08 03:26:51 UTC
My interest is in SFNT bitmap-only fonts, not BDF.

I don't think we need to do anything about this bug right now.
Comment 15 Dale Wang 2013-01-13 11:30:20 UTC
(In reply to comment #14)
> My interest is in SFNT bitmap-only fonts, not BDF.
> 
> I don't think we need to do anything about this bug right now.

Can you give me some advice if I still want to use those fonts on my system? Get another symbol font that uses the standard UTF encoding to display the symbols provided by Wingdings? Or just change some configuration? Thank you very much.
Comment 16 Behdad Esfahbod 2013-01-14 06:22:45 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > My interest is in SFNT bitmap-only fonts, not BDF.
> > 
> > I don't think we need to do anything about this bug right now.
> 
> Can you give me some advice if I still want to use those fonts on my system?
> Get another symbol font that uses the standard UTF encoding to display the
> symbols provided by Wingdings? Or just change some configuration? Thank you
> very much.

What's your use case?  I don't think you can use it with Pango, since it has no reasonable character map.  You are still free to open the font directly (using FreeType API) and use it...
Comment 17 Dale Wang 2013-01-15 11:26:40 UTC
(In reply to comment #16)
> (In reply to comment #15)
> > (In reply to comment #14)
> > > My interest is in SFNT bitmap-only fonts, not BDF.
> > > 
> > > I don't think we need to do anything about this bug right now.
> > 
> > Can you give me some advice if I still want to use those fonts on my system?
> > Get another symbol font that uses the standard UTF encoding to display the
> > symbols provided by Wingdings? Or just change some configuration? Thank you
> > very much.
> 
> What's your use case?  I don't think you can use it with Pango, since it has
> no reasonable character map.  You are still free to open the font directly
> (using FreeType API) and use it...

OK, I got it. Thank you very much. I will try to persuade the developers who still use Symbol & Wingdings fonts in their softwares to change to anoter font or draw the character directly via FreeType API.
Comment 18 Diego Santa Cruz 2013-03-19 23:05:19 UTC
We are actually stumbling on the same problem with fontconfig 2.10.1 and pango 1.32.3 with the M$ Webdings font, but the issue boils down to the same thing I think.

The coverage computed by fontconfig is empty as the Apple Roman cmap is no longer read and the MS Symbol cmap is read but fontconfig only takes into account entries 0x20-0xFE, but Webdings has entries 0xF020-0xF0FF.

Reading the OpenType spec it appears to me that Windows treats Symbol fonts (i.e. having a cmap with platform ID 3 and encoding ID 0) specially, in that it remaps its entries to Unicode code points starting at usFirstCharIndex from the OS/2 table and can only map 234 characters at most. See the "Non-Standard (Symbol) Fonts" paragraph at http://www.microsoft.com/typography/otspec/recom.htm.

In our case, as fontconfig computes an empty coverage, pango never uses the font and always tries a fallback, so it is impossible to render with it (or at least we have not managed to do so in our application).

I do not know what would be the proper course of action:

- emulate Windows handling of symbol fonts as per the OpenType spec?

- emulate Mac by using the Apple Roman cmap (but only for fonts without a unicode cmap)?

- Read the MS symbol cmap entries in the UPA range and mark them as covered in fontconfig so that apps using such codepoints can render with these fonts? But in this case the fonts would not behave as one would expect from Windows or Mac experience, so I do not think it is very useful.

- other?
Comment 19 Qian Hong 2015-01-04 17:47:47 UTC
FWIW, the Wine project re-implemented wingdings font: http://source.winehq.org/source/fonts/
It is not complete yet.

Wine also support "symbol" encoding, which might be a useful reference for people want to support "symbol" encoding in other Linux program. BTW, anyone know if Mac OS support "symbol" encoding?
http://source.winehq.org/source/dlls/gdi32/freetype.c

For people who want to contribute to open source alternative fonts for windings/windings2/windings3/webdings, here is a good start:
http://unicode.org/~asmus/web-wing-ding-ext.pdf 
The above report analysed windings/windings2/webdings and found same/similar glyphs in unicode standard.
Nowadays more symbol glyphs are added to latest unicode standard, which covers more windings(n)/webdings glyphs.
We are working on extent the above report base on PCANet: http://arxiv.org/abs/1404.3606

Symbola font contains many glyphs which is useful for re-implementing windings(n)/webdings:
http://users.teilar.gr/~g1951d/
According to the above page, the license of Symbola is "free for any use".

Cheers.
Comment 20 Behdad Esfahbod 2015-01-21 22:25:59 UTC
I believe I've now fixed this:

commit d6d5adeb7940c0d0beb86489c2a1c2ce59e5c044
Author: Behdad Esfahbod <behdad@behdad.org>
Date:   Wed Jan 21 14:13:36 2015 -0800

    Fix symbol cmap handling
    
    A while back we removed Apple Roman encoding support.  This broke
    symbol fonts (Wingdings, etc) because those fonts come with two
    cmaps:
    
      1) platform=1,encoding=0, aka Apple Roman, which maps identity,
      2) platform=3,encoding=0, aka MS Symbol font
    
    Now, the reason the Apple Roman removal "broke" these fonts is
    obvious, and for the better: these fonts were mapping ASCII and
    other Latin chars to symbols.
    
    The reason the fonts didn't work anymore, however, is that we were
    mishandling the MS symbol-font cmaps.  In their modern incarnation
    they are like regular non-symbol-font cmap that map PUA codepoints
    to symbols.  We want to expose those as such.  Hence, this change
    just removes the special-handling for that.
    
    Now, the reason this confusion happened, if I was to guess, is either
    that FreeType docs are wrong saying that FT_ENCODING_MS_SYMBOL is
    the "Microsoft Symbol encoding, used to encode mathematical symbols":
    
      http://www.kostis.net/charsets/symbol.htm
    
    or maybe it started that way, but turned into also mapping MS symbol-
    font cmaps, which is a completely different thing.  At any rate, I
    don't know if there are any fonts that use this thing these days, but
    the code here didn't seem to produce charset for any font.  By now I'm
    convinced that this change is the Right Thing to do.  The MS Symbol
    thing was called AdobeSymbol in our code by the way.
    
    This fixes the much-reported bug that windings, etc are not usable
    with recent fontconfig:
    https://bugs.freedesktop.org/show_bug.cgi?id=58641
    
    Now I see PUA mappings reported for Wingdings.
    
    This also fixes:
    Bug 48947 - Drop the non-Unicode cmap support gradually
    https://bugs.freedesktop.org/show_bug.cgi?id=48947
    since the AdobeSymbol was the last non-Unicode cmap we were
    trying to parse (very incorrectly).
    
    Lots of code around this change can be simplified.  I'll push those
    out (including removing the table itself) in subsequent changes.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.