Bug 58641

Created attachment 71961 [details]
The Wingdings font copied from M$ Windows.

Comment on attachment 71961 [details]
The Wingdings font copied from M$ Windows.

This font is widely used in M$ Windows and Office suite. Some website uses this font to display some itemize bullets. It is a important font that is needed in WPS for Linux(A Office Suit) to correctly render office documents.

Downgrading fontconfig to 2.8.0 solves the problem.
A bug in fontconfig causes xfd cannot get the "range" information correctly.

I am sure that it is a bug in fontconfig.
I am an Arch Linux user and I suffers from this bug for a long time.
At first I suspected it was a bug in LibXft, but today I *downgraded my fontconfig from 2.10.2 to 2.8.0* and the bug disappear!
So apparently, it is a bug in fontconfig.

Please debug about it~

I can confirm this.  Please don't attach proprietary fonts to bugzilla.

(In reply to comment #5)
> I can confirm this.  Please don't attach proprietary fonts to bugzilla.

I am sorry for attaching improper files. I make it obsolete now but I can not delete it. If the administritor reads this comment, please help to delete that font file. Or if you have the administrator authority, please help delete it. Thank you for your concern. :)

The content of attachment 71961 [details] has been deleted by
    Alan Coopersmith <alan.coopersmith@oracle.com>
who provided the following reason:

Deleted as requested to avoid license issues

The token used to delete this attachment was generated at 2012-12-30 18:20:45 UTC.

Given that it doesn't work on even 2.9.0, is it maybe related to Bug#35517?

I have no idea.  That said, the font in question doesn't seem to have the MS Symbol encoding either.  Donno.  Someone should bisect.

(In reply to comment #9)
> I have no idea.  That said, the font in question doesn't seem to have the MS
> Symbol encoding either.  Donno.  Someone should bisect.

It seems that MS fonts, Wingdings & Symbol, have their own code pages.Those fonts map common characters(like 'C','D')'s ASCII codes to some different symbols. 

In the software WPS for Linux, they uses those fonts to correctly display Office file from M$ Office. It seems that M$ still uses those fonts in their Office suit. They find that 
----------------(in Chinese)
老版本，Symbol字体，绘制0x22能够正确的绘制倒三角，绘制0xF022也能正确绘制倒三角
老版本，Symbol [urw]字体，绘制0x22能够正确的绘制倒三角，绘制0xF022也能正确绘制倒三角
新版本，Symbol字体，绘制0x22不能正确绘制倒三角，0xF022也能正确绘制倒三角
新版本，Symbol [urw]字体，绘制0x22能够正确的绘制倒三角，绘制0xF022也能正确绘制倒三角
----------------(The following is the summary)
For Symbol font from M$ Window
In fontconfig 2.8.0 under Ubuntu 12.04(marked as Old Version) and fontconfig 2.10 under Ubuntu 12.10(marked as New Version),

Old Version can draw both characters 0x22 and 0xF022 correctly;
New Version con only draw character 0xF022 correctly, character 0x22 is displayed as a little square.

Thanks.  I'll bisect and see what regressed this.

The change indeed was a result of removing the Apple Roman encoding.  However, I don't think it's incorrect.

The font has two cmap tables:

  1. Apple Roman.  This one incorrectly maps Latin characters to the wingding symbols.

  2. MS Symbol.  This one maps private-use-area characters to the symbol.

Now, for some reason we are NOT reading the second table, so previously we were relying on the first one, which we don't read anymore.

The problem is, symbol fonts have no defined encoding.  One would never know what the glyph at position one really is.  So I think it's correct for fontconfig to NOT enumerate those glyphs.

Thanks for investigation.

(In reply to comment #12)
> The problem is, symbol fonts have no defined encoding.  One would never know
> what the glyph at position one really is.  So I think it's correct for
> fontconfig to NOT enumerate those glyphs.

I think there are similar issue about encoding on some BDF fonts which fontconfig can't deal with. so just wonder if fontconfig provides any way to embed the encoding information into the configuration as the meta data?

As you have implemented the bitmap scaling, such fonts is still somewhat usable?

My interest is in SFNT bitmap-only fonts, not BDF.

I don't think we need to do anything about this bug right now.

(In reply to comment #14)
> My interest is in SFNT bitmap-only fonts, not BDF.
> 
> I don't think we need to do anything about this bug right now.

Can you give me some advice if I still want to use those fonts on my system? Get another symbol font that uses the standard UTF encoding to display the symbols provided by Wingdings? Or just change some configuration? Thank you very much.

(In reply to comment #15)
> (In reply to comment #14)
> > My interest is in SFNT bitmap-only fonts, not BDF.
> > 
> > I don't think we need to do anything about this bug right now.
> 
> Can you give me some advice if I still want to use those fonts on my system?
> Get another symbol font that uses the standard UTF encoding to display the
> symbols provided by Wingdings? Or just change some configuration? Thank you
> very much.

What's your use case?  I don't think you can use it with Pango, since it has no reasonable character map.  You are still free to open the font directly (using FreeType API) and use it...

(In reply to comment #16)
> (In reply to comment #15)
> > (In reply to comment #14)
> > > My interest is in SFNT bitmap-only fonts, not BDF.
> > > 
> > > I don't think we need to do anything about this bug right now.
> > 
> > Can you give me some advice if I still want to use those fonts on my system?
> > Get another symbol font that uses the standard UTF encoding to display the
> > symbols provided by Wingdings? Or just change some configuration? Thank you
> > very much.
> 
> What's your use case?  I don't think you can use it with Pango, since it has
> no reasonable character map.  You are still free to open the font directly
> (using FreeType API) and use it...

OK, I got it. Thank you very much. I will try to persuade the developers who still use Symbol & Wingdings fonts in their softwares to change to anoter font or draw the character directly via FreeType API.

We are actually stumbling on the same problem with fontconfig 2.10.1 and pango 1.32.3 with the M$ Webdings font, but the issue boils down to the same thing I think.

The coverage computed by fontconfig is empty as the Apple Roman cmap is no longer read and the MS Symbol cmap is read but fontconfig only takes into account entries 0x20-0xFE, but Webdings has entries 0xF020-0xF0FF.

Reading the OpenType spec it appears to me that Windows treats Symbol fonts (i.e. having a cmap with platform ID 3 and encoding ID 0) specially, in that it remaps its entries to Unicode code points starting at usFirstCharIndex from the OS/2 table and can only map 234 characters at most. See the "Non-Standard (Symbol) Fonts" paragraph at http://www.microsoft.com/typography/otspec/recom.htm.

In our case, as fontconfig computes an empty coverage, pango never uses the font and always tries a fallback, so it is impossible to render with it (or at least we have not managed to do so in our application).

I do not know what would be the proper course of action:

- emulate Windows handling of symbol fonts as per the OpenType spec?

- emulate Mac by using the Apple Roman cmap (but only for fonts without a unicode cmap)?

- Read the MS symbol cmap entries in the UPA range and mark them as covered in fontconfig so that apps using such codepoints can render with these fonts? But in this case the fonts would not behave as one would expect from Windows or Mac experience, so I do not think it is very useful.

- other?

FWIW, the Wine project re-implemented wingdings font: http://source.winehq.org/source/fonts/
It is not complete yet.

Wine also support "symbol" encoding, which might be a useful reference for people want to support "symbol" encoding in other Linux program. BTW, anyone know if Mac OS support "symbol" encoding?
http://source.winehq.org/source/dlls/gdi32/freetype.c

For people who want to contribute to open source alternative fonts for windings/windings2/windings3/webdings, here is a good start:
http://unicode.org/~asmus/web-wing-ding-ext.pdf 
The above report analysed windings/windings2/webdings and found same/similar glyphs in unicode standard.
Nowadays more symbol glyphs are added to latest unicode standard, which covers more windings(n)/webdings glyphs.
We are working on extent the above report base on PCANet: http://arxiv.org/abs/1404.3606

Symbola font contains many glyphs which is useful for re-implementing windings(n)/webdings:
http://users.teilar.gr/~g1951d/
According to the above page, the license of Symbola is "free for any use".

Cheers.

I believe I've now fixed this:

commit d6d5adeb7940c0d0beb86489c2a1c2ce59e5c044
Author: Behdad Esfahbod <behdad@behdad.org>
Date:   Wed Jan 21 14:13:36 2015 -0800

    Fix symbol cmap handling
    
    A while back we removed Apple Roman encoding support.  This broke
    symbol fonts (Wingdings, etc) because those fonts come with two
    cmaps:
    
      1) platform=1,encoding=0, aka Apple Roman, which maps identity,
      2) platform=3,encoding=0, aka MS Symbol font
    
    Now, the reason the Apple Roman removal "broke" these fonts is
    obvious, and for the better: these fonts were mapping ASCII and
    other Latin chars to symbols.
    
    The reason the fonts didn't work anymore, however, is that we were
    mishandling the MS symbol-font cmaps.  In their modern incarnation
    they are like regular non-symbol-font cmap that map PUA codepoints
    to symbols.  We want to expose those as such.  Hence, this change
    just removes the special-handling for that.
    
    Now, the reason this confusion happened, if I was to guess, is either
    that FreeType docs are wrong saying that FT_ENCODING_MS_SYMBOL is
    the "Microsoft Symbol encoding, used to encode mathematical symbols":
    
      http://www.kostis.net/charsets/symbol.htm
    
    or maybe it started that way, but turned into also mapping MS symbol-
    font cmaps, which is a completely different thing.  At any rate, I
    don't know if there are any fonts that use this thing these days, but
    the code here didn't seem to produce charset for any font.  By now I'm
    convinced that this change is the Right Thing to do.  The MS Symbol
    thing was called AdobeSymbol in our code by the way.
    
    This fixes the much-reported bug that windings, etc are not usable
    with recent fontconfig:
    https://bugs.freedesktop.org/show_bug.cgi?id=58641
    
    Now I see PUA mappings reported for Wingdings.
    
    This also fixes:
    Bug 48947 - Drop the non-Unicode cmap support gradually
    https://bugs.freedesktop.org/show_bug.cgi?id=48947
    since the AdobeSymbol was the last non-Unicode cmap we were
    trying to parse (very incorrectly).
    
    Lots of code around this change can be simplified.  I'll push those
    out (including removing the table itself) in subsequent changes.