Bug 82

If this is related to an old bug, please reopen that one with new information. 
If not, please ammend this bug with enough information to find and fix the problem.

that bugs was fixed at Feb. 27 with talking in irc.gimp.net #unicode
channel.(not bugzilla) ;)

you were commented in fonts.conf like this :
  Some Asian fonts misadvertise themselves as monospaced when
  in fact they are dual-spaced (half and full).  This makes
  FreeType very confused as it forces all widths to match.
  Undo this magic by disabling the width forcing code

and fcfreetype.c :
     * don't even try this -- CJK 'monospace' fonts are really
     * dual width, and most other fonts don't bother to set
     * the attribute.  Sigh.

so, from fontconfig 2.1.90 to 2.1.92, it works fine, but after 2.1.93 appears
same bug. 

i don't know how appear that bugs again, cause there are many changes between
2.1.92 and 2.1.93. 

Ooops. I based my patch for Mozilla
(http://bugzilla.mozilla.org/show_bug.cgi?id=196312) on the assumption that this
is not an issue any more in fontconfig 2.0 or later and it's committed 12hrs ago
:-) Perhaps, I have to remove the condition ( GetFcVersion < 20200). 

> 2.0 or later

I meant  2.2 or later...


 

It seems like some versions of FT2 are also taking a  short-cut 
similar to the one taken by fontconfig before 2.2. In fcfreetype.c,
FC_SPACING is set to the value of spacing after calling 
FcFreeTypeCharsetAndSpacing which appears to (I just took a glance
at it) set spacing by checking whether all the glyphs 
have the same advance width as the global advance width of the font
(by invoking FcFreeTypeCheckGlyph that in turn relies on FT2)
If FT2 is also fooled by 'monospace' bit present in dual-width fonts and
sets metrics.horiAdvance without actually checking individual glyphs, 
this wouldn't work, either.   

I don't know whether this problem is still present in the current CVS
of FT2. If so, we have to report it after confirming that it's indeed
the case. Then, FC can check FT2 version and block the following code
if necessary.

    if (spacing != FC_PROPORTIONAL)
        if (!FcPatternAddInteger (pat, FC_SPACING, spacing))
            goto bail2;

I've just confirmed that it's indeed FT2 that is to blame.
FT2 (as of 2.1.2) API FT_Load_Glyph sets 
face->glyph->metrics.horiAdvance to the global advance width
whether a glyph is single width or double width.

Fontconfig has a kludge that ignores the global advance width that these fonts
are  setting, but that kludge is only used when loading the fonts in
applications, not when scanning them.  I think this is causing the problems that
you've discovered.  Perhaps the right fix is to have fontconfig run the pattern
through the font match/edit instructions before checking glyph widths so those
fixes can be detected.

Alternatively, we could always ignore the global advance when scanning the font;
that might miss some fonts which are fixed width because of a c.orrect global
advance, but shouldn't otherwise cause problems.  That's a *lot* easier.  I'd
like to have you give this a try; it's a trivial patch (add
FT_LOAD_IGNORE_GLOBAL_ADVANCE_WIDTH to load_flags in FcFreeTypeCheckGlyph)

> Fontconfig has a kludge that ignores the global advance width that these 
fonts
> are  setting, but that kludge is only used when loading the fonts in
> applications,

  I'm aware of the kludge, but even that doesn't seem to work for Mozilla(and 
some other applications like xterm), which is why I decided to apply a fix you 
suggested back in March for mozilla bug 196312
(http://bugzilla.mozilla.org/show_bug.cgi?id=196312). I also submitted a patch 
to xterm to work around that. (a similar patch was already applied to gnome-
terminal and vim-gtk as well). I might be missing something...

> I'd like to have you give this a try; it's a trivial patch (add
> FT_LOAD_IGNORE_GLOBAL_ADVANCE_WIDTH to load_flags in FcFreeTypeCheckGlyph)

  Yes, that should work (I can't test it at the moment).  I made a patch to 
get FcFreeTypeQuery to check FT2 version  (assuming that FT2 will be fixed in 
this regard) [1] before invoking FcFreeTypeCharSetAndSpacing (with an 
additional parameter, add_load_flag set to FT_LOAD_IGNORE_...TH) that in turn 
invokes FcFreeTypeCheckGlyph with the same add. parameter that is OR'd with 
other pre-set load flags. However, I guess it's not worth the trouble. 

  Will you bump up the version number of fontconfig after this fix so that 
Mozilla can delete FC_SPACING of the pattern conditionally? 

[1] I guess we can go even further and  check OS2 table to see if a font is 
for CJK. 

> > I'd like to have you give this a try; it's a trivial patch (add
> > FT_LOAD_IGNORE_GLOBAL_ADVANCE_WIDTH to load_flags in FcFreeTypeCheckGlyph)
> 
>   Yes, that should work (I can't test it at the moment).  I made a patch to 
> get FcFreeTypeQuery to check FT2 version  (assuming that FT2 will be fixed in 
> this regard)

I'm not sure FT2 is "broken" in this regard; I don't quite understand what
the globaladvance value is supposed to be used for.  From my understanding,
the FT2 load flag was the only fix they intend to provide.

> Will you bump up the version number of fontconfig after this fix so that 
> Mozilla can delete FC_SPACING of the pattern conditionally? 

Yes, of course.  I (try to) bump the version number each time I do a
release.  If this fix looks benign, and appears to work, I'll release
another 2.2 version (2.2.1), and the fix will be in any 2.3 release as
well.

When you get a chance to test this trivial fix, I'll commit it to CVS.

I'll try the fix and let you know. BTW, instead of setting 'ignore global 
width', can you check 'ls' in FcFreeTypeQuery()
to see if 'ls' is for CJK before adding spacing to 'pat? (see my comment 
posted at 2003-05-12 07:14.) There isn't likely to be a significant gain with 
this, but I'm not the best person to judge this. 

> I'm not sure FT2 is "broken" in this regard; I don't quite understand what
> the globaladvance value is supposed to be used for.  From my understanding,
> the FT2 load flag was the only fix they intend to provide

  I'm not sure what TTF spec. says about it. I wrote to the FT people 
suggesting that they have to take into account the way being 'monospace' means 
for CJK 'dual-width' fonts.[1] When 'monospace' bit/flag is set for CJK fonts, 
FT2 can refer to the East Asian width class table and return 1/2 of the value 
of globaladvance  for characters of EA width class 'N'arrow and the value of 
globaladvance for the rest (EA width class 'W' and EA width 
class 'A'mbiguous). This will work quite well(but speedwise, it might hurt) as 
long as East Asian fonts don't have any characters from Indic scripts and 
similar scripts.


[1] For instance, under Windows, CJK 'dual-width' fonts are listed along with 
genuine monospace fonts in the list of monospace fonts. I think WIN32 
API 'enumerate_font (spacing=monospace)' (I'm just using pseudo-name) returns 
genuine monospace fonts as well as CJK dual-width fonts. This may be a kludge, 
but that's what CJK users expect them to behave.....   

> can you check 'ls' in FcFreeTypeQuery()
> to see if 'ls' is for CJK before adding spacing to 'pat?

I'd like to avoid language-specific kludges if possible; fonts which don't quite
meet the orthography coverage will be mystically broken.

> When 'monospace' bit/flag is set for CJK fonts, FT2 can refer to the East
> Asian width class table and return 1/2 of the value of globaladvance  for 
> characters of EA width class 'N'arrow and the value of globaladvance for 
> the rest (EA width class 'W' and EA width class 'A'mbiguous).

I don't think we'll get the FT folks to include a giant table of attributes to
solve this problem.  I suspect we'll have to push this out to the layout library
where such information already needs to be present.

It took me ... I don't know how long. I recompiled Xft/fontconfig/my small test
program ( converted from basicwin.c included in ORA's X11book) and
nsFontMetricsXft.cpp of Mozilla many times with a 
number of combinations of several tweakings in library source and configuration
files[1] None of them worked. Setting XFT_DBG_?? to 104, I confirmed that glyphs
came up with the correct
width, but still spacing was wrong. Finally, I gdb'd basicwin-xft and found that
my version of FT2 apparently doesn't honor GLOBAL_IGNORE....flag. I upgraded FT2
to the CVS snapshot and it began to work. Hew.... 

Strange is that the CVS snapshot doesn't seem to do anything differently from
what used to be on my system (FT 2.1.4rc1)...  To be sure, I recompiled FT
2.1.4rc1 and reinstalled it. The symptom come back ! When I installed FT
post-2.1.4(CVS snapshot), it began to work again.  There are two changes that
appear related with the problem _remotely_ (bas/ftobjs.c and ttgload.c)

So, I have to check FT version in Mozilla, which is not so simple as checking Fc
version.

> I don't think we'll get the FT folks to include a giant table of 
> attributes to solve this problem.

 If they're not fond of including a giant table, they can, by default, turn on
'..GLOBAL_IGNORE....' for fonts with 'monospace' bit set and OS/2 table (or
othter indication) of 'CJK-ness' present, can't they? I can't think of any
application that wants 's p a c e d o u t' rendering _behind_  their back (as
opposed to in accordance with their explicit necessity/desire).   

A bit more info:

1. fonconfig 2.1 (+RH8 patch)
   freetype 2.1.4 rc1, 
   deleting FC_SPACING before actually opening XftFont 
   but after matching  
   -----> works

2. fontconfig CVS (with ...GLOBAL_IGNORE..) in scanning
   freetype 2.1.4 rc1 
   a. with or b. without FC_SPACING deleted in application
   ---> does not work
   
   It's beyond me why 2a didn't work unless it's sunspot or something..

3. fontconfig CVS with OR _without_ ...GLOBAL_INGORE... added
   freetype CVS snapshot
   with or without FC_SPACING deleted in application --> works 

In all cases, my fonts.conf has 'globaladvance' resetting rules for CJK fonts. 

In short, this bug has little to do with fontconfig but has to do with some
unknown changes in FT2 between 2.1.4 rc1 and the current CVS snapshot. 

Another factor is Xft, but it's in Xft 1.17 that 'global_advance' kludge was
introduced so that that couldn't have affected the outcome. 

Given that FreeType has long since been fixed, I'm closing this bug.