Bug 65258

Summary: HarfBuzz can not correctly shaping traditional Mongolian with free variation selector
Product: HarfBuzz Reporter: SiqinBilige <siqinbilige>
Component: srcAssignee: Behdad Esfahbod <freedesktop>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: medium CC: freedesktop, jfkthame
Version: unspecified   
Hardware: x86-64 (AMD64)   
OS: Linux (All)   
Whiteboard:
i915 platform: i915 features:
Attachments: screenshot
Result on Ubuntu firefox
forget the first screenshot

Description SiqinBilige 2013-06-02 08:59:37 UTC
On Ubuntu 12.10, firefox 21. http://www.mongolfont.com/test/sample.html.

HarfBuzz can not correctly shaping traditional Mongolian with free variation selector.

Mongolian free variation selector is FVS1(U+180B),FVS2(U+180C),FVS3(U+180D).

1. HarfBuzz can not handle lookupflag IgnoreMarks ?
or
2. HarfBuzz filtered Mongolian free variation selector ?
  Mongolian free variation selector can filtered in isol,init,medi,fina feature.
  But, can not filtered in calt or rlig feature.
Comment 1 Behdad Esfahbod 2013-06-03 23:02:11 UTC
I don't understand what's not working.  Please attach a font and the shortest sequence that shapes incorrectly.  Ideally with expected and actual rendering.
Comment 2 SiqinBilige 2013-06-04 02:16:11 UTC
Created attachment 80254 [details]
screenshot

1. The font can download from http://www.mongolfont.com/jAlmas/cms/documents/mongolfont/font/MongolianWhite.ttf.
  you also use Mongolian Baiti of Microsoft Windows Vista,7,8 to confirm.
2. The short test can access from http://www.mongolfont.com/test/sample_harfbuzz.html.
3. The attachment file HarfBuzzScreenshot.PNG will explain the details.
Comment 3 SiqinBilige 2013-06-04 02:25:48 UTC
Created attachment 80255 [details]
Result on Ubuntu firefox
Comment 4 SiqinBilige 2013-06-04 02:41:55 UTC
Created attachment 80257 [details]
forget the first screenshot

forget the first screenshot. see this one.
Comment 5 Behdad Esfahbod 2013-06-04 18:42:27 UTC
Ok, I see what's going on.  Will work on fixing it.
Comment 6 Behdad Esfahbod 2013-06-06 23:05:49 UTC
Discussion:
http://lists.freedesktop.org/archives/harfbuzz/2013-June/003348.html
Comment 7 Behdad Esfahbod 2013-06-13 23:06:53 UTC
commit 79d1007a501fd63c0ba4d51038c513e6b8b94740
Author: Behdad Esfahbod <behdad@behdad.org>
Date:   Thu Jun 13 19:01:07 2013 -0400

    If variation selector is not consumed by cmap, pass it on to GSUB
    
    This changes the semantics of get_glyph() callback and expect that
    callbacks return false if the requested variant is not available, and
    then we will call them back with variation_selector=0 and will retain
    the glyph for the selector in the glyph stream.
    
    Apparently most Mongolian fonts implement the Mongolian Variation
    Selectors using GSUB, not cmap.
    
    https://bugs.freedesktop.org/show_bug.cgi?id=65258
    
    Note that this doesn't fix the Mongolian shaping yet, because the way
    that's implemented is that the, say, 'init' feature ligates the letter
    and the variation-selector.  However, since currently the variation
    selector doesn't have the 'init' mask on, it will not be matched...
Comment 8 Behdad Esfahbod 2013-10-18 14:20:42 UTC
Will be fixed in tonight's push.
Comment 9 SiqinBilige 2013-10-19 01:59:16 UTC
(In reply to comment #8)
> Will be fixed in tonight's push.

Can you tell me howto test from source?
I only know test on browser, such as chrome.
Comment 10 Behdad Esfahbod 2014-07-16 17:33:37 UTC
Fully fixed now:

commit 3b861421a772f52eb232ff93bd74b5a8214801ec
Author: Behdad Esfahbod <behdad@behdad.org>
Date:   Wed Jul 16 13:22:05 2014 -0400

    Fix Mongolian Variation Selectors for fonts without GDEF
    
    Originally we fixed those in 79d1007a501fd63c0ba4d51038c513e6b8b94740.
    However, fonts like MongolianWhite don't have GDEF, but have IgnoreMarks
    in their LigatureSubstitute init/etc features.  We were synthesizing a
    GDEF class of mark for Mongolian Variation Selectors and as such the
    ligature lookups where not matching.  Uniscribe doesn't do that.
    
    I tried with more sophisticated fixes, like, if there is no GDEF and
    a lookup-flag mismatch happens, instead of rejecting a match, try
    skipping that glyph.  That surely produces some interesting behavior,
    but since we don't want to support fonts missing GDEF more than we have
    to, I went for this simpler fix which is to always mark
    default-ignorables as base when synthesizing GDEF.
    
    Micro-test added.
    
    Fixes rest of https://bugs.freedesktop.org/show_bug.cgi?id=65258
Comment 11 Behdad Esfahbod 2014-07-17 20:02:11 UTC
Another try:

commit 615d00ea252739da57edbd980ff27e573f88ee7e
Author: Behdad Esfahbod <behdad@behdad.org>
Date:   Thu Jul 17 13:36:09 2014 -0400

    [arabic] Apply init/medi/isol/fini/... in separate stages
    
    Follows the order of the Arabic/Syriac specs.  Also don't stop
    between rlig and calt in non-Arabic scripts.
    
    Micro-tests for Arabic and Mongolian added for the latter.


commit 164c13d73f67fdddba28e6409d76b4903e8ffab3
Author: Behdad Esfahbod <behdad@behdad.org>
Date:   Thu Jul 17 14:16:38 2014 -0400

    Another try to fix Mongolian free variation selectors
    
    This reverts bf029281 and fixes it properly.  That commit
    was not enough as it was only inheriting the shaping_action
    for prev_action, but not curr_action.
    
    Micro-test added.
    
    https://code.google.com/p/chromium/issues/detail?id=393896

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.