Bug 31969

further updates for the range support:

http://cgit.freedesktop.org/~tagoh/fontconfig/commit/?h=charset-modification&id=2264080ca1c4c68e8623d2d6d81a3908d65d5013

Doh. another change for fixing bug:
http://cgit.freedesktop.org/~tagoh/fontconfig/commit/?h=charset-modification&id=40fbd16fe9c0bc5d8c75d60e50d38ffd9e971180

To drop the charset for the entire ASCII part from the font:

<match target="scan">
    <test name="family">
        <string>blahblahblah font</string>
    </test>
    <edit name="charset" mode="remove">
        <range object="charset">
            <int>20</int>
            <int>127</int>
        </range>
    </edit>
</match>

For more convenience:

http://cgit.freedesktop.org/~tagoh/fontconfig/commit/?h=charset-modification&id=e16e3a397c850a675f7964c46c5bd608a2c58e89

<match target="scan">
    <test name="family">
        <string>blahblahblah font</string>
    </test>
    <edit name="charset" mode="remove">
        <range object="charset">
            <string>U+0020</string>
            <string>U+007F</string>
        </range>
    </edit>
</match>

(In reply to comment #3)
> For more convenience:
> 
> http://cgit.freedesktop.org/~tagoh/fontconfig/commit/?h=charset-modification&id=e16e3a397c850a675f7964c46c5bd608a2c58e89
> 
> <match target="scan">
>     <test name="family">
>         <string>blahblahblah font</string>
>     </test>
>     <edit name="charset" mode="remove">
>         <range object="charset">
>             <string>U+0020</string>
>             <string>U+007F</string>
>         </range>
>     </edit>
> </match>

The syntax doesn't make sense.  Something like this is more like what I expect, given how the matrix operations currently work:

  <edit name="charset" mode="assign">
    <minus>
      <name>charset</name>
      <charset>
          <int>0x06CC</int>  <!-- ARABIC LETTER FARSI YEH -->
          <int>0x06D2</int>  <!-- ARABIC LETTER YEH BARREE -->
          <int>0x06D3</int>  <!-- ARABIC LETTER YEH BARREE WITH HAMZA ABOVE -->
      <charset>
    <minus>
  </edit>


I deliberately want to avoid ranges for now.  But if you have compelling usecases for ranges, then something like this would do:

  <edit name="charset" mode="assign">
    <minus>
      <name>charset</name>
      <charset>
          <int>0x06CC</int>  <!-- ARABIC LETTER FARSI YEH -->
          <range>
            <int>0x06D2</int>  <!-- ARABIC LETTER YEH BARREE -->
            <int>0x06D3</int>  <!-- ARABIC LETTER YEH BARREE WITH HAMZA ABOVE -->
          </range>
      <charset>
    <minus>
  </edit>

(In reply to comment #4)
> The syntax doesn't make sense.  Something like this is more like what I expect,
> given how the matrix operations currently work:
> 
>   <edit name="charset" mode="assign">
>     <minus>
>       <name>charset</name>
>       <charset>
>           <int>0x06CC</int>  <!-- ARABIC LETTER FARSI YEH -->
>           <int>0x06D2</int>  <!-- ARABIC LETTER YEH BARREE -->
>           <int>0x06D3</int>  <!-- ARABIC LETTER YEH BARREE WITH HAMZA ABOVE -->
>       <charset>
>     <minus>
>   </edit>

Got it. it seems that I too did shortcut to process the operations. let me re-work with the above. thank you for the comment.

> I deliberately want to avoid ranges for now.  But if you have compelling
> usecases for ranges, then something like this would do:
> 
>   <edit name="charset" mode="assign">
>     <minus>
>       <name>charset</name>
>       <charset>
>           <int>0x06CC</int>  <!-- ARABIC LETTER FARSI YEH -->
>           <range>
>             <int>0x06D2</int>  <!-- ARABIC LETTER YEH BARREE -->
>             <int>0x06D3</int>  <!-- ARABIC LETTER YEH BARREE WITH HAMZA ABOVE
> -->
>           </range>
>       <charset>
>     <minus>
>   </edit>

Hmm, well, the example in my previous comment to mark ASCII characters into the blacklist was one of what I expect in this feature. though it's still possible to have listing of them in the file but just thought it would be good for convenience, but anyway.

(In reply to comment #5)
> (In reply to comment #4)
> > The syntax doesn't make sense.  Something like this is more like what I expect,
> > given how the matrix operations currently work:
> > 
> >   <edit name="charset" mode="assign">
> >     <minus>
> >       <name>charset</name>
> >       <charset>
> >           <int>0x06CC</int>  <!-- ARABIC LETTER FARSI YEH -->
> >           <int>0x06D2</int>  <!-- ARABIC LETTER YEH BARREE -->
> >           <int>0x06D3</int>  <!-- ARABIC LETTER YEH BARREE WITH HAMZA ABOVE -->
> >       <charset>
> >     <minus>
> >   </edit>
> 
> Got it. it seems that I too did shortcut to process the operations. let me
> re-work with the above. thank you for the comment.

Thanks.


> > I deliberately want to avoid ranges for now.  But if you have compelling
> > usecases for ranges, then something like this would do:
> > 
> >   <edit name="charset" mode="assign">
> >     <minus>
> >       <name>charset</name>
> >       <charset>
> >           <int>0x06CC</int>  <!-- ARABIC LETTER FARSI YEH -->
> >           <range>
> >             <int>0x06D2</int>  <!-- ARABIC LETTER YEH BARREE -->
> >             <int>0x06D3</int>  <!-- ARABIC LETTER YEH BARREE WITH HAMZA ABOVE
> > -->
> >           </range>
> >       <charset>
> >     <minus>
> >   </edit>
> 
> Hmm, well, the example in my previous comment to mark ASCII characters into the
> blacklist was one of what I expect in this feature. though it's still possible
> to have listing of them in the file but just thought it would be good for
> convenience, but anyway.

My vision for usecases of this feature is to excluded faulty characters from a font.  For anything beyond that, I expect editing based on lang to be adequate.  However, editing based on script is really what we need in reality, so maybe you're right that using charset instead of lang is preferred.

We can also add charsets representing scripts in Unicode....

(In reply to comment #6)
> My vision for usecases of this feature is to excluded faulty characters from a
> font.  For anything beyond that, I expect editing based on lang to be adequate.
>  However, editing based on script is really what we need in reality, so maybe
> you're right that using charset instead of lang is preferred.

That's also true. and it may be case-by-case. given that there are any fonts doesn't have some lang in the cache as a result due to missing a few glyphs, editing based on script doesn't help in this case. editing lang would makes more sense. so that would be nice to have both features of editing IMHO.

Yes, lets have both lang and charset editing.  Lang by itself is not enough.  If a font has, say, a faulty 'a' glyph, you don't want to be bothered with removing all the langs that use the character 'a'.

Ok, re-working done.

http://cgit.freedesktop.org/~tagoh/fontconfig/commit/?h=charset-modification-take2&id=a240c3e988be9eed2d4cba8ad2b4345a81ab958d

I modified the charset in FcConfigEvaluate() in the patch as matrix does. now I have the following elements in my .fonts.conf:

        <match target="scan">
                <test name="family">
                        <string>VL Gothic</string>
                </test>
                <edit name="charset" mode="assign">
                        <minus>
                                <name>charset</name>
                                <charset>
                                        <int>0x0040</int>
                                </charset>
                        </minus>
                </edit>
        </match>

and getting a result like:


FcConfigSubstitute test scan any family Equal "VL Gothic"
Substitute match
        scan any family Equal "VL Gothic"
edit
        Edit charset Assign charset Minus charset
;

Append list before 
Append list after  
        0000: 00000000 ffffffff fffffffe 7fffffff 00000000 ffffffff ffffffff fff
fffff

I see a bit of 0x0040 are dropped there. but I'm not sure if no output for "before" are expected behaviour. is it same for matrix?

Hm, nevermind. it looks like same with matrix:


FcConfigSubstitute test pattern any family Equal "AngsanaUPC" Comma "Angsana New
"
Substitute match
        pattern any family Equal "AngsanaUPC" Comma "Angsana New"
edit
        Edit family Assign "Kinnari";
        Edit matrix Assign matrix Times [0.67 0 0 0.67];
...
Append list before 
Append list after  (0.670000 0.000000; 0.000000 0.670000)(w)

(In reply to comment #11)
> Hm, nevermind. it looks like same with matrix:

Then that's a separate bug to be fixed :).

(In reply to comment #12)
> (In reply to comment #11)
> > Hm, nevermind. it looks like same with matrix:
> 
> Then that's a separate bug to be fixed :).

I'm not sure what exactly the bug is, but when running under FcMatchScan, "assign" mode always works as "assign_replace" and no "before" output on "assign_replace" mode because the original pattern is removed prior to invoke FcConfigAdd(). "after" output looks good because it was evaluated prior to invoke FcConfigDel().

additional change to fix a double-free:
http://cgit.freedesktop.org/~tagoh/fontconfig/commit/?h=charset-modification-take2&id=977232deb3afed58b92817b2cae5fc33f3c02bc4

Thanks Akira.  Can you also update the DTD?

For the <range>, we don't need a new allocated type.  Just embed the range struct in FcVStack directly, like we do with 'double'.  And no iterator needed.  No public API change.

The rest looks good.  Thanks!

(In reply to comment #15)
> Thanks Akira.  Can you also update the DTD?
> 
> For the <range>, we don't need a new allocated type.  Just embed the range
> struct in FcVStack directly, like we do with 'double'.  And no iterator needed.
>  No public API change.
> 
> The rest looks good.  Thanks!

Sure. updated:
http://cgit.freedesktop.org/~tagoh/fontconfig/log/?h=charset-modification-take2

Thanks.  Looks great!

Do you think we should also support CDATA for charset, in the format printed out by FcPatternPrint()?  We can always add that later, so I'm happy to go ahead and commit what you've done so far.

(In reply to comment #17)
> Thanks.  Looks great!
> 
> Do you think we should also support CDATA for charset, in the format printed
> out by FcPatternPrint()?  We can always add that later, so I'm happy to go
> ahead and commit what you've done so far.

I'm not sure. but yeah, if there are any strong demand for that, we could go back to work on that later. I'm comfortable with current implementation so far.

Can you also add the support for <range> to the <blank> element please?

(In reply to comment #19)
> Can you also add the support for <range> to the <blank> element please?

Sure. I'll work on that this weekend or Monday because I'm away from now

hmm, I seemed mess up the tree with git-rebase. so merged changes into one on new branch:
http://cgit.freedesktop.org/~tagoh/fontconfig/commit/?h=charset-modification-take3&id=41d3a5dc0f0aa6da80504684dc9d505651620106

Also adding the range support in blank was done:
http://cgit.freedesktop.org/~tagoh/fontconfig/commit/?h=range-support-for-blank&id=0461ef0c867c43eaa6d6524707df00a2cb617e5d


Hope I'm not missing anything there.

Added some document for <range> and <charset>:
http://cgit.freedesktop.org/~tagoh/fontconfig/commit/?h=charset-modification-take3&id=51e352a1bde91348888202539639a5a2d0d506d4

git-rebase for the range support in blank:
http://cgit.freedesktop.org/~tagoh/fontconfig/commit/?h=range-support-for-blank&id=d975cdda782bb88c8bb6706889a554b2afb9f939

Pushed to master.  Please test.  Thanks!

That works for me. thanks!