Bug 33644

Summary: Fontconfig doesn't match correctly in <test>
Product: fontconfig Reporter: Yichao Zhou <broken.zhou>
Component: libraryAssignee: Akira TAGOH <akira>
Status: RESOLVED FIXED QA Contact: Behdad Esfahbod <freedesktop>
Severity: normal    
Priority: medium CC: akira, fontconfig-bugs, freedesktop
Version: 2_1   
Hardware: All   
OS: All   
Whiteboard:
i915 platform: i915 features:
Attachments: My local.conf
Result of tree

Description Yichao Zhou 2011-01-28 04:12:42 UTC
Created attachment 42636 [details]
My local.conf

There is a line in my local.conf(I only modify it):

        <match target="pattern">
                <test name="family">
                        <string>Courier</string>
                        <string>Courier New</string>
                </test>
                <edit binding="strong" mode="prepend" name="family">
                        <string>monospace</string>
                </edit>
        </match>

$ fc-match "Courier"
DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"

$fc-match "Courier New"
cour.ttf: "Courier New" "Normal"

It seems that the "Courier New" doesn't match correctly.
I can't find the reason.

But after I just swap 2 line:
        <match target="pattern">
                <test name="family">
                        <string>Courier New</string>
                        <string>Courier</string>
                </test>
                <edit binding="strong" mode="prepend" name="family">
                        <string>monospace</string>
                </edit>
        </match>

The result changes!

$ fc-match "Courier New"
DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"

$ fc-match "Courier"
n022003l.pfb: "Nimbus Mono L" "Regular"

The order of string in <test> shouldn't affect the result.
The first line in <test> gets match, but I think the default behavior of test is OR.

I don't know fontconfig very much. So is it a bug?
Comment 1 Yichao Zhou 2011-01-28 04:13:21 UTC
Created attachment 42637 [details]
Result of tree
Comment 2 Behdad Esfahbod 2011-03-14 15:08:33 UTC
According to the docs you can test for one value at a time.  So your config is incorrect.  We should warn in that case though.

From fontconfig-user.html:

<test qual="any" name="property" target="default" compare="eq">

This element contains a single value which is compared with the target ('pattern', 'font', 'scan' or 'default') property "property" (substitute any of the property names seen above). 'compare' can be one of "eq", "not_eq", "less", "less_eq", "more", or "more_eq". 'qual' may either be the default, "any", in which case the match succeeds if any value associated with the property matches the test value, or "all", in which case all of the values associated with the property must match the test value. When used in a <match target="font"> element, the target= attribute in the <test> element selects between matching the original pattern or the font. "default" selects whichever target the outer <match> element has selected.
Comment 3 Akira TAGOH 2011-06-17 01:58:25 UTC
proposed patch to warn.

http://cgit.freedesktop.org/~tagoh/fontconfig/commit/?h=bz33644&id=b803ff89a601035b4a49f01244b4c64a9024226e

should we get rid of "return;" on fail perhaps? is it too strict? and may be better giving aid to the first value for backward compatibility?
Comment 4 Akira TAGOH 2011-06-21 00:19:56 UTC
Behdad, what is the expected syntax to match "Courier" or "Courier New" on the family name FWIW?

I've tried to figure it out with:

<match>
  <test name="family">
    <or>
      <string>Courier New</string>
      <string>Courier</string>
    </or>
  </test>
  ...
</match>

and

<match>
  <test name="family">
    <string>Courier New</string>
    <or>
      <test name="family">
        <string>Courier</string>
      </test>
    </or>
  </test>
  ...
</match>

I didn't get any luck yet. there might be a bug on it too?
Comment 5 Behdad Esfahbod 2011-06-22 08:38:46 UTC
I don't remember.  Either try, or check the code?
Comment 6 Akira TAGOH 2011-06-23 22:36:18 UTC
I did try both. it looks like not supported so far, or I might be misunderstanding of <or> operator.
that's why I'm asking for the correct syntax. that needs to be clarified to get it working...
Comment 7 Akira TAGOH 2011-09-04 19:13:53 UTC
I guess current behavior of the <or> operator isn't what we expect in this issue because it results the boolean value against the evaluation, but not the string as the font name, which we expect here.

I thought we should have a warning if <test> has more font names though, but we may need to consider this again, including the aspect from the documentation bug.

just for updates.
Comment 8 Akira TAGOH 2012-04-02 00:43:42 UTC
thinking about this again, supporting FcOpComma in <test> sounds making sense.
For the case of object == A and object == B:

<test name="object" qual="all">
  <string>A</string>
  <string>B</string>
</test>

For the case of object == A or object == B:

<test name="object" qual="any">
  <string>A</string>
  <string>B</string>
</test>
Comment 9 Akira TAGOH 2012-04-02 01:17:14 UTC
Ah, I think this isn't a bug...

(In reply to comment #0)
> $ fc-match "Courier"
> DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"
> 
> $fc-match "Courier New"
> cour.ttf: "Courier New" "Normal"

This is because you have "Courier New" font and it appears earlier in the pattern. the default qual is "any". as I wrote in comment#8, that is equal to "or". if you have "Courier" in the pattern, fontconfig will apply the rules for the first matched value in the test pattern. thus, the above doesn't work, because fontconfig adds "monospace" prior to "Courier" in the family object. it looks like:

[..., "Courier New", ... "monospace", "Courier", ...] then. so the folowing rule just works:

> But after I just swap 2 line:
>         <match target="pattern">
>                 <test name="family">
>                         <string>Courier New</string>
>                         <string>Courier</string>
>                 </test>
>                 <edit binding="strong" mode="prepend" name="family">
>                         <string>monospace</string>
>                 </edit>
>         </match>
> 
> The result changes!

This really depends on how the family names appears in the pattern.
To do what you want regardless of the order of the family name in the pattern, you may want to try:

<edit name="family" mode="prepend_first" binding="strong">
  <string>monospace</string>
</edit>

BTW I will check the code if test element supports the multiple values. we need to update docs at least.
Comment 10 Akira TAGOH 2012-04-02 01:37:07 UTC
just a note:

This may be a bit relevant to Bug#22862, having a lot of rules instead of using FcOpComma waste the memory. however using "prepend_first" for the workaround of the above issue isn't necessarily a good idea, because it behaves differently.

we may want to have another mode to add the value to where appears earlier in the pattern. another idea is to change the behavior of "prepend", "append", "assign" and "assign_replace" like so for multiple values.
Comment 11 Yichao Zhou 2012-04-02 01:45:08 UTC
Thanks for responding.  But I do not quiet understand what you means.

Rule is
        <match target="pattern">
                <test name="family">
                        <string>Courier</string>
                        <string>Courier New</string>
                </test>
                <edit binding="strong" mode="prepend" name="family">
                        <string>monospace</string>
                </edit>
        </match>

Suppose we do
$ fc-match "Courier New"

The query before the rule apply is [ .. "Courier New" .. ].

Why after the rule applied, monospace is not added before "Courier New"?
Comment 12 Akira TAGOH 2012-04-02 01:56:16 UTC
(In reply to comment #11)
> Thanks for responding.  But I do not quiet understand what you means.
> 
> Rule is
>         <match target="pattern">
>                 <test name="family">
>                         <string>Courier</string>
>                         <string>Courier New</string>
>                 </test>
>                 <edit binding="strong" mode="prepend" name="family">
>                         <string>monospace</string>
>                 </edit>
>         </match>
> 
> Suppose we do
> $ fc-match "Courier New"
> 
> The query before the rule apply is [ .. "Courier New" .. ].
> 
> Why after the rule applied, monospace is not added before "Courier New"?

Because you probably have "Courier" in the pattern and your rule tries to test it first. fontconfig finds it out earlier than "Courier New" and apply the edit pattern to it then. the bad news for you is "Courier" appears after "Courier New". so your pattern after FcConfigSubstitute() should looks like the pseudo array in comment#9.

You can see with FC_DEBUG=4 fc-match ... what happened there.
Comment 13 Yichao Zhou 2012-04-02 01:59:45 UTC
Oh.  That's reasonable.
Comment 14 Behdad Esfahbod 2012-04-02 14:00:59 UTC
(In reply to comment #4)
> Behdad, what is the expected syntax to match "Courier" or "Courier New" on the
> family name FWIW?
> 
> I've tried to figure it out with:
> 
> <match>
>   <test name="family">
>     <or>
>       <string>Courier New</string>
>       <string>Courier</string>
>     </or>
>   </test>
>   ...
> </match>
> 
> and
> 
> <match>
>   <test name="family">
>     <string>Courier New</string>
>     <or>
>       <test name="family">
>         <string>Courier</string>
>       </test>
>     </or>
>   </test>
>   ...
> </match>

How about:

<match>
  <or>
    <test ...>...</test>
    <test ...>...</test>
  </or>
</match>

?
Comment 15 Behdad Esfahbod 2012-04-02 14:02:18 UTC
(In reply to comment #8)
> thinking about this again, supporting FcOpComma in <test> sounds making sense.
> For the case of object == A and object == B:
> 
> <test name="object" qual="all">
>   <string>A</string>
>   <string>B</string>
> </test>
> 
> For the case of object == A or object == B:
> 
> <test name="object" qual="any">
>   <string>A</string>
>   <string>B</string>
> </test>

This would be abusing current semantics.  Currently qual="all" means all values of the elements match the provided value.  You are turning it into something else.  I don't think we should do that.
Comment 16 Akira TAGOH 2012-04-02 19:52:32 UTC
(In reply to comment #14)
> How about:
> 
> <match>
>   <or>
>     <test ...>...</test>
>     <test ...>...</test>
>   </or>
> </match>
> 
> ?

According to the DTD, it's not supposed to work.

(In reply to comment #15)
> This would be abusing current semantics.  Currently qual="all" means all values
> of the elements match the provided value.  You are turning it into something
> else.  I don't think we should do that.

Well, I may be misunderstanding what you are saying. assuming that "elements" means "objects" in the pattern, because you were referring to "provided value" there. I have no idea anything else. if my guess is wrong, sorry in advance - That sounds inconsistent. the document says:

    if 'qual' is 'any', then the match succeeds if any value in the field matche
s.
    if 'qual' is 'all', then the match succeeds only if all values match.
    if 'qual' is 'first', then the match succeeds only if the first value matche
s.
    if 'qual' is 'not_first', then the match succeeds only if any value other th
an

it doesn't talk about all values of the *elements* and other _qual_ doesn't behave that way. given that your explanation is true, that is meaningless with specifying 'name' in qual="all".
In fact, the target object is determined when creating the test object with FcTestCreate(). there are no way to see what else the objects is available after that. and actually "qual" is evaluated every time once a test unit is done. thus qual="all" behaves the AND expression and qual="any" behaves the OR expression in current implementation.
Comment 17 Akira TAGOH 2012-05-01 03:28:14 UTC
(In reply to comment #12)
> Because you probably have "Courier" in the pattern and your rule tries to test
> it first. fontconfig finds it out earlier than "Courier New" and apply the edit
> pattern to it then. the bad news for you is "Courier" appears after "Courier
> New". so your pattern after FcConfigSubstitute() should looks like the pseudo
> array in comment#9.
> 
> You can see with FC_DEBUG=4 fc-match ... what happened there.

For reasonable solution, we could refer the binding to determine which is better to match and edit the pattern.

for example:
FcConfigSubstitute test pattern any family Equal "Helvetica" Comma "Arial"
Substitute match
        pattern any family Equal "Helvetica" Comma "Arial"
edit
        Edit family Prepend "VL Gothic";

Prepend list before  "Arial"(s) "Liberation Sans"(s) "Albany"(s) "Albany AMT"(s)
 "Helvetica"(w) "Nimbus Sans L"(w) "sans-serif"(w)
Prepend list after  "Arial"(s) "Liberation Sans"(s) "Albany"(s) "Albany AMT"(s) 
"VL Gothic"(w) "Helvetica"(w) "Nimbus Sans L"(w) "sans-serif"(w)
FcConfigSubstitute editPattern has 1 elts (size 16)
        family: "Arial"(s) "Liberation Sans"(s) "Albany"(s) "Albany AMT"(s) "VL 
Gothic"(w) "Helvetica"(w) "Nimbus Sans L"(w) "sans-serif"(w)

As I mentioned in the comment#12, this is correct behavior in current implementation, because of the matching order depends on the order of the rule. so adding "VL Gothic" prior to "Helvetica". however what one expects here should be to be added prior to "Arial". so we can check the binding value of "Helvetica" and "Arial" and do the thing for higher priority.
Comment 18 Akira TAGOH 2012-05-01 03:37:58 UTC
though it won't works if other rules gives binding="strong" to Helvetica in this case. so maybe a good idea to output a debug log how fontconfig decide the position to edit the pattern.
Comment 19 Akira TAGOH 2012-05-20 23:13:54 UTC
After some discussion on the list, I think it may be a good idea to warn when multiple values are sets into <test>:

http://cgit.freedesktop.org/~tagoh/fontconfig/commit/?h=bz33644
Comment 20 Akira TAGOH 2012-05-24 21:26:02 UTC
Merged into master. just for references:

For the AND operator:

<match>
  <test name="object".../test>
  <test name="object".../test>
  <edit.../edit>
</match>

For the OR operator:
<match>
  <test name="object".../test>
  <edit.../edit>
</match>
<match>
  <test name="object".../test>
  <edit.../edit>
</match>

the above should works.
Comment 21 Yichao Zhou 2012-05-24 21:52:25 UTC
Thanks!

However, the OR operator isn't very user-friendly and beautiful, since we need to do same <edit> command twice.  Will the feature to simplify this operation be added in the future?
Comment 22 Akira TAGOH 2012-05-24 22:20:56 UTC
(In reply to comment #21)
> Thanks!
> 
> However, the OR operator isn't very user-friendly and beautiful, since we need
> to do same <edit> command twice.  Will the feature to simplify this operation
> be added in the future?

I know that. although I can't promise anything so far but we may have something later if anyone propose the reasonable solution; the problem we are aware of on this matter is, the target selection for <edit> isn't necessarily suitable for everyone when multiple values appears in <test>. current implementation also isn't designed to be used that way.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.