Bug 102986

Summary: mkostemp weak linking on macOS Sierra causes a crash if library is run on 10.11 or older
Product: fontconfig Reporter: Gregorio Litenstein <g.litenstein>
Component: libraryAssignee: fontconfig-bugs
Status: RESOLVED NOTOURBUG QA Contact: Behdad Esfahbod <freedesktop>
Severity: major    
Priority: medium CC: akira
Version: 2.12   
Hardware: x86-64 (AMD64)   
OS: Mac OS X (All)   
Whiteboard:
i915 platform: i915 features:

Description Gregorio Litenstein 2017-09-25 22:50:24 UTC
The MacOS SDK did not include mkostemp until Sierra; building on previous versions works fine because it looks for the function, doesn't find it and it moves on. However, if somebody builds fontconfig against the 10.12 (and presumably also 10.13) SDK, it will be impossible to run on computers running 10.11.6 or older, since the symbol is weak-linked in libSystem.B.dylib.

The fix for this is extremely simple: passing ac_cv_func_mkostemp=no to configure.

I made a patch to fix this for macports (PR: https://github.com/macports/macports-ports/pull/802) but the maintainer of the port asked me to consult here whether you consider this to be an issue that would best be solved on your end or by the various package managers.

I personally am of the mind that this is more of an XCode issue than a fontconfig issue  but that's just me.
Comment 1 Akira TAGOH 2017-09-26 02:11:35 UTC
the binary should be provided per releases because of the sort of this. hm, does ac_cv_func_mkostemp=no works? what if your SDK got rid of mkstemp and so on then? that's why we check them at the build time. this sort of issue has to be addressed by the build time but not the runtime. this isn't our issue.
Comment 2 Gregorio Litenstein 2017-09-26 02:24:56 UTC
Yes, building it like that works as expected.

Trying to convey what the people at macports must have meant: Is it possible to implement what I did for macports on your part? i.e.: before checking for those functions, checking which version of Darwin is running and whether a target where mkostemp will not be available was specifically requested?
Comment 3 Akira TAGOH 2017-09-26 02:40:40 UTC
Hm, are you talking about the cross-compile for older system on new one? that may still has a problem if other libraries' ABI is different between releases. if you want to cross-compile, the same version of libraries which is actually available on the system has to be there to build. your suggestion is totally wrong.
Comment 4 Gregorio Litenstein 2017-09-26 02:47:04 UTC
I'm not really talking about cross-compiling. The current behavior ia the equivalent of running a windows 10 executable on windows 8.1 and getting a crash. That should be (and AFAIK _is_) an exceedingly rare occurrence.

It's not really a matter of differing ABIs at all.
Comment 5 Akira TAGOH 2017-09-26 03:32:50 UTC
> I'm not really talking about cross-compiling. The current behavior ia the
> equivalent of running a windows 10 executable on windows 8.1 and getting a
> crash. That should be (and AFAIK _is_) an exceedingly rare occurrence.

That example makes no sense. they have the proper runtime to get things working. in that sense, you should have libraries at the runtime which contains mkostemp.

See what linux distributions does. they usually have packages per releases, in some cases except when ABI/API isn't changed.

also mkostemp is preferrable more than mkstemp for security reason. I don't know how macports works though, if providing binaries per releases is difficult, you could maintain that patch there.

Anyway, according to what you said, it worked when they had mkostempin the library. they broke the compatibility. then you know something which requires missing pieces will also miss the compatibility unless rebuilding. that is my position on this.
Comment 6 Gregorio Litenstein 2017-09-26 03:47:20 UTC
It actually makes perfect sense but I thnk you don't actually understand what I'm trying to say.
mkostemp is provided not by macports but by a core/standard macOS library present in the SDK, which resides in /usr/lib/libSystem.B.dylib

During building, fontconfig checks where the function exists, if one is using the 10.12 SDK (or newer), it does and it builds under that assumption. Since this function lives in a core library, it is weakly linked, and it definitely makes no sense to want to ship core system libraries (not to mention it would probably be against EULA and/or copyright) with a project.

So, if this version of fontconfig is run on any older version of macOS (like 10.11.6), it will look for mkostemp in libSystem.B.dylib, and since it isn't there, the library crashes.
Comment 7 Akira TAGOH 2017-09-26 04:02:12 UTC
(In reply to Gregorio Litenstein from comment #6)
> It actually makes perfect sense but I thnk you don't actually understand
> what I'm trying to say.
> mkostemp is provided not by macports but by a core/standard macOS library
> present in the SDK, which resides in /usr/lib/libSystem.B.dylib

I know.

> During building, fontconfig checks where the function exists, if one is
> using the 10.12 SDK (or newer), it does and it builds under that assumption.
> Since this function lives in a core library, it is weakly linked, and it
> definitely makes no sense to want to ship core system libraries (not to
> mention it would probably be against EULA and/or copyright) with a project.

I know that too. thus, I'm suggesting you or probably macports should provides binanries per releases which is built against the proper deps of libraries. I didn't hear the reason yet why they don't do that.

> So, if this version of fontconfig is run on any older version of macOS (like
> 10.11.6), it will look for mkostemp in libSystem.B.dylib, and since it isn't
> there, the library crashes.

That is wrong I'm saying. that works only when everything keeps compatibility. in this case, that assumption is already broken.
Comment 8 Gregorio Litenstein 2017-09-26 04:19:03 UTC
> I know that too. thus, I'm suggesting you or probably macports should provides binanries per releases which is built against the proper deps of libraries. I didn't hear the reason yet why they don't do that.

Macports for the most part builds from source and requires a reinstall and rebuild of all packages on every major OS upgrade.

> That is wrong I'm saying. that works only when everything keeps compatibility. in this case, that assumption is already broken.

Adding a function (without removing anything) shouldn't break compatibility, should it?
Comment 9 Akira TAGOH 2017-09-26 04:41:36 UTC
(In reply to Gregorio Litenstein from comment #8)
> Macports for the most part builds from source and requires a reinstall and
> rebuild of all packages on every major OS upgrade.

?? so how come this issue is coming? getting confused. are you sharing binaries built against new SDK with old version of OS X? or who does enforce that stupid thing?

> Adding a function (without removing anything) shouldn't break compatibility,
> should it?

?? No, it doesn't break. and? what was added without removing?
Comment 10 Gregorio Litenstein 2017-09-26 16:21:41 UTC
> ?? so how come this issue is coming? getting 
> confused. are you sharing binaries built against
> new SDK with old version of OS X? or who does 
> enforce that stupid thing?

Oh! This is actually pretty simple. What I'm actually distributing is a game. In Mac, like most of the time in windows, you ship the dependencies along with with an app.

So by my compiling with the latest XCode and SDK, it means nobody running 10.11 or older can run our app. I just submitted the issue (and fix) to macports after investigating the issue a bit.

> ?? No, it doesn't break. and? what was added
> without removing?

I think you've got it backwards. Sierra added mkostemp and I don't think it removed anything. Before that, fontconfig presumably used mkstemp but in Sierra, since it has mkostemp it will use it and compile/link successfully but crash when distributed to a system without the latest OS.
Comment 11 Akira TAGOH 2017-09-27 04:18:19 UTC
let me try to say again, your understanding of the binary compatibility is quite poor. I don't see why you think you could re-use the binary which depends on the incompatible system library. you can do that *only* when the required symbols from libraries isn't missing. in this case, fontconfig binary built against new system library doesn't work with old one *because* mkostemp wasn't available there.

I didn't see how macports was working though, after some conversation with you, they seem expecting to rebuild from the source to use. i.e. they didn't guarantee the binary compatibility between releases. that's true.

An action you can take would be:
1) to provide your app built against the proper deps on both OSes.
2) to stop supporting older OS for your app.
3) to make an wrapper library for mkostemp for old OS

Anyway, this isn't our bug.
Comment 12 Gregorio Litenstein 2017-09-28 02:30:09 UTC
> let me try to say again, your understanding of the binary compatibility is quite
> poor. I don't see why you think you could re-use the binary which depends on the
> incompatible system library. you can do that *only* when the required symbols from
> libraries isn't missing. in this case, fontconfig binary built against new system 
> library doesn't work with old one *because* mkostemp wasn't available there.

No... I understand this, but don't agree with you. I think libraries should be able to adapt their build/configuration settings for different systems when they're things as small as this.

>I didn't see how macports was working though, after some conversation with you, they 
> seem expecting to rebuild from the source to use. i.e. they didn't guarantee the 
> binary compatibility between releases. that's true.

You are always expected to rebuild packages when a new release comes up; most of the time they will still be compatible, but not always. Nonetheless, macports has an option (not by default) to target a specific version of OSX. Generally I target my releases for 10.9 (i.e. try to provide compatibility for as-far-back as possible)

> An action you can take would be:
> 1) to provide your app built against the proper deps on both OSes.
> 2) to stop supporting older OS for your app.
> 3) to make an wrapper library for mkostemp for old OS

1 Makes no sense considering the actual requirement for compatibility is something so small. TBH I already submitted the fix to macports but was more-or-less playing devil's advocate in case you could be persuaded to implement it in your build system.

2 Is not reasonable because it's not like this would break the app on an OS that is 3 or 4 years old, it was literally second-to-latest major update until about a week ago.

3 It's certainly easier to configure fontconfig to ignore mkostemp as it was doing before.

Eventually I have to drop support for older OS, I know that (Until a couple versions ago we supported 10.6) but it seems a bit... excessive to all-of-a-sudden drop support for 3 major releases of the OS.

Anyway... Thanks, I guess.
Comment 13 Akira TAGOH 2017-09-28 03:40:50 UTC
(In reply to Gregorio Litenstein from comment #12)
> No... I understand this, but don't agree with you. I think libraries should
> be able to adapt their build/configuration settings for different systems
> when they're things as small as this.

I do. it is obvious thing as it always works when you build it on the proper env with the proper deps of libraries.

> You are always expected to rebuild packages when a new release comes up;

Yes, because it is safest way to keep compatibility and check it at the build time. this would helps to avoid a trouble at the runtime like what you are facing at this moment.

> most of the time they will still be compatible, but not always. Nonetheless,
> macports has an option (not by default) to target a specific version of OSX.
> Generally I target my releases for 10.9 (i.e. try to provide compatibility
> for as-far-back as possible)

Sure. so the problem should be that your target release is different with the target release on macports then. if you build fontconfig against 10.9 which shouldn't have mkostemp in the system library, it should works fine. so another option would be to build fontconfig yourself and statically link it to avoid using one installed on the users' system and/or statically link everything to avoid such situation.

> 1 Makes no sense considering the actual requirement for compatibility is
> something so small. TBH I already submitted the fix to macports but was
> more-or-less playing devil's advocate in case you could be persuaded to
> implement it in your build system.

unlikely. otherwise I can't use any symbols from libraries to settle deps on various releases of OSes and have to implement everything. our target OS isn't only OS X.
so I do support the compatibility and the portability in the source code level as much as possible.

> 3 It's certainly easier to configure fontconfig to ignore mkostemp as it was
> doing before.

Apparently you don't get what I meant. you better learn how the shared library works.
Comment 14 Ryan Schmidt 2017-10-03 23:42:38 UTC
Hello, I'm the MacPorts maintainer of fontconfig. I've read the previous comments here. Things seem to have gotten a bit off-track.

To clarify, MacPorts has nothing to do with the problem. The user just happened to be using MacPorts when they noticed the problem.

The problem would occur for any user building fontconfig from source using the macOS SDK on macOS 10.12 when the MACOSX_DEPLOYMENT_TARGET environment variable is set to a value of less than 10.12. Apple provides the deployment target capability in their toolchain to allow a user running on one macOS version to build software that can be distributed to users running on earlier versions of macOS.

Gregorio found that overriding the configure check ac_cv_func_mkostemp=no worked around the problem. We can commit that in MacPorts, and that will help MacPorts users who have chosen to build for an earlier deployment target, but it will not help any other fontconfig users who wish to do that. For this reason, I asked Gregorio to submit this bug to see if there is anything you can do in your build system to help all of your users who want to build for a different deployment target. If you don't wish to make any changes to your build system that's fine. I just wanted to make sure that, since the problem is not MacPorts-specific, you were given the opportunity to fix the problem in the most central place.

Note that Apple has been in the habit of releasing new versions of Xcode that are compatible with a version of macOS while no longer containing that OS's SDK. For example, Xcode 8 is compatible with macOS 10.11 but only contains the 10.12 SDK, and Xcode 9 is compatible with macOS 10.12 but only contains the 10.13 SDK. It can therefore be expected that a user with Xcode 8 on macOS 10.11 building using the SDK would encounter the problem even if they have not customized the value of MACOSX_DEPLOYMENT_TARGET.
Comment 15 Alan Coopersmith 2017-10-03 23:55:21 UTC
autotools expects a user who is cross-compiling for a target platform other
than the one they are building on to provide all necessary settings like this,
and strongly discourages adding a bunch of platform specific special cases.

How would we handle this in configure.ac other than hardcoding a check for
MACOSX_DEPLOYMENT_TARGET and having a bunch of non-mac users have to maintain
a list of which Mac versions do and do not support this function instead of
relying on the autotools check?

(Hint: a proposed patch is a lot more understandable than 100 lines of arguing.)

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.