Bug 107098 - Segfault after munmap(kms_sw_dt->ro_mapped)
Summary: Segfault after munmap(kms_sw_dt->ro_mapped)
Alias: None
Product: Mesa
Classification: Unclassified
Component: Other (show other bugs)
Version: 18.0
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: mesa-dev
QA Contact: mesa-dev
Depends on:
Reported: 2018-07-03 08:13 UTC by Jan Palus
Modified: 2018-08-17 16:57 UTC (History)
0 users

See Also:
i915 platform:
i915 features:


Description Jan Palus 2018-07-03 08:13:26 UTC
Recently I've done a big update in my Linux guest running under VirtualBox (Mesa 18.1.2, xorg server 1.20, kernel 4.17) and found X server segfaulting. 

Linux guest inside VirtualBox runs modesetting driver, backed by vboxvideo KMS driver and from what I understand modesetting is trying to use gallium acceleration through swrast (I'm not really fluent in graphic stack so please excuse me if I'm using some terms incorrectly).

After few debugging sessions, in which places of segfault were different depending on glibc version, it appears the problem is caused by line in kms_dri_sw_winsys.c:

munmap(kms_sw_dt->ro_mapped, kms_sw_dt->size);

Judging by code kms_sw_dt->mapped / kms_sw_dt->ro_mapped are expected to be not NULL if they point to mmaped memory or NULL otherwise. Now the problem is that in my case kms_sw_dt->ro_mapped is NULL, so probably munmap() call is expected to be a no-op, however it does not seem to be the case. On the contrary it screws entire process big time, as if it unmapped process memory? All pointers are no longer valid and it seems first dereference afterwards throws segfault. With newer glibc it even occurs within symbol lookup mechanism.

Two things worked for me as a workaround:
* setting AccelMethod=none
* adding condition on above munmap() to call it only if ro_mapped is not NULL

Now I can't really tell if it's ok whether kms_sw_dt->ro_mapped is NULL, or how should munmap() behave when called with NULL. Perhaps there should be a flag whether mmap() was performed instead of relying on pointer value.
Comment 1 Nubosch 2018-07-04 09:06:35 UTC
I can confirm this issue. Using tinydrm display with
weston 4.0.90
linux 4.17.2

KMS-DEBUG: unmapped buffer 2 (was 0x69eaa000)
KMS-DEBUG: unmapped buffer 2 (was (nil))
KMS-DEBUG: destroyed buffer 2

The process shows really weird behavior after this, like missing memory regions.

>* adding condition on above munmap() to call it only if ro_mapped is not NULL
This was the solution I went with too, but added it for both pointers ;)
Comment 2 Jan Palus 2018-07-16 15:15:30 UTC
Patch sent to mailing list, maybe this way it will get any attention:

Comment 3 Jan Palus 2018-08-16 16:28:25 UTC
Note that I ultimately went for "setting AccelMethod=none" since rxvt-unicode lags pretty badly in VirtualBox with glamor.
Comment 4 Emil Velikov 2018-08-17 16:57:04 UTC
Should be fixed with the commit 9baff597ce021f7691187b0d1d1bbc16d07b13e1.
Fix will find it's route to stable in due time.

Thanks for the work everyone!

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.