Recently I've done a big update in my Linux guest running under VirtualBox (Mesa 18.1.2, xorg server 1.20, kernel 4.17) and found X server segfaulting.
Linux guest inside VirtualBox runs modesetting driver, backed by vboxvideo KMS driver and from what I understand modesetting is trying to use gallium acceleration through swrast (I'm not really fluent in graphic stack so please excuse me if I'm using some terms incorrectly).
After few debugging sessions, in which places of segfault were different depending on glibc version, it appears the problem is caused by line in kms_dri_sw_winsys.c:
Judging by code kms_sw_dt->mapped / kms_sw_dt->ro_mapped are expected to be not NULL if they point to mmaped memory or NULL otherwise. Now the problem is that in my case kms_sw_dt->ro_mapped is NULL, so probably munmap() call is expected to be a no-op, however it does not seem to be the case. On the contrary it screws entire process big time, as if it unmapped process memory? All pointers are no longer valid and it seems first dereference afterwards throws segfault. With newer glibc it even occurs within symbol lookup mechanism.
Two things worked for me as a workaround:
* setting AccelMethod=none
* adding condition on above munmap() to call it only if ro_mapped is not NULL
Now I can't really tell if it's ok whether kms_sw_dt->ro_mapped is NULL, or how should munmap() behave when called with NULL. Perhaps there should be a flag whether mmap() was performed instead of relying on pointer value.
I can confirm this issue. Using tinydrm display with
KMS-DEBUG: unmapped buffer 2 (was 0x69eaa000)
KMS-DEBUG: unmapped buffer 2 (was (nil))
KMS-DEBUG: destroyed buffer 2
The process shows really weird behavior after this, like missing memory regions.
>* adding condition on above munmap() to call it only if ro_mapped is not NULL
This was the solution I went with too, but added it for both pointers ;)
Patch sent to mailing list, maybe this way it will get any attention:
Note that I ultimately went for "setting AccelMethod=none" since rxvt-unicode lags pretty badly in VirtualBox with glamor.
Should be fixed with the commit 9baff597ce021f7691187b0d1d1bbc16d07b13e1.
Fix will find it's route to stable in due time.
Thanks for the work everyone!