I'm filing this currently so I have a place to keep notes on this bug.

Running the atom text editor under various OSes (tried Linux Mint 18.3, Ubuntu 18.04, and currently using Fedora 28) results in minor screen glitches, eventually followed by drawing going completely haywire. I recompiled vmwgfx.ko from the current HEAD which resulted in fewer glitches, but it never completely goes away.

The hangs are always immediately preceded by:
[drm:vmw_cmdbuf_work_func [vmwgfx]] *ERROR* Command "(null)" causing device error.
[drm:vmw_cmdbuf_work_func [vmwgfx]] *ERROR* Command buffer offset is 28
[drm:vmw_cmdbuf_work_func [vmwgfx]] *ERROR* Command size is 24

With the caveat that I don't know much about the internals here, both the offset and size seem reasonable here, so I don't think it's a stream-sync issue (client overrunning a buffer, etc.).

The interesting bit is this is resulting in the 'Command "(null)" causing device error' log instead of 'Unknown command causing device error' immediately above.

Peeling through vmw_cmd_describe:
        u32 cmd_id = ((u32 *) buf)[0];

        if (cmd_id >= SVGA_CMD_MAX) {
                SVGA3dCmdHeader *header = (SVGA3dCmdHeader *) buf;
                const struct vmw_cmd_entry *entry;

                *size = header->size + sizeof(SVGA3dCmdHeader);
                cmd_id = header->id;
                if (cmd_id >= SVGA_3D_CMD_MAX)
                        return false;

                cmd_id -= SVGA_3D_CMD_BASE;
                entry = &vmw_cmd_entries[cmd_id];
                *cmd = entry->cmd_name;
                return true;
        }

This appears to indicate the command being issued is falling between SVGA_CMD_MAX (47) and SVGA_3D_CMD_BASE (1040). I'll add some logic here to try to see what's actually being passed.

vmware.log on the host doesn't show anything interesting here.

Platform details:
uname -a: Linux fedora.seattle.kanga.org 4.16.11-300.fc28.x86_64 #1 SMP Tue May 22 18:29:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
VMware® Workstation 14 Pro 14.1.2 build-8497320
Host OS: Windows 10, 64-bit  (Build 15063) 10.0.15063

Guest is allocated 4 virtual cores, 8 GB (host has 8 hyperthreads/4 cores, 16 GB).

Graphics settings:
Accelerate 3D graphics enabled
Use host setting for monitors
768 MB (recommended) graphics memory allocated