Bug 110249

Summary: IGT command line tools load redundant GUI libraries
Product: DRI Reporter: Eero Tamminen <eero.t.tamminen>
Component: IGTAssignee: Default DRI bug account <dri-devel>
Status: RESOLVED MOVED QA Contact:
Severity: minor    
Priority: medium    
Version: DRI git   
Hardware: Other   
OS: All   
Whiteboard:
i915 platform: i915 features:

Description Eero Tamminen 2019-03-26 13:11:01 UTC
IGT command line tools load many unnecessary libraries, because they use libigt.so which links in a huge amount of libraries:

$ ldd libigt.so 
	linux-vdso.so.1 (0x00007ffdf24d6000)
	libcairo.so.2 => /usr/lib/x86_64-linux-gnu/libcairo.so.2 (0x00007f8d5daa4000)
	libglib-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f8d5d78e000)
	libdrm.so.2 => /usr/lib/x86_64-linux-gnu/libdrm.so.2 (0x00007f8d5d57d000)
	libdw.so.1 => /usr/lib/x86_64-linux-gnu/libdw.so.1 (0x00007f8d5d331000)
	libkmod.so.2 => /lib/x86_64-linux-gnu/libkmod.so.2 (0x00007f8d5d11a000)
	libprocps.so.6 => /lib/x86_64-linux-gnu/libprocps.so.6 (0x00007f8d5cef6000)
	libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007f8d5ccd8000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8d5c93a000)
	libpciaccess.so.0 => /usr/lib/x86_64-linux-gnu/libpciaccess.so.0 (0x00007f8d5c731000)
	libpixman-1.so.0 => /usr/lib/x86_64-linux-gnu/libpixman-1.so.0 (0x00007f8d5c48c000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f8d5c284000)
	libdrm_intel.so.1 => /usr/lib/x86_64-linux-gnu/libdrm_intel.so.1 (0x00007f8d5c061000)
	libunwind.so.8 => /usr/lib/x86_64-linux-gnu/libunwind.so.8 (0x00007f8d5be46000)
	libgsl.so.23 => /usr/lib/x86_64-linux-gnu/libgsl.so.23 (0x00007f8d5b9e4000)
	libgslcblas.so.0 => /usr/lib/x86_64-linux-gnu/libgslcblas.so.0 (0x00007f8d5b7a5000)
	libasound.so.2 => /usr/lib/x86_64-linux-gnu/libasound.so.2 (0x00007f8d5b49e000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8d5b27f000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8d5ae8e000)
	libfontconfig.so.1 => /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007f8d5ac49000)
	libfreetype.so.6 => /usr/lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007f8d5a995000)
	libpng16.so.16 => /usr/lib/x86_64-linux-gnu/libpng16.so.16 (0x00007f8d5a763000)
	libxcb-shm.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-shm.so.0 (0x00007f8d5a560000)
	libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f8d5a338000)
	libxcb-render.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-render.so.0 (0x00007f8d5a12b000)
	libXrender.so.1 => /usr/lib/x86_64-linux-gnu/libXrender.so.1 (0x00007f8d59f21000)
	libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007f8d59be9000)
	libXext.so.6 => /usr/lib/x86_64-linux-gnu/libXext.so.6 (0x00007f8d599d7000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f8d597ba000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f8d59548000)
	libelf.so.1 => /usr/lib/x86_64-linux-gnu/libelf.so.1 (0x00007f8d5932e000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8d5912a000)
	liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f8d58f04000)
	libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f8d58cf4000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f8d5e035000)
	libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007f8d58a70000)
	libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f8d5883e000)
	libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007f8d5863a000)
	libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f8d58434000)
	liblz4.so.1 => /usr/lib/x86_64-linux-gnu/liblz4.so.1 (0x00007f8d58218000)
	libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007f8d57efd000)
	libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f8d57ce8000)
	libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f8d57ad3000)


For example "intel_gpu_top" seems to need just these symbols from it:
$ readelf -s intel_gpu_top |grep igt
    10: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND igt_perf_open_group
    28: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND igt_perf_open
    58: 00000000002060a0   200 OBJECT  GLOBAL DEFAULT   25 igt_subtest_jmpbuf
   113: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND igt_perf_open_group
   124: 00000000002060a0   200 OBJECT  GLOBAL DEFAULT   25 igt_subtest_jmpbuf
   139: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND igt_perf_open

And sone other functions it needs are some json and printing helpers + normal libc stuff.  It does NOT need freetype, pixman, Cairo, ALSA sound, X11 libraries, libglsl etc stuff brought in by libigt.

It's annoying to need to install these redundant dependencies on e.g. headless media transcoding server if one just wants to use intel_gpu_top to monitor GPU utilization.

Wouldn't it make sense to split libigt up a bit, e.g. to GUI and non-GUI bits?
Comment 1 Eero Tamminen 2019-03-26 13:41:48 UTC
(In reply to Eero Tamminen from comment #0)
...
> X11 libraries, libglsl etc stuff brought in by libigt.

Sorry, that wasn't libglsl, but libgls (GNU Scientific Library).
Comment 2 Emil Velikov 2019-03-26 14:31:10 UTC
Some food for thought:

The old autotools build did not have this problem since it:
 - was creating a static libigt library
 - yet ^^ would lead to slightly longer build times and overall larger bundle
 - ^^ would only pull the required dependencies into the linking process

For reference:
$ objdump -x /usr/bin/intel_gpu_top  | grep -w NEEDED
  NEEDED               libpthread.so.0
  NEEDED               libc.so.6


If it were me I would:
 - static link
 - drop all the extra rpath from the meson build
 - if size is an issue - enable LTO
Comment 3 Chris Wilson 2019-03-26 14:35:38 UTC
The autotools build doesn't have this problem because in there intel-gpu-top is not linked against libigt.la, as nothing outside of tests/ should be. It looks like a bit of carelessness introduced the extra deps when writing the meson.build.
Comment 4 Eero Tamminen 2019-03-26 14:44:36 UTC
(In reply to Chris Wilson from comment #3)
> The autotools build doesn't have this problem because in there intel-gpu-top
> is not linked against libigt.la, as nothing outside of tests/ should be. It
> looks like a bit of carelessness introduced the extra deps when writing the
> meson.build.

Almost all the installed tools link it:

$ for i in intel*; do echo $i; readelf -d $i|grep libigt; done
/opt/install/bin/intel_aubdump
readelf: Error: Not an ELF file - it has the wrong magic bytes at the start
/opt/install/bin/intel_audio_dump
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_backlight
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_bios_dumper
/opt/install/bin/intel_display_crc
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_display_poller
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_dp_compliance
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_dump_decode
/opt/install/bin/intel_dump_gpu
readelf: Error: Not an ELF file - it has the wrong magic bytes at the start
/opt/install/bin/intel_error2aub
/opt/install/bin/intel_error_decode
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_firmware_decode
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_forcewaked
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_framebuffer_dump
/opt/install/bin/intel_gem_info
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel-gen4asm
/opt/install/bin/intel-gen4disasm
/opt/install/bin/intel_gpu_abrt
readelf: Error: Not an ELF file - it has the wrong magic bytes at the start
/opt/install/bin/intel_gpu_frequency
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_gpu_time
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_gpu_top
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_gtt
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_guc_logger
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_gvtg_test
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_infoframes
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_l3_parity
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_lid
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_opregion_decode
/opt/install/bin/intel_panel_fitter
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_perf_counters
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_reg
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_reg_checker
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_residency
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_sanitize_gpu
readelf: Error: Not an ELF file - it has the wrong magic bytes at the start
/opt/install/bin/intel_stepping
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_vbt_decode
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
/opt/install/bin/intel_watermark
 0x0000000000000001 (NEEDED)             Shared library: [libigt.so]
Comment 5 Chris Wilson 2019-03-26 16:05:35 UTC
One down, the rest left to an adventurous sole.

commit a350b9f9f606296b1599c3617c8530a8985709e2 (HEAD, upstream/master)
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date:   Tue Mar 26 13:26:26 2019 +0000

    tools/intel-gpu-top: Only link against igt_perf.la
    
    Fix the meson build to not link intel-gpu-top against the whole test
    infrastructure; it's a tool not a test!
    
    Reported-by: Eero Tamminen <eero.t.tamminen@intel.com>
    Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=110249
    Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
    Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
    Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

But first, it would be wise for Eero to confirm that does the trick,

$ ldd build/tools/intel_gpu_top 
	linux-vdso.so.1 (0x00007ffdf29f9000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f64a15b2000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f64a179d000)
Comment 6 Eero Tamminen 2019-03-26 16:20:10 UTC
intel_gpu_top works fine now.  Large difference in ldd output, and it's still small:
$ size intel_gpu_top 
   text	   data	    bss	    dec	    hex	filename
  21102	   1204	    304	  22610	   5852	/opt/install/bin/intel_gpu_top


For this bug to be considered fixed, I think the rest of tools should be also fixed although I personally care only about the intel_gpu_top...
Comment 7 Eero Tamminen 2019-04-09 11:16:13 UTC
Following tools still link libigt:
-------------------------
for i in intel*; do
  echo $i;
  readelf -d $i | grep libigt;
done 2>/dev/null | awk '
/^intel/ {
  name=$1
}
/NEEDED.*libigt.so/ {
  print "-", name
}
'
------------------------
- intel_audio_dump
- intel_backlight
- intel_display_crc
- intel_display_poller
- intel_dp_compliance
- intel_error_decode
- intel_firmware_decode
- intel_forcewaked
- intel_gem_info
- intel_gpu_frequency
- intel_gpu_time
- intel_gtt
- intel_guc_logger
- intel_gvtg_test
- intel_infoframes
- intel_l3_parity
- intel_lid
- intel_panel_fitter
- intel_perf_counters
- intel_reg
- intel_reg_checker
- intel_residency
- intel_stepping
- intel_vbt_decode
- intel_watermark
Comment 8 Martin Peres 2019-11-12 07:40:02 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/drm/igt-gpu-tools/issues/16.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.