Description ======================================= Piglit does not recognize these two tests: igt@drm_mm igt@drv_selftest Versions ====================================== Component: intel-gpu-tools tag: intel-gpu-tools-1.19-96-gfb1ddc4 commit: fb1ddc47003ad6a683db79beeb81b6cbab1feb7c Component: piglit tag: piglit-v1 commit: 56e7e5583cd4a3ca15a8cda154d46d168959dd25 How to reproduce ===================================== Run an IGT execution on a platform. Testslit in use must contain the two problematic tests mentioned in description. After initiating the execution, piglit will fail 1) Run as this command: $ : /home/gfx/intel-graphics/intel-gpu-tools/piglit/piglit run igt -o /home/gfx/iterationT --test-list /home/gfx/dead.testlist -l verbose --sync Traceback (most recent call last): File "/home/gfx/intel-graphics/intel-gpu-tools/piglit/piglit", line 174, in <module> main() File "/home/gfx/intel-graphics/intel-gpu-tools/piglit/piglit", line 170, in main sys.exit(runner(args)) File "/home/gfx/intel-graphics/intel-gpu-tools/piglit/framework/exceptions.py", line 51, in _inner func(*args, **kwargs) File "/home/gfx/intel-graphics/intel-gpu-tools/piglit/framework/programs/run.py", line 357, in run profile.run(profiles, args.log_level, backend, args.concurrency) File "/home/gfx/intel-graphics/intel-gpu-tools/piglit/framework/profile.py", line 387, in run profiles = [(p, list(p.itertests())) for p in profiles] File "/home/gfx/intel-graphics/intel-gpu-tools/piglit/framework/profile.py", line 387, in <listcomp> profiles = [(p, list(p.itertests())) for p in profiles] File "/home/gfx/intel-graphics/intel-gpu-tools/piglit/framework/profile.py", line 317, in itertests opts[n] = self.test_list[n] File "/home/gfx/intel-graphics/intel-gpu-tools/piglit/framework/profile.py", line 161, in __getitem__ return self.__container[key.lower()] KeyError: 'igt@drm_mm' Expected behavior ===================================== Execution should begin normally Current Behavior ===================================== Execution does not begin due to unrecognized tests
The bug is actually due to those tests not respecting "the contract". IGTs are supposed to exit with code 79 if --list-subtests is supplied and we have a binary without any subtests. Those tests used the wrong igt_main macro and the ./bin --list-subtests exited with 0. Yet another thing to add to the distcheck :-) Thanks for reporting, I'll send a patch in a minute.
Actually it's not as straightforward as I thought. Quote form the igt_command_line.sh checker: # Subtest enumeration of kernel selftest launchers depends # on the running kernel. If selftests are not enabled, # they will output nothing and exit with 0. I'll have to figure out how to handle this more gracefully and on which side this special case should be handled, before jumping to a solution. This also means that you are running a testlist which has the whole binary instead of a subtest and/or you ran it on a kernel with selftests disabled.
(In reply to arkadiusz.hiler@intel.com from comment #1) > The bug is actually due to those tests not respecting "the contract". > > IGTs are supposed to exit with code 79 if --list-subtests is supplied and we > have a binary without any subtests. > > Those tests used the wrong igt_main macro and the ./bin --list-subtests > exited with 0. > > Yet another thing to add to the distcheck :-) > > Thanks for reporting, I'll send a patch in a minute. This may help: Actually there're subtests (05:22 AM) [gfx@KBL-3] [tests]$ : sudo ./drm_mm --l sanitycheck init debug reserve insert replace insert_range align align32 align64 evict evict_range bottomup topdown color color_evict color_evict_range (05:23 AM) [gfx@KBL-3] [tests]$ : $? 0: command not found $ : sudo ./drv_selftest --l mock_sanitycheck mock_fence mock_scatterlist mock_syncmap mock_uncore mock_breadcrumbs mock_timelines mock_requests mock_objects mock_dmabuf mock_vma mock_evict mock_gtt live_sanitycheck live_uncore live_requests live_objects live_dmabuf live_coherency live_gtt live_contexts live_hangcheck $ : $? 0: command not found
(In reply to arkadiusz.hiler@intel.com from comment #2) > Actually it's not as straightforward as I thought. > > Quote form the igt_command_line.sh checker: > > # Subtest enumeration of kernel selftest launchers depends > # on the running kernel. If selftests are not enabled, > # they will output nothing and exit with 0. > > I'll have to figure out how to handle this more gracefully and on which side > this special case should be handled, before jumping to a solution. > > This also means that you are running a testlist which has the whole binary > instead of a subtest and/or you ran it on a kernel with selftests disabled. By "you ran it on a kernel with selftests disabled" you refer to the "CONFIG_STATIC_KEYS_SELFTEST" variable in the kernel config file?
Let me explain how it works and what may be the issue. How it works ============ drv_selftest and drm_mm (DRM_DEBUG_MM_SELFTEST and CONFIG_DRM_I915_SELFTEST options accordingly) are thin shims around in-kernel selftests. It just provides convinient way of triggering them. Result of --list-subtests depends on the kernel - it lists all the subtests that are available in the kernel currently running. Piglit executes test binaries with --list-subtests to build internal testlist. If the binary ends with return code 0 then everything listed is added in form igt@binary@subtest. If the return code is 79 then the whole binary is added in the form igt@binary. Note that igt@binary is not in the list if the binary is supposed to have subtests. The issue ========= In your testlist you have igt@drm_mm and igt@drv_selftest and those won't be handled by piglit, as those binaries are in the category of "the ones having subtests". You should revise the way you create your testlist, and list drv_selftest and drm_mm with the subtests (e.g. igt@drm_mm@sanitycheck). I'll take a look into Piglit internals to have some saner way of reporting that the test is "not available" than just blowing up with enigmatic exceptions :-)
(In reply to arkadiusz.hiler@intel.com from comment #5) > Let me explain how it works and what may be the issue. > > > How it works > ============ > > drv_selftest and drm_mm (DRM_DEBUG_MM_SELFTEST and CONFIG_DRM_I915_SELFTEST > options accordingly) are thin shims around in-kernel selftests. It just > provides convinient way of triggering them. > > Result of --list-subtests depends on the kernel - it lists all the subtests > that are available in the kernel currently running. > > Piglit executes test binaries with --list-subtests to build internal > testlist. If the binary ends with return code 0 then everything listed is > added in form igt@binary@subtest. If the return code is 79 then the whole > binary is added in the form igt@binary. > > Note that igt@binary is not in the list if the binary is supposed to have > subtests. > > > The issue > ========= > > In your testlist you have igt@drm_mm and igt@drv_selftest and those won't be > handled by piglit, as those binaries are in the category of "the ones having > subtests". > > You should revise the way you create your testlist, and list drv_selftest > and drm_mm with the subtests (e.g. igt@drm_mm@sanitycheck). > > I'll take a look into Piglit internals to have some saner way of reporting > that the test is "not available" than just blowing up with enigmatic > exceptions :-) Hi arkadiusz thanks for this explication about how to works those tests, after read this comment we noticed that in the server where we compile igt the current kernel did not has the kernel flags that you mention enabled (DRM_DEBUG_MM_SELFTEST and CONFIG_DRM_I915_SELFTEST), so we compiled a new kernel for our server enabling these flags and now we are able to list the subtests from drv_selftest and drm_mm. Thanks a lot closing this bug because we found the fix in our side ;)
Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.