Bug 93856

Summary: piglit summary aggregate problem
Product: piglit Reporter: Tomi Sarvela <tomi.p.sarvela>
Component: infrastructureAssignee: Dylan Baker <baker.dylan.c>
Status: RESOLVED FIXED QA Contact: Piglit Mailing List <piglit>
Severity: normal    
Priority: medium    
Version: unspecified   
Hardware: Other   
OS: All   
Whiteboard:
i915 platform: i915 features:

Description Tomi Sarvela 2016-01-25 15:36:22 UTC
Hello,

I'm trying to create a results.json from incomplete testresults: when hang and reboot happens, I don't want to resume.
It seems that "summary aggregate" should do the right thing, but I'm not getting anything:

# /opt/igt/piglit/piglit summary aggregate -o results.json tests/
Warning: A python exception that should have been handled was not. This is bug and should be reported.
BUG: No backend found for any file in tests

Example of the file I'm trying to feed to piglit:

# cat tests/0.json
{"igt@gem_mmap_gtt@basic-short": {"returncode": 0, "pid": 5652, "result": "pass", "subtests": {"__type__": "Subtests"}, "out": "IGT-Version: 1.13-g9bf4e19 (x86_64) (Linux: 4.5.0-rc1-gfxbench+ x86_64)\nSubtest basic-short: SUCCESS (0.072s)\n", "dmesg": "", "exception": null, "err": "", "traceback": null, "__type__": "TestResult", "environment": "PIGLIT_SOURCE_DIR=\"/opt/igt/piglit\" PIGLIT_PLATFORM=\"mixed_glx_egl\"", "command": "/opt/igt/tests/gem_mmap_gtt --run-subtest basic-short", "time": {"start": 1453728427.467537, "end": 1453728427.567162, "__type__": "TimeAttribute"}}}

Version b3474da8330b8c8a1d0ad5a1fbfc9f3c3a7a1dba xts: Add a rendercheck case for a bug I made in glamor.
Comment 1 Dylan Baker 2016-01-27 20:41:52 UTC
Hmmm, I can't replicate this myself (I'm testing with dEQP and piglit opengl, because I don't really want to have to reboot my system when igt hoses it). You are running an older version of piglit, this may have been fixed already.

Could you try updating to the latest version of piglit, and if that doesn't work could you attach a compressed archive of your tests directory?
Comment 2 Dylan Baker 2016-02-01 22:06:04 UTC
Okay, I have confirmed the bug.

I'll fix it, but in the mean time the problem is that `piglit summary aggregate` expects to be passed a folder containing the 'tests' folder. So if the file heirarchy looks like this:
mytest
-> tests
-> metadata.json

Pointing at 'mytest' will work, but pointing at mytest/tests wont.

So, as a work around point at 'mytest' (or whatever your test name is), not 'tests'.

I'll see about a patch.
Comment 3 Dylan Baker 2016-02-01 22:32:04 UTC
A patch has been posted to the mailing list:
http://patchwork.freedesktop.org/patch/72268/
Comment 4 Tomi Sarvela 2016-02-02 07:55:18 UTC
Thanks for the information, this helped me.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.