Skip to content
This repository has been archived by the owner on Nov 6, 2019. It is now read-only.

Timeout following results collection #567

Open
jugglinmike opened this issue May 31, 2018 · 0 comments
Open

Timeout following results collection #567

jugglinmike opened this issue May 31, 2018 · 0 comments

Comments

@jugglinmike
Copy link
Collaborator

Today, there was a failure in the process of collecting results for "chunk" 17 of 100 from the Edge browser. Unlike the issue reported in gh-564, this timeout occurred after the WPT process had completed. This process invokes the WPT CLI via a Python wrapper script which validates output and re-tries in the event of invalid/partial data. That wrapper script failed to exit properly. I have not yet identified the reason for this; very little processing occurs following the script's final output ("Found x results" / "Found y unexpected results" / "Found z missing results"), so I'm not sure what would interfere with termination. I will post more information here if this error occurs again.

Final lines of stalled process output (including Buildbot diagnostics)
2018-05-30 17:10:23,315 INFO validate-wpt-results wpt-run:stdout SUITE-END | took 896s
2018-05-30 17:10:23,330 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:23 - Got signal terminated
2018-05-30 17:10:23,339 INFO validate-wpt-results wpt-run:stdout Closing logging queue
2018-05-30 17:10:23,341 INFO validate-wpt-results wpt-run:stdout queue closed
2018-05-30 17:10:23,352 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:23 - Cleaning up.
2018-05-30 17:10:23,353 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:23 - Removing tunnel 5ffa1b0fda3a4861aa0bfd4ad0bf7c43.
2018-05-30 17:10:23,353 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:23 - Waiting for any active jobs using this tunnel to finish.
2018-05-30 17:10:23,353 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:23 - Press CTRL-C again to shut down immediately.
2018-05-30 17:10:23,353 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:23 - Note: if you do this, tests that are still running will fail.
2018-05-30 17:10:23,458 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:23 - Number of jobs using tunnel: 1.
2018-05-30 17:10:23,724 INFO validate-wpt-results WPT CLI exited with return code 1
2018-05-30 17:10:23,726 INFO validate-wpt-results Expected 60 results
2018-05-30 17:10:23,727 INFO validate-wpt-results Found 60 results
2018-05-30 17:10:23,727 INFO validate-wpt-results Found 0 unexpected results
2018-05-30 17:10:23,727 INFO validate-wpt-results Found 0 missing results
2018-05-30 17:10:31,357 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:31 - Number of jobs using tunnel: 1.
2018-05-30 17:10:38,159 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:38 - Number of jobs using tunnel: 1.
2018-05-30 17:10:43,797 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:43 - Number of jobs using tunnel: 1.
2018-05-30 17:10:51,022 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:51 - All jobs using the tunnel have finished.
2018-05-30 17:10:51,022 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:51 - Waiting for the connection to terminate...
2018-05-30 17:10:52,023 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:52 - Connection closed (8).
2018-05-30 17:10:52,023 INFO validate-wpt-results wpt-run:stdout 30 May 17:10:52 - Goodbye.

command timed out: 1200 seconds without output running ['run-and-verify.py', '--max-attempts', '3', '--log-wptreport', '/tmp/tmprBuk46/report.json', '--log-raw', '/tmp/tmprBuk46/log-raw.txt', '--', '--log-tbpl', '-', '--this-chunk', '63', '--total-chunks', '100', '--sauce-platform', 'windows 10', '--sauce-user', 'wpt-daisy', '--sauce-key', '<sauce_labs_key_daisy>', '--sauce-tunnel-id', 'daisy', '--sauce-connect-binary', 'sc', '--no-restart-on-unexpected', '--run-by-dir', '3', 'sauce:MicrosoftEdge:17'], attempting to kill
SIGKILL failed to kill process
using fake rc=-1
program finished with exit code -1

remoteFailed: [Failure instance: Traceback from remote host -- exceptions.RuntimeError: SIGKILL failed to kill process
]
@jugglinmike jugglinmike changed the title WPT Runner timeout following results collection Timeout following results collection May 31, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant